[{"data":1,"prerenderedAt":233},["ShallowReactive",2],{"project-36976333":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":35,"aspects":36,"strip":39,"post_text":41,"what":42,"said":95,"maker_in_thread":165,"shape":173,"grid":176,"attempts":194,"same_week":195,"maker_shows":224,"maker_hidden":225,"reader_version":226,"card":227},"36976333","llama2_aided_tesseract","Using LLama2 to Correct OCR Errors","title","Show HN: Using LLama2 to Correct OCR Errors","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fllama2_aided_tesseract","github.com","eigenvalue",1691005989,2023,54,10,1026,3109,34,"notable","ai","ocr error correction","cli",[24,25,26,27,28],"llama2","ocr correction","tesseract","llm","text recognition","curious",-0.167,{"positive":32,"neutral":33,"mixed":32,"critical":34},0,4,1,5,{"idea":37,"performance":38},{"positive":32,"critical":32},{"positive":32,"critical":34},[40,32,40,32,40,32,40,34,32,32],3,"I've been disappointed by the very poor quality of results that I generally get when trying to run OCR on older scanned documents, especially ones that are typewritten or otherwise have unusual or irregular typography. I recently had the idea of using Llama2 to use common sense reasoning and subject level expertise to correct transcription errors in a \"smart\" way-- basically doing what a human proofreader who is familiar with the topic might do.\n\nI came up with the linked script that takes a PDF as input, runs Tesseract on it to get an initial text extraction, and then feeds this sentence-by-s",{"problem":43,"audience":49,"pricing":53,"motivation":54,"time_to_build":61,"stack":65,"openness":92,"stage":93,"team_size":94},{"value":44,"source":45,"evidence":46},"Tesseract OCR output is very poor for old typewritten or irregular scanned documents","maker",[47],{"id":48},"post",{"value":50,"source":45,"evidence":51},"People digitising old scanned documents, such as archives",[52],{"id":48},null,{"value":55,"source":45,"evidence":56},"Existing OCR results on old documents were poor; wanted smarter correction without per-request API costs",[57,58],{"id":48},{"id":59,"by":11,"text":60},"36981331","Cool. I'm sure that would result in better results, but then you have to pay per request and I'm sure it could get pretty expensive if you're talking about long documents. It's…",{"value":62,"source":45,"evidence":63},"A couple of days",[64],{"id":48},[66,72,76,80,84,88],{"value":67,"source":45,"evidence":68},"Llama2 (13B Chat ggml weights)",[69],{"id":70,"by":11,"text":71},"36978552","I’d say that it does work pretty well. It could simply be that I’m sampling too much from the LLM which is causing it to hallucinate more than it should, hence why I needed to…",{"value":73,"source":45,"evidence":74},"Tesseract",[75],{"id":48},{"value":77,"source":45,"evidence":78},"llama-cpp-python",[79],{"id":48},{"value":81,"source":45,"evidence":82},"langchain",[83],{"id":48},{"value":85,"source":45,"evidence":86},"pytesseract",[87],{"id":48},{"value":89,"source":45,"evidence":90},"SQLite",[91],{"id":48},"open_source","prototype","unknown",{"praise":96,"objections":116,"requests":130,"questions":144,"alternatives":156},[97,109],{"value":98,"source":99,"evidence":100},"Idea of LLM-based proofreading of OCR is promising","commenter",[101,105],{"id":102,"by":103,"text":104},"36981178","AaronNewcomer","I’ve been doing something similar recently that impressed me. I have been taking handwritten manuscripts from the early 1800s and feeding them into AWS Textract and then feeding…",{"id":106,"by":107,"text":108},"36978437","version_five","Very cool. I think this is an interesting benchmarking task for a language model (as well as the practical uses). I tried the same thing some time ago, just on a random snippet…",{"value":110,"source":45,"evidence":111},"Local LLM is free to run compared with paid APIs",[112,113],{"id":59,"by":11,"text":60},{"id":114,"by":11,"text":115},"36979376","I just realized that a very similar approach could also be very useful in correcting and reformatting automatic speech recognition transcripts. I made another tool recently for…",[117,122,126],{"value":118,"source":45,"evidence":119},"Llama2 hallucinates new sentences, needing filtering",[120,121],{"id":48},{"id":70,"by":11,"text":71},{"value":123,"source":99,"evidence":124},"Hallucination risk makes it unsuitable without quality control",[125],{"id":70,"by":11,"text":71},{"value":127,"source":45,"evidence":128},"Extremely slow processing",[129],{"id":48},[131,140],{"value":132,"source":99,"evidence":133},"Try fine-tuning Llama2 on synthetic corrupted OCR text",[134,137],{"id":135,"by":107,"text":136},"36979036","You should be able to build a text->image->corrupt with noise->tesseract pipeline to generate some synthetic supervised fine tuning examples (or just try corruption the text, I'm…",{"id":138,"by":107,"text":139},"36979346","As long as you can get tesseract to make similar text errors, the actual corruption doesn't matter though. You'd have to play with it but I'd guess the most important thing is to…",{"value":141,"source":45,"evidence":142},"Parallelise across pages with a Ray cluster",[143],{"id":48},[145,152],{"value":146,"source":99,"evidence":147},"Would different hyperparameters reduce hallucination?",[148],{"id":149,"by":150,"text":151},"36979277","Reubend","Is it possible that with different hyperparameters, LLAMA 2 would be more faithful to the original? I'm pretty surprised that it would hallucinate anything majorly different than…",{"value":153,"source":99,"evidence":154},"Which Llama2 model was used and how well does it perform?",[155],{"id":106,"by":107,"text":108},[157,161],{"value":158,"source":99,"evidence":159},"AWS Textract with GPT-4 or Claude 2 correction",[160],{"id":102,"by":103,"text":104},{"value":162,"source":45,"evidence":163},"ChatGPT for reformatting transcripts",[164],{"id":114,"by":11,"text":115},{"comments":35,"answered":166,"key":167},2,[168,169,170],{"id":59,"text":60},{"id":70,"text":71},{"id":171,"text":172},"36979137","Yeah that's a great idea. Would be a good excuse to learn about fine-tuning Llama2. Although corrupting with simple Gaussian noise might not be \"hard\" enough to help it learn how…",{"commenters":33,"max_depth":35,"maker_replies":35,"first_comment_mins":174,"lifespan_hours":175},137,8,[177,179,181,183,185,186,187,188,190,192],[102,32,40,178,103,32],"project\u002Fneutral\u002Fidea",[59,32,32,180,11,34],"commenter\u002Fneutral\u002Fpricing",[106,34,40,182,107,32],"project\u002Fneutral\u002Fperformance",[70,34,32,184,11,34],"project\u002Fpositive\u002Fperformance",[135,34,40,178,107,32],[171,34,32,178,11,34],[138,34,40,178,107,32],[149,166,34,189,150,32],"project\u002Fcritical\u002Fperformance",[191,166,32,182,11,34],"36979321",[114,40,32,193,11,34],"other\u002Fpositive\u002Fidea",[],[196,201,205,210,215,219],{"id":197,"name":198,"points":199,"category":200},"36933603","San Francisco Compute",727,"infrastructure",{"id":202,"name":203,"points":204,"category":20},"36933452","Khoj",565,{"id":206,"name":207,"points":208,"category":209},"36944152","Markwhen",327,"productivity",{"id":211,"name":212,"points":213,"category":214},"36973400","Pretty Polly",325,"learning_science",{"id":216,"name":217,"points":218,"category":209},"36942308","Linkwarden",280,{"id":220,"name":221,"points":222,"category":223},"36958731","Magic Loops",248,"dev_tools",15,false,"v2",{"id":228,"name":5,"line":6,"ai":225,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":229,"top":18,"kw":230,"maker":11,"domain":10,"tier":19,"attempts":32,"strip":231,"url":9,"mix":232,"praise":98,"objection":118,"sub":21},36976333,-0.17,[24,25,26,27,28],"3030303100",[32,33,32,34],1794036639376]