nlm-ingestor
Data
pdf parsing for rag
SHOW HN2024
How HN took it
9 positive, 9 neutral or asking, 2 mixed, 3 critical, out of 23 comments about the project. Thread score +0.20.
Execution: 2 for, 2 againstIdea: 2 for, 0 againstDocs: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Chunking keeps context together and table parsing is good
From 1 comment
Thanks, I always forget I can do that! I've given it a go and it's really impressive – the default chunker is very smart and manages to keep most of the chunk context together…
mpeg on HN
Fills a need for better RAG document loading
From 2 comments
We've been looking for something exactly like this, thanks for sharing!
xfalcox on HNLast time I tried Langchain (admittedly, that was ~6 months ago) the implementations for content extraction from PDFs and HTML files were very basic. Enough to get a prototype…
rmsaksida on HN
Objections
Docker image does not build natively on Apple silicon
From 1 comment
I couldn't try this tool as it doesn't build on apple silicon (and there's no ARM docker image) However, I have a PDF parsing use-case that I tried those RAG tools for, but the…
mpeg on HN
Hard to evaluate without example PDFs or input/output pairs in the repo
From 2 comments
Do you ave any examples? There doesn't seem to be a single PDF file in the repo.
jvdvegt on HNThank you for sharing. Are there some example input output pairs somewhere?
firtoz on HN
Tesseract OCR makes mistakes
From 1 comment
I am also curious about this. ADI is reliable but does have edge case issues on malformed PDF I fear tesseract OCR is a potential limitation though. I’ve seen it make so many…
cdolan on HN
Requests
Provide example inputs and outputs
From 2 comments
Thank you for sharing. Are there some example input output pairs somewhere?
firtoz on HNDo you ave any examples? There doesn't seem to be a single PDF file in the repo.
jvdvegt on HN
Benchmark comparison against other loaders
From 1 comment
Tesseract OCR fallback sounds great! There are now a lot of file loaders for RAG (langchain, LLMindex, unstructured, ...), any reasons, like a leading benchmark score, to prefer…
lmeyerov on HN
Questions
How does it compare to Azure Document Intelligence
From 1 comment
How does this differ from Azure Document Intelligence, or are they effectively the same thing?
guidedlight on HN
How does it compare to PaddleOCR
From 1 comment
How does this compare to PaddleOCR? Looks like Apache 2 license which is nice.
ilaksh on HN
Alternatives named
llmsherpa
From 1 comment
Thanks for the post. Please use this server with the llmsherpa LayoutPDFReader to get optimal chunks for your LLM/RAG project: https://github.com/nlmatics/llmsherpa. See examples…
asukla on HN
Azure Document Intelligence
From 1 comment
How does this differ from Azure Document Intelligence, or are they effectively the same thing?
guidedlight on HN
PyMuPDF/fitz
From 1 comment
For me, PyMuPDF/fitz has been the best way to retain natural reading order and set dynamic enough rules to extract text in complex layouts. None of the mentioned tools did this…
ramoz on HN
pdfminer.six
From 1 comment
Off-topic, but do you know how Tika compares to other pdf parsing libraries? I was very unimpressed by pdfminer.six (what unstructured uses) as the layout detection seems pretty…
mpeg on HN
Tabula
From 1 comment
Tika uses PDFBox under the hood, using its built-in text extractor (which is "ok"). If you're looking for table extraction specifically, check out Tabula…
jahewson on HN