Your RAG fails at the PDF, not the retriever

When a RAG answer is wrong, the first instinct is to blame retrieval: tune the reranker, swap the embedding model, add hybrid search. Often the fact never made it into the index in the first place. A mangled table, a scrambled two-column page, or a scanned invoice that was never OCR'd means the right sentence does not exist as clean text anywhere in your store. Retrieval cannot find what parsing already destroyed.

• Most RAG failures start at ingestion, not retrieval: if the parser drops a table or scrambles reading order, the answer is gone before you ever embed a chunk.
• Tables, multi-column layouts, merged cells, and scanned pages are where naive text extraction breaks.
• Use layout-aware parsing that preserves headings, reading order, and table structure, then chunk on those boundaries.
• Build a small parse-quality test set and measure extraction before you spend a week tuning retrieval.

Retrieval cannot fix what parsing dropped

A PDF is a visual layout, not a stream of text. Tools like a basic pdf-to-text extractor read the raw text objects in document order, which is not the order a human reads. On a two-column page they interleave the left and right columns line by line, producing sentences that switch topic mid-clause. On a table they flatten rows and columns into a run-on string where "Q1 42 Q2 58" no longer tells you which number belongs to which quarter.

Once that happens, every downstream stage inherits the damage. The embedding is computed over garbled text, the chunk boundary falls in the wrong place, and the chunk that should have held the answer holds noise instead. You can add all the retrieval sophistication you want, including the gains from reducing hallucinations in production RAG, and still return wrong answers, because the source of truth was corrupted at ingestion.

Where naive extraction breaks

Tables

Tables are the single hardest element. Dense financial tables, merged cells, and borderless tables all lose the row-to-column relationship that carries the meaning. A number without its row label and column header is useless to a language model, and worse than useless if the model guesses the label.

Multi-column layouts

Academic papers, datasheets, and many reports use two or three columns. Reading-order-naive parsers zigzag across columns and produce text that no chunking strategy can rescue, because the sentences themselves are interleaved.

Scanned pages and images

A surprising share of enterprise documents are scans with no text layer at all. Without OCR your parser returns empty strings or a handful of stray characters, and those pages silently vanish from your index. Nobody notices until a user asks about the one contract that was a scan.

Headers, footers, and footnotes

Repeated page furniture injects the same header and footer into every chunk, and footnotes get spliced into the middle of sentences. This is low-grade noise that quietly lowers retrieval precision across the whole corpus.

A parsing pipeline that survives production

The fix is to treat the document as a visual artifact and parse it layout-first, then hand clean structured text to the rest of the pipeline.

Start with a layout-aware parser that detects blocks (headings, paragraphs, tables, figures) and establishes true reading order before extracting any text. Convert tables to a structure that preserves rows and columns, such as Markdown or HTML tables, so the row label and column header travel with every cell. Run OCR on any page that has no text layer, and detect that case explicitly rather than trusting that a page is genuinely blank.

Carry metadata through every stage: source document, page number, and section heading. That metadata is what lets you cite a page back to the user and what enforces boundaries in a multi-tenant RAG setup with per-tenant data isolation. Only after parsing is clean do you chunk, and you chunk on the structural boundaries the parser found rather than on a blind character count, following RAG chunking strategies for retrieval quality. From there the enriched text flows into embedding and hybrid search over BM25 and embeddings the same way it would in any production RAG architecture.

Test extraction before you tune retrieval

You cannot improve what you do not measure, and almost nobody measures parsing. Build a small golden set of 30 to 50 real documents that includes your hard cases on purpose: a merged-cell financial table, a dense two-column page, a scanned invoice, a form with checkboxes. For each, write down what a correct extraction looks like.

Then score parser output against it. Did the table survive with rows and columns intact. Is the reading order correct. Are the numbers unchanged. Did the scanned page produce real text. This is the same discipline we push for on the retrieval side in common RAG evaluation mistakes, applied one stage earlier. When you compare parsers, or decide whether a layout-aware service is worth its cost, this test set gives you a number instead of a hunch.

Parsing is unglamorous and it is where most RAG projects quietly lose their accuracy. Spend the first week of any RAG build on ingestion quality, prove it with a golden set, and the retrieval tuning that follows will actually stick.

Frequently asked questions

Why not just use a basic PDF-to-text library?

Basic extractors read raw text objects in storage order, not human reading order. They work on simple single-column prose and fail on tables, multi-column layouts, and scans, which is exactly where the answers your users want tend to live.

How should tables be represented for a language model?

Preserve structure. Convert each table to Markdown or HTML so every cell keeps its row label and column header. A flat string of numbers strips the relationships that make the table meaningful and invites the model to guess.

Do I need OCR if my documents are mostly digital?

You need to detect the exception. Even a mostly-digital corpus usually hides a few scanned pages, and without OCR those pages become invisible gaps in your index. Flag pages with no text layer and route them through OCR rather than assuming they are blank.

How do I know my parsing is good enough?

Build a golden set of 30 to 50 documents that includes your hardest tables, columns, and scans, define the correct extraction for each, and score parser output against it. That number tells you whether to invest in a better parser before you touch retrieval.