Text completeness is not semantic correctness
Consider a two-column report. Sorting glyphs only by vertical position alternates left and right columns line by line. Every sentence fragment exists, but no paragraph is coherent. A similar failure in a table attaches a value to the wrong header while preserving all tokens.
Downstream language models may smooth this corruption into a plausible answer, making the error harder to detect. The parser should expose ordered blocks and page coordinates so the sequence can be inspected.
Recover layout with layered evidence
Digital PDFs provide coordinates, fonts, and drawing rules. These signals can identify lines, columns, headings, and table grids deterministically. Scans require OCR boxes and may need layout analysis when the visual structure is complex.
Use the least variable method that solves the cohort. Keep route provenance and warnings, and do not claim a high-confidence table merely because the output is formatted as Markdown.
- Segment page regions before linearizing text.
- Detect columns and establish region order.
- Associate captions and footnotes with nearby objects.
- Retain row, column, and page evidence for tables.
Represent tables for two consumers
Markdown tables are readable and work well for compact model context. Application logic often needs a structured matrix with header relationships and cell coordinates. Keep both when the source provides sufficient evidence.
For very large tables, derive row-group chunks that repeat the table title and header schema. Preserve the complete table artifact so a retrieved row can be verified in context.
Test order explicitly
Create questions whose answer changes if two columns are interleaved or if a value moves across a header. Compare the extracted block sequence with annotated regions, not just the final rendered page.
Track failures by layout class. If one path consistently fails on multi-column scientific papers, route that cohort differently instead of adding cleanup rules to every downstream consumer.
Frequently asked questions
Questions teams ask before building
Why does PDF text come out in the wrong order?
PDF often stores positioned glyphs rather than semantic paragraphs. The parser must infer columns and region order from layout evidence.
Is Markdown enough to represent complex tables?
It is useful for readable text but cannot represent every merged-cell or geometric relationship. Keep structured table and page metadata when those details matter.
Can an LLM repair scrambled reading order?
Sometimes, but the repair can be inconsistent and may invent relationships. Recover order during parsing and retain evidence instead of relying on downstream guessing.