Why plain PDF text extraction breaks downstream
PDF stores presentation instructions more reliably than semantic reading order. A naive extractor can interleave two columns, detach table cells, repeat headers, and flatten a heading into the paragraph below it. The text may look complete by character count while producing poor chunks and misleading retrieval.
A useful PDF-to-Markdown stage reconstructs a readable hierarchy and keeps a trail back to the page. DocParse treats Markdown as one output view, not the entire record. DocIR carries typed blocks and positional context so applications can chunk by heading, cite a page, filter headers, or re-render a table without reparsing the original file.
Route by evidence, not one expensive default
Digital PDFs with a sufficient embedded text layer take a deterministic container route. That path avoids model variance while retaining fonts, blocks, table evidence, reading order, and bounding boxes. Suspected scans and complex pages can use OCR or layout vision when the workspace enables those capabilities.
The route is visible in the result. This matters for evaluation: teams can segment accuracy by document class and parser rather than averaging clean reports with degraded scans. It also creates a cost lever—model-backed work is reserved for documents whose structure justifies it.
- Digital reports: deterministic text and layout recovery.
- Image-only pages: OCR after page rendering.
- Complex tables or formulas: approved layout-aware processing.
- Failures: explicit warnings and terminal errors instead of silent empty output.
Design the Markdown for retrieval
Good Markdown preserves semantic separators. Headings should delimit sections, lists should remain lists, and tables should not be dissolved into comma-separated prose. Before embedding, remove repeated furniture only when the evidence is strong, retain page metadata separately, and choose chunk boundaries that follow document structure rather than a fixed character window.
Use Markdown as the model-facing layer and retain DocIR as the audit layer. If a cited answer is wrong, the application can inspect the source page, block type, route, and warning metadata. That feedback is much more actionable than debugging an opaque string.
Frequently asked questions
Questions teams ask before building
Can DocParse convert scanned PDFs to Markdown?
Yes, scanned PDFs can be routed through OCR or approved layout-aware processing. The free workspace validates deterministic text-based documents; model-backed scan processing requires an approved workspace.
Does PDF-to-Markdown preserve tables?
The parser attempts to reconstruct tables in readable Markdown and retains structured block information so downstream code can identify and inspect table regions.
Why return JSON in addition to Markdown?
Markdown is ideal for reading and chunking, while DocIR JSON keeps pages, block types, order, boxes, warnings, and provenance needed for product logic and debugging.