Build enterprise search on traceable document structure

Keep parsing separate from authorization

The search connector should resolve repository permissions and deliver only authorized content to the parsing boundary. DocParse scopes service access by tenant, but it does not replace the source system’s document-level ACL model.

Carry the source ACL version with every derived search record. If permissions change, update or remove indexed chunks even when the document content did not change.

Index fields that improve retrieval and audit

Besides text, index title, heading path, page number, block type, source collection, source modified time, and parser provenance. Preserve stable pointers back to the source system rather than inventing citations from text alone.

Tables may need a dual representation: readable Markdown for keyword and semantic retrieval, plus structured rows for filters or numeric comparison.

• Source and ACL identifiers.
• Heading hierarchy and page range.
• Block types and route warnings.
• Parser, schema, and index versions.

Operate ingestion as a state machine

Track discovered, fetching, parsing, parsed, indexing, active, failed, and deleted states. A source should become searchable only after the entire path succeeds. Retry network and transient parser failures with limits; route permanent format failures to review.

DocParse’s job and event endpoints provide the parsing portion of this state machine without requiring the search product to operate parser containers directly.