01
Similarity-first retrieval
Image and text embeddings can find pages that resemble a query. In dense, repetitive documents, resemblance is not the same as relevance—and it cannot reliably follow a reference to the evidence it names.
We have taught machines to read documents by first flattening them. A financial statement becomes a stream of characters. A diagram becomes a caption. A table becomes a sequence with its columns, cells, and relationships quietly dissolved. That is useful—until the answer depends on what the page looked like.
01 / THE QUESTION
Do we still need OCR?
Optical Character Recognition has long been the default first step in document question answering. It identifies page elements, recognizes text, and produces something a language model can consume.
Vision-language models change the premise. They can inspect a page and a question together: text, geometry, figures, visual emphasis, and all. The question is no longer whether OCR is useful. It is whether it should always sit between a document and the answer.
If the task is to answer questions about a visual document, why discard visual information before reasoning begins?
02 / THE LOSSY TURN
Flattening is a decision—and a loss.
Classic OCR pipelines convert a two-dimensional page into a one-dimensional representation. Markdown and HTML can retain some structure, but complex tables, merged cells, callouts, diagrams, and spatial relationships remain difficult to carry across intact.
Visual documentlayout · cells · figures · hierarchyfull evidence
→
OCR texttokens arranged in sequencespatial context lost
→
Language modelanswers from what remainsceiling on certainty
A non-reversible transformation: downstream reasoning cannot recover relationships that were never preserved.
03 / A BETTER DEFAULT
Keep the page whole.
An information-preserving system lets a vision-language model reason over the original page image. It sees the table as a table, the footnote attached to its figure, and the value in the correct row and column—not merely a reconstructed string.
This is closer to how people actually read. We do not parse every page into plain text before we search for an answer. We take in typography, structure, imagery, and meaning as one integrated signal.
04 / THE REAL CONSTRAINT
Seeing every page is not reading.
Direct vision reasoning does not make context limits disappear. A 300-page report still cannot be handed wholesale to a model without performance and cost consequences. The critical capability is not extraction alone. It is knowing which pages deserve attention.
02
Reasoning-first navigation
A model uses document structure to infer where an answer belongs, then reads the relevant original pages in full visual context. It investigates instead of guessing.
05 / Coworker
Index the map. Read the evidence.
Coworker creates a model-friendly, hierarchical map of a document—an in-context table of contents that can be examined and navigated. The model reasons over this map, selects a promising section, and sends the corresponding original page image to a vision-language model for detailed inspection.
If the evidence is incomplete, it follows the next relevant branch. If a page says “see Appendix G,” it can navigate there. The loop continues only until the answer is supported.
▾ Annual report [1–220]
▾ Financial statements [72–108]
▸ Consolidated balance sheet [75]
▸ Note 8 — Deferred assets [76–82] ← selected
▸ Appendix G — Statistical tables [195–214]
▾ Financial statements [72–108]
▸ Consolidated balance sheet [75]
▸ Note 8 — Deferred assets [76–82] ← selected
▸ Appendix G — Statistical tables [195–214]
The hierarchy is the navigation layer. Original pages remain the source of truth.
The takeaway
OCR is not obsolete. It is no longer the only door in.
When the answer lives mostly in linear text, OCR remains practical and valuable. But documents are often visual systems: tables encode relationships, layouts communicate priority, and figures carry meaning beyond their labels.
For these documents, Coworker pairs reasoning-based navigation with direct visual understanding—preserving the evidence before asking the model to explain it. Less reconstruction. More understanding.
Coworker / Document intelligence