If “the model will eat it all,” do we still need document processing pipelines?

Entire companies, libraries, and careers have been built around making documents usable by machines. But how much of that work will remain necessary as models become more capable?

A few observations from our class today at ITAM, where we started with a customer survey from our imaginary Lucy and her ice cream shop. Why ice cream? Because I love ice cream. One question in the questionnaire had been printed incorrectly due to an encoding issue. After scanning and parsing the document, we obtained:

XQUX te gustX mks? / What did you like most?

A Spanish speaker can reasonably infer that the intended question was “¿Qué te gustó más?” Even without the English translation, “¿Qué te gusta más?” would be a plausible reconstruction.

The corrupted text leaves some ambiguity about the tense, but the basic meaning seems recoverable. Surely a small local model could handle this?

We tried Ministral 3 running on Ollama, with a simple prompt:

Provide a clean and legible version of this: XQUX te gustX mks?

The response was:

Aquí tienes una versión limpia y legible de tu mensaje:

“¿Te gusta XQUX?”

Si te refieres a algo específico (como un artista, banda, juego, etc.), ¡avísame para ayudarte mejor! 😊

The model treated part of the corrupted text as a name. It produced fluent Spanish, but failed to recover the question.

Perhaps our instructions were too vague. We tried again:

Provide a clean and legible version in Spanish of the following text. Only provide the text and nothing else: XQUX te gustX mks?

The answer:

¿Te gusta XQUX?

Better formatting. Same misunderstanding.

We then gave the same prompt to more capable cloud models. Here is what we observed in these classroom attempts:

Only the last response recovered a plausible version of the intended question.

View full-size figure

This was a small demonstration, not a systematic benchmark. Still, it illustrated something useful: following the requested output format is different from recovering the correct meaning. Every answer looked like a perfectly readable Spanish question. And yet, most were wrong.

Then we tried something that looked much harder.

We used a page from a publicly traded company’s annual report. It contained several pie charts, many numbers, and little explanatory context beyond “Sales by product and market.”

View full-size figure

A few years ago, extracting information from a page like this could have involved substantial engineering: detecting chart regions, reading labels, associating percentages with categories, and reconstructing the results as a table.

We first cropped individual charts and asked Opus to return the country breakdown as a markdown table. And it worked.

Then we removed the cropping step.

We supplied a screenshot of the entire page and asked for the geographic sales breakdown of a particular product. The model had to locate the relevant chart, distinguish it from the others, read its labels, and organize the answer.

Both Opus and Sol handled our example successfully.

That contrast was the most interesting part of the class. A small model struggled to decipher a short sentence that a Spanish speaker could. A stronger model handled a visually dense page that previously might have required a specialized extraction workflow.

The amount of engineering we need depends partly on what the model can already do.

So where does this leave document processing?

It helps to separate two problems.

The first is extracting meaning from a document: reading text, interpreting layout, connecting labels to values, and turning the result into structured data.

The second is making that result dependable enough to use: checking accuracy, preserving sources, handling uncertainty, controlling access, and deciding what happens when something goes wrong.

Our examples suggest that stronger models can absorb more of the first problem. Steps we once implemented separately may become unnecessary for particular tasks.

But successfully reading one page does not establish that a system can reliably process thousands of documents. A correct-looking table still needs checking. A reconstructed survey question is still an inference. In fact, the Spanish example shows why this matters: a model can silently replace corrupted text with a fluent question that means something else.

For Lucy’s survey, a useful system might preserve the original text alongside the proposed correction and flag ambiguous cases for review. The model can suggest what the question meant; the workflow determines whether that suggestion is safe to use.

There are also practical constraints: latency and cost.

In our short decoding exercise, Astra took almost ten seconds to respond. That is noticeable for a single sentence. Across a large collection of documents, response times and differences in model pricing can materially change what is practical.

A pipeline that uses a cheaper model for straightforward cases and escalates difficult ones may therefore make sense even when the strongest model can solve both.

But that trade-off can change.

If a future inexpensive model performs these tasks as well as today’s strongest models, some of the preprocessing and routing we build now may no longer justify its complexity.

This is exciting. It also creates an uncomfortable question for anyone building in this space:

Which parts of our system solve a lasting problem, and which parts compensate for a temporary limitation of the model?

That does not make today’s engineering wasted effort. A pipeline that delivers reliable results at an acceptable cost today has value today. But it does suggest that we should build it so that individual steps can be simplified or removed as capabilities improve.

The same question applies to our skills.

Knowing how to extract information from difficult documents remains useful. So does knowing how to evaluate the extraction, recognize ambiguity, preserve evidence, and connect the result to a real business process.

As models take on more of the extraction work, those judgments may account for more of the value we provide.

Our classroom experiments show how quickly the boundary between model capability and application engineering can move.

And that brings us back to the broader question:

What is the value of code in the age of AI, and what should we learn when a growing share of the work we once programmed explicitly can be performed directly by the model?

What the examples ask us to keep