Best AI model for extracting data from documents

The extracting data from documents is the most requested AI project by SMEs and the one that seems the simplest. In part it is: on well formatted documents any modern model is doing well.

Difficulty, and risk, lie elsewhere: in bad documents and in the way the model reacts when a data is not there.

The task is more varied than it seems

‘Extracting eight fields from an invoice’ describes two different jobs, and confusing them is the origin of much of the projects they disappoint.

On a well formatted invoice is pure recognition: the data is there, it must be found and reported. An economic model does it well and quickly.

On a crooked scan, with a stamp above the number and a description that ends in the middle of an amount, the task changes nature: one must distinguish between an absent data, an ambiguous one and an unreadable one, and decide what to report.

The Evidence That Separates Everything

There is a single test that distinguishes a usable model from a dangerous one, and it takes two minutes: give the model a document which lacks the required field and ask her.

The last line describes how an extraction project fails without anyone noticing. On high volumes that wrong data gets into business systems, propagates into records and reports, and is discovered months later — when reconstructing what happened costs more than the entire project.

And it's a particularly insidious mistake because a plausible value pass the automatic controls: has the right format, stays in the expected range, does not trigger anything.

The two-level setting

Since documents are naturally divided into ordinary and problematic, the setting that works best uses two models. A structured format must be verified in both type and meaning: extraction of JEV-typed data it delves into missing values and field controls, without replacing reading the document.

The economic model receives all the documents. For each produces the extracted fields and an indication of how safe it is. Ordinary documents close there; those on which it declares uncertainty rise to a more capable model, and what even that solves does not reach a person.

• Define a explicit threshold for level crossing
• Ask the model for quote where he found each field
• Add Coherence checks on extracted values
• Predict a queue for cases that no one closes
• Check documents by sample automatically closed
• Monitor the percentages over time, not just at the beginning

The second point is worth double in the extraction: if the model indicates where in the document the amount was found, the verification takes two seconds. The setting detail is in Mixed Model Strategy.

What band and how much it costs

Economically, the numbers are favorable: at the light band, one hundred thousand documents per month with 1,500 input tokens and 200 outputs cost in the order of a few tens of dollars. It is the kind of project that the economic belt makes possible.

The voice that really weighs is another: the review time. If five percent of documents are to be reviewed by hand, out of a hundred thousand are five thousand revisions per month. At one minute each is over eighty hours: much more than API spending.

Mistakes to Avoid

• Testing only on clean documents.
• Not testing a document without the required field.
• Not asking where it found the data.
• Using a single model for heterogeneous documents.
• Not planning for spot checks on high volumes.
• Optimizing the price list instead of human review.

Practical examples

It is worth following a realistic case, because it shows where the problems really focus.

The starting point. A company receives 8,000 supplier invoices per month, from about four hundred different suppliers. Eight fields to extract. Today the work is done by two people.

First week: One hundred papers with total revision. Every outcome is checked. It turns out that ninety-two per cent of documents are extracted correctly, but also that errors cluster around approximately twenty suppliers with particular formats — they are not distributed randomly.

Second week: a thousand documents with threshold. You introduce the confidence indication and you set the threshold. Seven percent of the documents go up to the second level, two percent go to one person. Percentages are manageable.

Week Three: Full Volume. Weekly sample of twenty documents, automatic consistency checks on amounts, percentage monitoring.

The last line is the real result: from eight thousand documents to look at to one hundred and sixty. And API spending is irrelevant compared to time freed.

Coherence checks, the forgotten ally

On structured data you can do automatic verifications that do not require AI and that intercept most of the residual errors.

• The taxable plus VAT must give the total
• The date must be plausible and consistent with the period
• The supplier must exist in the registry
• The amount must be within a reasonable range for that supplier
• The document number It doesn't have to be there already

They are trivial controls to implement and capture just the most dangerous category of error: the plausible but wrong value. An invented amount passes the format control but almost never the arithmetic one.

It's why a well-done extraction project doesn't just rely on the model: it combines AI to read and deterministic rules to verify. Each does the part where it's strong.

How to set instructions

In extraction the quality of the result depends on the instructions at least as much as on the model. Four tricks are worth more than a sash jump.

Describe each field accurately. "Date" is ambiguous: document date, date of receipt, expiration date? A misdescribed field produces errors that look like model tantrums and are inaccurate instructions instead.

State what to do when the data is missing. It must be said explicitly: if you do not find the field, write that it is not present. Without this instruction many models try to be useful and fill the void.

Give an example of a difficult document. A single example of a boundary case, with the correct answer, improves behavior across the entire category.

Ask for the location of the data. Knowing that the amount was read in the row ‘Total document’ makes the verification immediate and immediately reveals when the model has reconstructed.

When extraction is not the right problem

A final consideration that saves entire projects: sometimes the document should not be read at all.

If a provider sends the same invoice in PDF each month and that provider also has an electronic track, reading the PDF with AI is the most complicated way to get a data that already exists in structured form. The same applies to documents generated by a system that could be accessed directly.

• Check if the data already exists in structured form Somewhere
• Ask the main suppliers if they can send a Path
• Restrict AI extraction to documents that come only as images

In practice almost always there remains a substantial share of documents why AI mining is the only way — that's why these projects make sense. But reducing that quota at the origin is more effective than any downstream optimization.

How to apply it in the company

The prudent path does not start from the full volume. Three stages, and the first two are the cheapest insurance of the project.

One hundred papers with total revision. Every outcome controlled by a person. It serves to measure real accuracy and to find out where errors are concentrated —which in practice are never randomly distributed, but grouped on a few suppliers or a few formats.

A thousand documents with threshold. The indication of confidence is introduced, the threshold of passage to the second level is calibrated and how many cases arrive at a person.

Full volume with continuous controls. Weekly sample, automatic consistency checks, percentage monitoring.

It always joins the AI of the deterministic controls: taxable plus VAT equals total, plausible date, supplier present in the registry, document number not duplicated. They are trivial to implement and capture just the most dangerous category of error —the plausible but wrong value, which passes the format controls. Everyone does the part where he is strong: AI reads, rules verify.

On the economic side, remember that at these rates the biggest item is not the API but the review time. Reducing the percentage of documents to be reviewed is worth more than choosing the model with the lowest list, according to the calculation of cost per task or price per token. The two-level setting is described in Mixed Model Strategy, and the criterion for choosing the starting band in when an economic model is enough.

A final verification that saves entire projects: check if the data already exists in structured form somewhere. Reading a PDF with AI is the most complicated way to get information that an electronic path would provide ready.

If you want to automate the extraction from your documents in a secure and verifiable way, Giallo Studio helps SMEs and teams to build concrete, measurable and sustainable automations.

To complete the reasoning. The method for building the test bench is in how to read AI benchmarks. The calculation of cost per completed task, which is the number on which you really decide, is in when it is best to change the AI model. To understand when it is advisable to question the configuration, the criterion is in because vendor benchmarks are not comparable.

Conclusion

For extraction from documents the economic belt is sufficient on ordinary cases, which are the majority. The difficulty lies in the bad documents, and you deal with it with a second level instead of paying the high end on everything.

But the property that decides if the project is safe is only one: the model must state when the data is not. An invented plausible value passes the controls, enters the company data and is never discovered.

If you want to automate the extraction from your documents in a secure and verifiable way, Giallo Studio designs tailored AI solutions for SMEs.