Mixed model strategy: light and powerful

The question ‘which AI model to choose’ often has a surprising answer: two. An economic model that handles the bulk of traffic and a more capable one that intervenes only on difficult cases.

It is the setting that in practice gives the best relationship between quality and spending, and this guide explains how to build it and what to check for why it really works.

Why it works

The principle is simple and applies in almost every business process: the difficulty of the cases is not evenly distributed.

Out of a thousand invoices, nine hundred are ordinary — known format, fields present, nothing ambiguous. Eighty have some irregularities. Twenty are true cases, which even a person must look at twice.

Using the same model for all a thousand means paying the rate of the twenty difficult cases even for the easy nine hundred. It is the most widespread waste in AI projects, and it is corrected by separating the flows.

Architecture, in concrete

• The first level receives every request
• Produce an outcome and an indication of confidence
• A explicit criterion Decides whether to close or raise
• The second level treats past cases
• A Third Level Human receives what no one closes
• Everything Comes recorded to be able to calibrate

The second point is the one that distinguishes a setting that works from one that seems to work. It is not enough for the model to produce a response: it must also produce a signal about how reliable that response is.

In practice this is achieved by explicitly asking the model to declare uncertainty, and —more reliably still —to quote the point of the document from which it draws each statement. If he can't cite, the case goes up.

The Condition That Decides Everything

There is one way this setting fails, and it is silent: if the light model Securely close cases that he didn't understand, those cases never get to the second level.

The result is worse than not having the second level at all, because the system gives the impression of having a safety net that never actually clicks.

How to define the passing criterion

The criterion should not be decided by intuition: it should be chosen from measurable options and then calibrated to the true data.

Confidence declared. The model indicates how safe it is; below a threshold the case rises. It is the most flexible criterion, but requires verifying that the declared confidence really corresponds to correctness.

Case category. Some types always go up, regardless. It is the most predictable and the easiest criterion to explain to those who have to trust the system.

Characteristics of the document. Length beyond a threshold, unrecognised format, missing fields. It is the easiest criterion to implement.

Outcome of the first attempt. If the model does not close, the case rises. It is the most direct criterion but also the most expensive, because it always pays two steps on difficult cases.

In practice they combine: a rule about critical categories, plus a confidence threshold for everything else.

How much you really save

We take 200,000 cases per month, 1,500 input tokens and 300 outputs. We compare three configurations with an economic model at $0.20/1.20 and one capable at $5/25.

The middle line is the one that everyone would like and that hardly anyone can afford in terms of quality. The third offers the quality of the first on the cases that require it, at one-seventh of the cost.

The full calculation of the real cost, which includes repeated context and failed attempts, is in cost per task or price per token.

The Third Level: People

In almost all business processes a small percentage of cases should not be closed automatically. Designing it from the start is cheaper than adding it after an accident.

The last row of the right column is the one that determines whether the system will be used. If it requires opening another tool or changing habits, adoption collapses within a few weeks — regardless of how good the model is.

Mistakes to Avoid

• Not checking that the lightweight model admits when it does not know.
• Defining the criterion to intuition instead of data.
• Not checking a sample cases closed to the first level.
• Setting the threshold once and never return it.
• Forgetting the third level human.
• Not recording outcomes: without data it does not optimize.

The third point is the security check of the entire setting. It serves to notice if the first level is closing cases that it should have moved on, and it is the only way to discover silent failure before it produces damage.

How to calibrate the threshold

The threshold of passage is not guessed: it is found with the data, in three steps that require a few weeks of operation. If the first level uses a decision-making model, it is necessary to check how to interpret the uncertainty; confidence and review with JEV deepens the choice of thresholds without treating them as a guarantee of correctness.

First: cautious parties. At first it's better to raise more cases than necessary. It costs something more, but it avoids discovering the problem from complaints instead of data.

Second, it records everything. For each case you need the outcome of the first level, the declared confidence, if it has risen and — where available — the correct outcome. Without this data there is no calibration possible.

Third, look at the two mistakes. There are two ways of mistaking thresholds, and they need to be measured separately because they have different costs.

The second line costs much more than the first, and this should guide the calibration: Better to bring up a few too many cases than to let an incorrect one through. The cost of an extra case at the second level is fractions of a cent; the cost of incorrect data entering business systems is orders of magnitude higher.

When the mixture doesn't suit

It's not always the right answer, and it's worth saying to avoid adding complexity where it's not needed.

Low volumes. Under a few thousand cases a month, the difference in spending between configurations is modest and does not justify the work of building and maintaining two levels. Better a single model, chosen well.

Homogenous cases. If all cases are similar, there is no hard minority to separate. Mixed strategy exploits just the inhomogeneity: without that, it has nothing to work on.

High-risk tasks. If each individual case has significant consequences, it is better to use the most capable model directly and focus optimization elsewhere — on the prompt, on the context selection, on the human review.

First project. Building a two-tier system first adds complexity before you understand the problem. Better start simple, measure, and introduce the second level when the data shows that it is needed.

The last line is a tip that applies in general: a simple system that works and is measured is the best basis for any subsequent optimization. The complexity added before having the data is almost always wasted complexity.

Practical examples

Accounting firm, extracting from supplier invoices. Eight thousand documents a month, four hundred suppliers with different formats. The economic model closes about ninety percent; seven percent rise to the second level and two to one person. From eight thousand documents to look at one hundred and sixty, with an API expense in the order of tens of euros.

E-commerce, first response to customers. The lightweight model recognises recurring intents and answers from the document collection. Messages outside the expected pattern — those where the customer describes an ambiguous situation or is angry for a reason not yet explained — are not resolved at all costs: they escalate to the second level and, if necessary, to an operator.

Manufacturing company, classification of a historical archive. Hundreds of thousands of documents to label, without urgency. Here the second level is especially useful for documents that the first one cannot classify with confidence: leaving them unclassified is better than putting them in the wrong category, because a mislabeled document becomes unavailable.

How to apply it in the company

The setting is built in three stages, and the first is the one that protects all the others.

Check the condition. Before anything else, prove that the light model recognizes its own limitations: give it a case that lacks the required information and look at what it does. If it invents a plausible value, the whole setting doesn't hold up, because those cases will never go up to the second level.

Define the pass criterion. Not to intuition but between measurable options: confidence declared below threshold, category of case, irregularities of the document, first attempt that does not close. In practice they combine —a rule on critical categories plus a threshold for the rest.

Tara on data. Start cautiously, raising more cases than necessary, and record everything. After a few weeks you will have the numbers to tighten the threshold. Wrong for excess costs fractions of a cent per case; wrong for defect costs unintercepted errors.

The expected savings calculation, with the numbers of the different bands, is in how much you save with a lightweight model; the criterion to understand which band to use as the first level is in when an economic model is enough.

If you want to design an AI system that uses the right model at the right time, Giallo Studio designs agents and automations tailored to SMEs.

To complete the reasoning. The method for building the test bench is in how to read AI benchmarks. The calculation of cost per completed task, which is the number on which you really decide, is in when it is best to change the AI model. To understand when it is advisable to question the configuration, the criterion is in because vendor benchmarks are not comparable.

Conclusion

Mixed model strategy is the most effective way to reduce the expense of an AI system without losing quality, and takes advantage of a true property of almost all processes: most cases are ordinary.

But it rests on a condition that must be verified before, not after: the light model must recognize its own limits. Without that, the second level does not trigger and the system fails silently.

If you want to design an AI system that uses the right model at the right time, Giallo Studio It builds agents and automations tailored to SMEs.