Jev AI, confidence and probability: when to get a person to intervene

The confidence of Jev AI helps to choose which decisions to review, but does not equate to accuracy. A useful threshold depends on the action, the cost of an error and the results of your process: the number, alone, does not authorize automation.

Before the number, determine what can happen

Imagine a message asking for assistance with a machine. You can assign it to a department, mark it as urgent, prepare a response or authorize a paid intervention. It's four decisions with different costs. Reuniting them under a generic safety threshold hides the real problem: how much damage causes a wrong choice and who can correct it.

In the Jev AI Guide Find the big picture. Here the next moment is concerned: the model returned a result and the software has to decide whether to use it. The process holder should write a precise sentence for each action. For example: a wrong forwarding corrects itself in a few minutes; a wrong contractual promise can bind the company. The operating rules must reflect this difference.

A sensible path has at least three outputs: execution allowed, review needed, and insufficient information. The third output avoids turning any doubt into a complete manual work. Sometimes the order number is simply missing: asking for that data can solve the case better than a supervisor trying to guess it.

The official pattern of Confidence-driven routing separates the result from the decision to execute it. The thresholds of examples are not universal standards; authorization and action remain in the application. In the project, this means you can change a threshold without granting new permissions to the system.

The calculation that makes the difference visible

The Documentation Confidence defines for Choice the formula (pmax − 1/n) / (1 − 1/n). With three hypothetical options and probabilities 0.8, 0.1 and 0.1, the result is 0.7. It's an instructional calculation, not an observed call. Score uses a different formula: Choice should not be transferred to other results.

The 0.8 value of our example belongs to the most likely option. The value 0.7 is derived from the formula. Neither of them, in this example built at the table, shows how many correct assignments an Italian service office would get. Real or representative messages, reference decisions, and a recorded comparison are needed to find out.

This distinction also counts in the internal interface. An indicator labeled simply “reliability” invites the colleague to interpret it as a guarantee. Better to show the name of the measure, the proposed action and the rule that sent the case under review. The auditor must understand why the system stopped, without being pressured to approve by the color of a number.

For NoulInstead, the field .noul expresses a zero to one probability with respect to a proposition; there is no separate confidence field. A positive question and a reverse one require consistent rules. Probability of yes and intensity of a characteristic are different concepts.

If you ask if a request contains a complaint, a high probability is that proposition. It does not measure how angry the customer is. To design the ticket sorting, keep the category of the message separate, the operational urgency and the need for a person: the three decisions should not automatically inherit the same treatment.

Building a Table of Consequences

The following table is a design proposal. It does not contain vendor-recommended thresholds or measured performance. It serves to make discussions and responsibilities concrete before adding numbers to the software.

In the CRM lead scoring, an evaluation can serve to order the day's work. It should not automatically become a commercial waste. A request placed at the bottom must have a maximum reading time, otherwise an imperfect prediction turns into permanent exclusion.

Revision also comes at a cost. If the rule sends almost everything to a person, the system adds a pass without freeing up capacity. If it sends very little, it can hide errors. A measurable balance must be sought: automated quantity, residual errors and sustainable load for those who control. The threshold is a tool to find that balance, not the goal of the project.

Hypothetical case: a technical assistance in Brianza

Consider a hypothetical Italian SME that installs plants and receives mixed emails: maintenance, invoices, spare parts and commercial requests. The manager starts from sorting, because correcting a department is reversible. It keeps out of the pilot the sending of quotes and promises on the availability of technicians.

Two colleagues classify a sample separately. When they don't agree, they clarify the company rule first. Does a message about a warranty replacement belong to the service or to the warehouse? If the response changes from person to person, the model cannot resolve the absence of an internal convention. The correct sample also becomes a small guide for new operators.

In the pilot, automation proposes the department but does not move the emails yet. The group compares the proposal with the approved allocation, records the differences and distinguishes harmless errors from those that would delay urgent intervention. A single average would lose this information: miscarrying stock and neglecting a stationary plant do not weigh the same.

After the evaluation, the company can activate the shift for part of the cases. Messages without identifiable implantation go into integration; ambiguous ones remain in the queue of the person responsible. A sample of the moved emails is still checked. They are hypothetical process choices: no success percentage was measured by Giallo Studio in this scenario.

The principle is found in the comparison with the Probabilistic ticket decisions: the judgment becomes useful when it has a defined operational path. A dashboard that shows probabilities without indicating who needs to intervene leaves the bottleneck intact.

Hypothetical case: priority for a B2B distributor

A second hypothetical Italian SME distributes industrial components. The salesperson wants to read the relevant requests first, but some short emails come from important customers and do not contain all the details. If the system only rewards information-rich messages, it could penalize those who already know the catalog.

The person responsible separates relevance, completeness and urgency. For completeness a list of data required by the process is enough; for priority it decides which elements really matter. The result feeds a work list, while each contact still receives a reading within an agreed time limit. The automated action remains limited to the ordering.

Cases under review are divided by reason: insufficient description, a request outside the catalogue or conflicting rules. The salesperson should not merely correct a score. They can change the priority and record a concrete reason, such as a customer already being served or compatibility that needs checking. That reason helps establish whether the system is missing a piece of data or the criterion is poorly written.

Here a strategy with different models can serve when a step requires a different job, for example preparing a draft after the choice. The escalation must not, however, become a race between models until the desired result is achieved. If two evaluations diverge, the process must retain the doubt and assign its resolution.

How to choose the threshold without chasing a number

Start with an evaluation set that you don't use to continually rewrite criteria. If you look at each error and edit the question about the same group of examples, you can improve the apparent result without knowing how it will behave on subsequent messages. Keep separate cases useful for design and those intended for final control.

For each candidate threshold, it counts decisions that would be automatic, errors in that group, and cases sent to review. Don't just choose greater automation. Compare the queue load to the available time and check if errors are concentrated in a category. A rare class can deserve a dedicated path even when the overall average looks good.

If the sample is small, write down the absolute number of cases. To say that a category has zero errors on four messages is different from saying it on four hundred. The first result may be useful in identifying an obvious problem, but it is not enough to treat that category as stable. It also avoids comparisons between periods with a very different composition of requests.

The cost of an error can be expressed first in concrete consequences: minutes lost, customer contacted, repeated activity. Turning it into euros is only useful if the company has credible estimates. Inventing a precise amount to make the table look stylish would make the decision less solid.

Open a practice tab for review

Before the decision: show the relevant message, the options allowed and the reason for the review.

During the check: allow approval, correction or request for information, without obliging you to choose an unsuitable category.

After: records the concrete reason for the change and the time needed. A manager periodically reviews the criteria that generate the most disagreements.

Size the queue before activating it

Let's take a hypothetical organizational calculation: forty cases per day under review, ninety seconds per case, mean one hour of work. The calculation does not measure Jev and does not include interruptions, requests for information or practices that require a second colleague. It serves to check whether the person referred to as responsible really possesses that ability in the day.

The average volume is not enough. If the requests arrive especially in a time slot, a queue that can be managed during the day can still delay emergencies. In the project, we propose to record the moment of entry, the beginning of the revision and the closure. The time spent waiting is distinct from the time taken to read the case.

Also define how absences and backlogs are handled. The manager can transfer cases to a substitute, temporarily narrow the automated scope or return to the previous process. They should not raise the threshold randomly to empty the queue, because this would change the risk without an assessment.

An overloaded auditor tends to approve more quickly. To observe this problem, compare the reasons for the corrections and the number of approved cases without modification under different loading conditions. A difference does not prove inattention on its own, but indicates where to delve with the colleague. The system must support its work, not make it a final stamp.

The decision to automate a new category should include this capacity estimate. Even with a few errors, a stream that generates too much revision may not be useful. The expected result is a sustainable path for the office, with someone knowing what to do when the case does not close.

Control continues after startup

The human review queue is not a bottomless container. It must have an assigned person, a handling time and a rule for overdue cases. If an unresolved question remains idle for three days, the process may be worse than the previous routing. The office should be able to suspend automation for one category without stopping the whole workflow.

It also requires monitoring of decisions already made. The review of only doubtful cases measures the work of the auditor, but does not reveal errors passed through the automatic path. A periodic sample, also chosen from the least frequent categories, allows you to see those cases. It retains the distinction between criterion error, missing data and wrong decision.

When a business policy changes or a department is born, repeat a relevant assessment. Keep track of the version of the criteria and configuration used, so you can explain why a September decision differs from an October one. The citation verification requires the same principle when the result concerns the support of a source.

Before using corporate messages, also prepare the Italian Pilot and Data Evaluation. Reviewers need the useful elements, not an indiscriminate copy of the CRM. Designing the human circuit through a AI Consulting It means defining responsibilities and measures before putting a threshold into production.

We build a pilot with criteria, control sample and human review for your process.