Jev AI in RAG: choose sources before answering

Jev AI can serve in the RAG at Assessing Recovered Steps before a generative model writes the answer. The filter should be designed as a source decision: it does not replace search, embedding, or permission control.

A document close to the question can lead astray

In a business knowledge base, manuals, operational emails, price lists, and old procedures coexist. The search can find a text with the right words but it describes a different situation. A policy dedicated to retailers, for example, does not necessarily answer an end customer's question. An outdated document can be very similar to the question precisely because it uses the company's usual vocabulary.

The explanation of the RAG helps to read the complete process. The point here is more circumscribed: after finding some candidates, what steps deserve to arrive at the response phase? Separating this choice allows you to evaluate a precise problem. There is no need to attribute the responsibility of the entire chatbot to the filter.

The official cookbook for RAG passes proposes four assessments: relevance, evidence, contradiction and the presence of suspicious instructions. The application separates evidence, conflicts, and excluded materials; a distinct LLM generates the response. It is a design example, not a guarantee that every answer will be correct.

In our project, questions addressed to the filter must correspond to understandable criteria. “Is this a good document?” confuses quality of writing, authority, and usefulness. “Does this step describe the conditions applicable to the request?” Instead, it obliges to clarify which conditions matter. The Operations Manager can check the policy before seeing any results of the model.

Draw the path with separate responsibilities

The recommended order for this project is to: recognize the user, limit the accessible archive, retrieve candidates, check metadata, evaluate steps, compose the response and verify its support. Each step should have a reason to hold or discard a document. Without this separation, a wrong result becomes difficult to diagnose.

The Documentation State distinguishes materials from evaluation questions: a state may contain JSON text or structures. Here we propose separate fields for question, passage and metadata. It is a choice to make the case inspectable, not a ready-to-copy payload API.

The user's demand must be kept in its concrete form. Rewriting it to make it more generic may make text that does not respond to the initial problem seem relevant. If the recovery uses a variant of the question, it records both versions to figure out which one selected the candidate.

The Documentation System One describes Jev as a component that evaluates text input and does not generate conversations. In the stream discussed here, therefore, the composition of the response remains a separate work. The general guide to Jev AI clarifies this location without confusing it with a chat interface.

This is an architecture proposal. The permission column requires rules of the software: it should not be delegated to a judgment on the text. The AI Agents Guardrail they face the very distinction between classifying a risk and preventing an unauthorised transaction.

The four evaluations must not become a single vote

A passage may be relevant and, at the same time, contradict another source. If you add everything into a single score, you risk losing the reason why that text matters. It is preferable to keep the outcomes separate and translate them into a company-determined behavior.

For relevance, the useful question concerns subject matter and scope. A manual from the same product family can talk about a different version. For support, instead, the link between the passage and what is meant to be affirmed is needed. A correct premise does not authorize a broader conclusion. For the conflict, compare the conditions: two different texts are not always incompatible.

The fourth assessment concerns material that attempts to give instructions. In the project, that text is treated as data to be examined, not as an authority on the operation of the system. The access rule and generator discipline must remain independent even when the filter reports nothing.

Then decide the exits: usable material, conflict to show to the manager, candidate to exclude and case to deepen. review of uncertain judgments should be dimensioned on the work of the referent, before activating the filter. There is no need to use these same labels in the customer interface. However, it should be obvious to the internal operator why a response has been stopped. A simple "no result" inscription hides important differences between empty archive and discordant sources.

Hypothetical case: spare parts for a Lombard manufacturer

A hypothetical Italian SME produces machinery and wants to help after-sales respond to the compatibility of spare parts. In the archive there are tabs for similar versions, workshop notes and an updated catalog. Demand indicates the model, but not the year of production.

The recovery finds two tabs close by. The project manager does not ask the filter to guess which one is right. Set a rule first: For some families, compatibility also requires the revision code. When it is missing, the answer must ask for that code. The filter can select the steps that explain the condition, while the final choice remains suspended.

For complete cases, the valid catalogue has a different role from an informal note. The company assigns documents a field of authority and a relationship with the product. This work belongs to the Classification of documents: if each file is just an anonymous text, the filter receives a less defined problem.

In the pilot, the wrong couplings between versions are mainly controlled. A very relevant but applicable text to the previous model is a more interesting error than a patently unrelated result. The aftermarket compares the steps chosen with those that a technician would use. The outcomes remain hypothetical: we are not reporting a customer test or a measured percentage.

Another useful measure is the number of appropriate integration requests. If the system always asks for a code even when the question already contains everything, the flow slows down. If he does not ask in incomplete cases, he prepares a fragile answer. Quality is seen in final behavior, not just the number of discarded documents.

Hypothetical case: commercial conditions of a distributor

A second hypothetical Italian SME distributes materials for installers. The knowledge base contains standard conditions, exceptions for some contracts and an old procedure on delivery charges. An operator asks if an order can be shipped at no additional cost.

The search retrieves a text that speaks of free shipping, but the clause concerns a different geographical area. The filter must be able to report that the topic is relevant without considering it sufficient to respond. The deterministic metadata rule verifies area and date; the pass evaluation looks for any additional conditions in the text.

If two applicable procedures do not agree, the project shows the contrast to the commercial office. The manager updates the policy or clarifies which one prevails. It is not enough to choose the most recent source when the oldest represents a still valid contractual exception. The authority of the document is a decision of the company, not a property derivable from its linguistic fluidity.

The prepared response uses the authorized sources and indicates the relevant conditions. Before external use, the citation verification check if those conditions really support the claim. The two controls complete each other: choosing a good pass before writing does not prevent the writing from extending its meaning.

Recover well before filtering a lot

A filter can only evaluate what it receives. If the correct procedure does not enter the whole of the candidates, it cannot be chosen. For this reason the comparison between recovery and filtering must be done on two levels: availability of the useful source and selection from the available sources.

In the guide to embedding and PDFs The theme is the representation of content. Here the operational point is to preserve the provenance of the passages. If a clause is split between two extracts, an isolated fragment may lose its limit. The size of the passage should be chosen with real examples, looking at where the conditions begin and end.

Do not turn this issue into a request to always send entire documents. A long procedure also contains non-relevant sections. Prepare extracts with title, clause and context close, keeping reference to the original file. When it takes a previous paragraph to interpret an exception, include it consciously.

The number of candidates is a trade-off. Evaluating many increases the workflow’s workload; evaluating few may leave out the correct source. In the pilot, we suggest testing several configurations on the same set of questions, recording both quality and time. This article does not provide a measured optimal number valid for all SMEs.

Open the tab of a step to evaluate

Question: the concrete problem and the data already available.

Source: document, version, scope, owner and precise point from which the extract comes.

Outcome: useful, insufficient, incompatible or to be deepened, according to written business criteria.

Next action: use the step, look for a further source or involve the referent.

Retain the reason for exclusion without duplicating the archive

When a passage is discarded, the project should retain an understandable motivation. "Not relevant" helps little if the real problem was the product version. A distinction between incorrect scope, outdated source, insufficient content, and conflict allows the knowledge base owner to intervene at the correct point.

This register must not become a second complete copy of the documents. It may contain identification, version, pattern and reference to the case. The auditor opens the text in the authorized environment when needed. The choice of information to be retained must be linked to the accesses and the duration of the register.

We also propose a reading test for the operator: he receives a question, the passages withheld and the reasons for the exclusions, then indicates if he can reconstruct the decision. If it has to open ten systems or infer which version was used, the explanation of the filter is not yet useful. This test concerns the interface of the work, in addition to the classification.

A recurrently excluded document may have two different fates. If it is valid but recovered out of scope, metadata or search needs to be improved. If it is ambiguous and confusing even for technicians, it is necessary to rewrite the content. It is not advisable to delete it just to make the filter metrics look better: it could be useful in other cases.

The group may periodically examine some exclusions and admissions. He looks especially at cases where the reason does not correspond to the judgment of the referent. This review allows you to improve the process without turning every single response into a long manual survey.

Evaluate the filter against a baseline

Prepare a set of questions that represent the work of the office: frequent cases, exceptions, missing information and conflicting documents. For each application, a contact person shall indicate the necessary steps and conclusions allowed. Don't just score which answer "sounds better." It is necessary to be able to explain which condition has been respected or forgotten.

Compare a stream without the new filter and one that uses it, keeping the other steps equal. It counts the useful evidence eliminated, the misleading materials withheld, the conflicts made visible, and the responses that should have stopped. It also observes the added time and work required of operators. None of these measures should be replaced by the number of citations present.

When a result worsens, identify the stage responsible. Absent source in recovery, wrong metadata, too restrictive filter and too wide response require different corrections. Adding a generic question to the model does not automatically solve the four causes.

The knowledge base owner must also receive the recurring gaps. If ten similar requests each time require a person because a procedure is missing, the solution may be to write that procedure. The RAG makes an organizational need visible: it should not mask it with plausible responses.

To arrive at a useful system, a AI Consulting can start from sources, permissions and sample questions. The decision to introduce Jev should be made after the comparison, when it is clear which error corrects and what cost it adds.

We design the filter of your knowledge base and measure the result on the real work of the office.