Skip to content
Giallo

Deduplication with Jev AI: connecting records without merging the wrong ones

Use Jev AI to compare candidate pairs in a CRM or catalogue. Separate matching, different and uncertain pairs, with review before every merge.

Reading time: 11 min

Two fictitious business records compared with Jev AI between themselves, different and uncertain, before the review

Jev AI may contribute to deduplication Assessing Two Record Candidates as the same entity, different entities or case to be verified. The merger must remain a next step, prepared by the software and reviewed by an operator before modifying the data.

First the comparison, then the data decision

  • A similarity may propose a couple, but it does not prove identity.
  • Candidate pre-selection should be measured together with the comparison.
  • Same, different and uncertain require distinct activities.
  • The auditor must see the sources and any discordant fields.
  • A proposed merger must indicate what will change before confirmation.

The higher cost can be a false link

A duplicate in CRM can generate two tasks for the same contact and split the history. A wrong merger can instead combine negotiations, contact details and responsibilities of different companies. Before choosing the method, determine which errors are most expensive and which data cannot be solved with a simple record preference.

The Guide on CRM and AI duplicates It addresses the general problem. Here the perimeter is a controlled comparison between pairs, with a review queue. There is no need for the component to have write access to the CRM to evaluate cases: the project can start on a collection of records prepared for testing.

The official cookbook on entity alignment uses evaluations on candidate pairs and distinguishes correspondence, difference and uncertain relationship. The application to a corporate CRM is a design transposition. The vendor demonstration does not certify the identity of its records nor does it constitute a ready-made CRM connector.

The data manager must then define the entity sought. Is it the company, a location, a person, a division or a saleable product? Two cards may relate to the same company and remain separate because they represent operational locations. The model can evaluate a relationship only after the project has clarified what it means itself.

Define identity before similarity

For a company, business name and company name may differ. For a product, description and packaging may change. Text similarity finds commonalities, but the corporate rule states which differences are compatible with an identity and which require keeping distinct records.

Write the rule with border examples. A distributor can treat a one-piece pack and a ten-piece pack as separate items. An archive of products could instead link them to the same family without merging them. They are useful relationships, but have different consequences on availability, price and orders.

In CRM, it also distinguishes between duplicate and relationship. Company, branch and group can be connected. A merge eliminates this distinction if the software retains only one card. When the relationship is the useful data, it prepares an explicit link and leaves the records separate. The overview on Jev AI helps to place judgment in the process without attributing to it the management of the archive.

The table is a proposal to discuss the rules with those who own CRM and catalog. It does not report model tests or a universal policy.

Evidence available What Can It Suggest What Doesn't Prove Alone Useful Check
Very similar name Alias or variation of writing Identity of the company Identification and source checking
Shared address Headquarters or joint organisation Duplicate of single tab Entity type and role of delivery
Coinciding item code Possible same product Same packaging or variant Reference Catalogue and Units
Equivalent description Possible semantic matching Commercial compatibility Specifications that distinguish products
Disagreeing identifiers Possible distinct entity Certain Cause of Discordance Error or update verification

Prepare only couples who deserve attention

Comparing all records with each other grows rapidly. As a simple illustrative calculation, one hundred records produce 100 × 99 / 2 = 4,950 distinct pairs. It is not a measure of one's archive; it shows why the project should separate candidate search and pair evaluation.

Preselection, often called candidate blocking, can use name normalization, available identifiers, product families, or other relevant fields. The rules depend on the archive. Removing spaces and uniforming capitals can help find variations; deleting significant parts of the name or code can instead create improper pairs.

An overly narrow rule misses real matches. An overly broad one produces a queue full of unnecessary comparisons. Therefore, verify candidate selection on known duplicates and difficult cases. If the duplicate does not enter the candidate group, a good Jev assessment of the other pairs does not solve the problem.

It records which rule each couple proposed. The auditor can then recognize if two cards have been approached by a similar name, code or address. When it finds a recurring error, this information indicates which step to correct. It is not necessary to ask for a free explanation to the model to reconstruct a rule applied by the software.

Compare named fields and maintain contradictions

The documentation on state allows you to organize materials in recognizable fields. For comparison, prepare two separate records with relevant source and data. Avoid mixing information into a single text that makes it unclear which tab each value belongs to.

The question must refer to the identity chosen by the company. If you are looking for the same company, a field as a department should not automatically produce a different outcome. If you are looking for the same location, the address can be central. Differences should be evaluated against the role of the fields, not the number of matching strings.

The project must preserve conflicts. An identifier present in both cards but discordant requires a source verification. An absent field is not a discordance: it is an information that is missing. This distinction prevents a poorer card from being judged differently simply because it does not contain the same details.

The Primitive Score can express a position on described levels. In comparison, the column must indicate evidence of difference, uncertainty, and correspondence; an intermediate value is not a percentage of identity. The project decides how to translate the evaluation into a proposal to be reviewed.

Three outcomes, three activities for the auditor

A proposal of the same leads to the control of the merger or link. Different closes the comparison without changing the tabs. Uncertain preserves the case with the information to be verified. None of the three outcomes should result in an automatic cancellation during the pilot.

For a pair proposed as equal, the reviewer sees a preview: records to keep, fields that will change, linked history, and conflicts still open. The software controls the permissions to perform the operation. The judgment of the model is an entry of the procedure; it does not grant permissions or replace the liability on the data.

The Guide to confidence and human review It deepens uncertainty. In the deduplication project, a net proposal must also pass through the review before the merger. Effects can propagate to negotiations, documents, and activities. If the software cannot show what will change, keep the comparison as a suggestion until this part is ready.

Open the check before confirming a merger
  • Do the cards represent the same type of entity as the policy?
  • Are the identifiers and sources consistent or has the conflict been resolved?
  • Does the proposal show which values and links will be changed?
  • Does the auditor have clearance and a procedure to correct an error?

If a check is missing, keep the pair as a proposal to verify. The desire to reduce the number of records is not a sufficient reason to merge them.

Hypothetical scenario: a B2B CRM in Lombardy

A hypothetical Italian SME has two fictitious records named Azienda Esempio Srl and Esempio SRL. One contains recent sales activities; the other contains linked documents and updated contact details. The similarity of the names suggests the pair, but the history is not sufficient to establish that they are duplicates.

The stream presents the two records with the available identifiers and the provenance of the fields. The contact verifies that they represent the same company and not a separate holding subsidiary. If a discrepancy emerges, the pair remains uncertain until a proper source resolves it. No tasks are moved during this comparison.

When the referent confirms the identity, the software prepares the preview. The updated delivery must not automatically prevail on every other field; the commercial history must not disappear because the main card is chosen for completeness. The auditor shall decide according to rules by field and shall only confirm the transaction he has been able to control.

The synchronization between CRM and management software with AI It addresses the problem of connected systems. After a merger, these links must follow a consistent map of the identifiers. The scenario is hypothetical and serves to show that the quality of the comparison is only part of the quality of the operation.

Hypothetical scenario: catalogue of a distributor in Prato

A hypothetical Italian distributor receives catalogues with abbreviated descriptions. Two items have the same trade name, but one indicates a single package and the other a multiple. The images might look the same; in the proposed path the comparison uses the prepared text data and available specifications.

The referent defines that the entity sought is the item that can be sold with its unit and packaging. The two records are therefore to be kept distinct, although they may belong to the same family. A high similarity in the name must not erase the difference that governs quantity and price in management software.

Another pair has different descriptions but the same verified code in the reference source. It is proposed for control. Before any merger, the auditor checks the variant, packaging, and validity of the code, then checks the field preview. If the catalogue does not contain the decisive specification, the case remains uncertain.

To prepare the supplier files, the guide to Document classification with Jev can help to separate up-to-date catalogues, price lists and documents. Deduplication, on the other hand, concerns the entities described within them. A properly classified file may contain products that still require comparison and verification.

A correct pair does not authorize joining an entire group

When multiple records share an address or word in the name, they can form a group of candidates. Don't automatically turn proposals between pairs into a collective merger. The relationship assumed by the model may be imperfect and records may refer to different companies, locations or variants.

If A appears to correspond to B and B to C, the group must also be checked for contradictions between A and C. For example, a generic group card can act as a bridge between two distinct branches. The revision should show the group, the pairs that created the link and the fields that do not match.

Prepare field survival rules before using the merge function. For each information choose the preferred source, the treatment of absences and the fate of alternative values. An old address can remain in the history; a discordant identifier can prevent the operation. The policy must be specific and applied by the software.

Evaluate separately search, comparison and operation

A useful pilot measures three steps: the known duplicates enter among the candidates; the comparison proposes the correct outcome; the auditor understands and confirms a consistent change. It also collects cases without duplicate, because a system that proposes too much can consume time and increase the risk of improper fusion.

It counts the false proposals of the same, the lost correspondences and the uncertain cases solved by the operator. Mark the cause: preselection rule, missing data, semantic evaluation or unclear policy. A total of deleted cards is not a sufficient measure, and in the initial path the comparison does not delete any records.

Data cleanup can make the lead scoring in CRM, because requests and tasks are associated with the correct entities. However, do not attribute future sales or a quantitative improvement to the project before measuring it. Check first the cost of the revision and the effect on daily work.

To link comparison to systems, the article on Jev AI API integration treats the technical flow. Giallo can design the Business Software with preselection, comparison, preview and revision. The result to be made concrete is a controllable proposal, before any change on the CRM or the catalog.

Design with Giallo the deduplication of CRM and catalogs, with sources and revision before merging.

Giallo Studio guides: methodology and sources

How we apply it in the company

Related resources

AI consulting services and process automation

Jev AI in business processes

APIs, Decisions, Data, and Controls: Choose the step that relates to your process.

FAQ

Can Jev AI directly merge duplicates in the CRM?

In the project described, it proposes an assessment of the pair. Merging is the responsibility of the company's software and is prepared only after review by an authorised member of staff. The comparison must not delete or overwrite records.

Do two similar registered company names identify the same company?

Name similarity is useful for finding candidates, but it is not enough to establish identity. Check the available identifiers, sources and contradictions. Locations, divisions or related companies may need relationships other than a merge.

Why preselect pairs before comparing them?

Comparing every record with all the others can generate many irrelevant pairs. Preselection narrows the work using stated rules, but it may also miss real duplicates. It must therefore be verified separately from semantic comparison.

What does an uncertain pair mean?

It means the available data does not support the decision required by the policy. The case stays in a queue with sources and checks still to complete. The pair should not be merged merely to reduce the number of records.

If A corresponds to B and B to C, can all three be merged?

Not solely on the basis of the two proposals. You need to verify the group, contradictions and rules for the entity you want to represent. A shared name or contact detail can link records that must remain distinct.

How do you choose which value to retain when fields disagree?

Prepare rules for each field and show its provenance to the member of staff. The choice may depend on a verified source, update date and type of information. Do not apply a single rule such as 'keep the most complete record' to every field.

Put it into practice

Let's turn this guide into your first live workflow.

Tell us what process you want to lighten: we evaluate feasibility, return and first operational step.