Entity resolution software for fraud: three names, one caller

Entity resolution software finds the records that refer to the same real person, address, phone or business even when they are written differently, scores each candidate match, merges the confident ones and sends the uncertain ones to a person. In insurance fraud work it is the step that decides whether a ring is visible at all: a caller who gives three different names, a clinic that changes its trading name, a passenger whose surname is misspelled on the intake form. Botminds' Fraud Network Analysis, a link analysis solution for special investigations units, has an entity resolver as one of its six agents; it compares names, dates of birth, phones, addresses and voice-line IDs, merges on its own only above a confidence the fraud analyst sets, and sends the middle band to the analyst. This guide uses it as the worked example, alongside the record linkage research that the method comes from.

The records below come from a fictional sample carrier, Keywater Insurance Company, and its South Florida SIU; every name, phone and claim is invented. The research is the U.S. Census Bureau's own work on record linkage, linked where it appears.

What is entity resolution software, and what is record linkage?

Entity resolution and record linkage are two names for the same discipline: deciding which records describe the same thing. The Census Bureau, which built much of the method to count the population, defines it in one line: "Record linkage is the means of combining information from a variety of computerized files" (Winkler, Census Bureau, 2006). Software for it does three things: standardizes the raw values, compares candidate pairs, and decides.

The vocabulary varies by industry. Marketers say identity resolution, data teams say master data management or deduplication, statisticians say record linkage. The test is the same. Given a phone on a call log, a name on a clinic form and an address on a PIP application, does the software know they belong to one caller, and can it say why?

What is entity resolution in fraud detection?

In fraud detection, entity resolution is the step that defeats deliberate variation. Honest data is messy by accident; a ring's data is messy on purpose. A federal prosecutor's 2011 release on New York no-fault clinics, announcing charges, alleged that the defendants "changed the names or locations of the Clinics, at various times, to avoid detection of the fraud by insurance companies or law enforcement" (FBI New York, 2011). In Florida's Operation Sledgehammer prosecutions, the true owners of chiropractic clinics put licensed chiropractors forward as nominal owners, so the name on the paperwork was never the person in control (U.S. Attorney, S.D. Fla.).

That changes what good matching means. A customer-data tool is tuned to avoid annoying a customer with a duplicate mailing. A fraud tool must catch a caller who uses an initial one month and a nickname the next, without merging two cousins who share a surname and an apartment. Insurers' own descriptions show the trade-off: one carrier told NAIC surveyors that "Address is used to identify possible personal relationships in fraud ring detection" (NAIC private passenger auto AI/ML survey, 2022). An address can reveal a relationship, and it can just as easily be a forty-unit building.

How does entity resolution work? Block, compare, score, decide

Entity resolution works in four steps: narrow the pairs worth comparing, compare each field with a measure that tolerates errors, combine the comparisons into one score, and decide with two thresholds rather than one. The method was formalized by Fellegi and Sunter in 1969 and is still the backbone of most matching software.

Blocking is the step buyers ask least about and should ask most. The Census Bureau describes it as considering only "pairs that agreed on a characteristic such as surname or date-of-birth", and reports that classical matching might run ten blocking passes over a pair of files (Winkler, 2006). Each pass is a choice about which true matches you are willing to miss. A system that blocks only on surname will never compare "J. Ruiz" with "Jay"; one that also blocks on the callback number will.

The score is a likelihood ratio: how much more often this pattern of agreements appears among true matches than among non-matches. Fellegi and Sunter's rule then sorts every pair into match, non-match or "possible match and hold for clerical review". That middle region is not a weakness of the method; it is the method telling you where a person is needed.

Can entity resolution match misspelled names? Yes, up to a point

Good entity resolution tolerates misspellings with string comparators, functions that score partial agreement between 0 and 1. Without them, matching on names fails far more often than most buyers expect. In one major census application, the Census Bureau found that "more than 25% of matches would not have been found via exact character-by-character matching" (Winkler, 2006).

The same report shows the limit. Its table of comparators scores JONES against JOHNSON at 0.832 on the Jaro-Winkler measure, high enough to look like a match, though they are different surnames. And no spelling measure joins "Jay" to "Javier": the letters barely overlap. That is why name similarity can only ever be one input. In the sample ring the match is carried by the phone and the context of the calls; the names only fail to contradict it.

The errors compound. The Census Bureau worked an example: if first names carry a 10% typographical error rate and last name and each part of the date of birth carry 5%, "then exact character-by-character matching across quarters could miss 25% of matches" (Winkler, 2006). Claim files, keyed by different people from phone calls, handwritten intake forms and scanned letters, are rarely cleaner than the Census Bureau's own forms.

What is the clerical review region, and who works it?

The clerical review region is the band of scores between the two thresholds, where the software is not confident enough to merge or to separate, and a person decides. The Census Bureau calls it "the no-decision region or clerical review region" (Winkler, 2006). In an SIU, the clerk is the fraud analyst.

What lands in the band is telling. When the Census Bureau matched its 1990 post-enumeration survey, "The clerical review region of possible matches consisted almost entirely of individuals within the same household who were missing first name and age" (Winkler, Census Bureau, 2018). Fraud data has the same shape. The hard pairs are relatives at one address with similar names, and the cost of a wrong call is high either way: merge them and a household becomes a ring, separate a real alias and the ring breaks in two. In the sample unit, one group of claims the ring finder drew was cleared with a recorded reason, a household rather than a ring, and the clearance stays on record with the reason beside it.

Every identifier shown in its document, with what it links to

How accurate is entity resolution?

Entity resolution can be very accurate, but only measured against reviewed pairs on your own data, and accuracy has two sides: false matches and missed matches. The Census Bureau's record makes the scale of the gain clear. For the 1990 survey, Hogan and Wolter had estimated that "manual matching would require 3000 individuals for six months with a false match error rate of a 5%"; the computerized matching was done in six weeks with about 200 people for clerical review and field follow-up, at a false match rate "of at most 0.2%" (Winkler, 2018).

For an SIU, two numbers matter and both can be measured in a pilot. The first is the false merge rate among automatic merges: draw a few hundred merged pairs above the threshold and have an analyst check them against the documents. The second is the miss rate: take rings your investigators already proved and count how many of their aliases the software joined without help. A vendor quoting one accuracy figure without saying which of the two it is, on whose data, has not answered the question.

What does a false merge cost in a fraud investigation?

A false merge puts an innocent person inside a ring; a missed merge splits a ring so it never crosses the threshold. Both are expensive, but only the first can harm someone who did nothing. If two different people become one node, every claim each touched now looks connected, and an investigator may interview, examine under oath or refer someone because of a typing coincidence.

That is why some rules should be absolute rather than scored. In Fraud Network Analysis, two records with different dates of birth never merge, whatever the other fields say; a father and son with the same name and address are exactly the pair a score gets wrong. The law-enforcement analytic standards make the same demand of any analytic product: "Dubious statements and sources must be noted so the person making a judgment is able to decide the weight or validity to ascribe to the statement" (Law Enforcement Analytic Standards, 2nd edition, 2012). A merge is a statement. It needs its sources and its confidence on the page.

Choosing entity resolution software for an SIU: nine tests

Choose entity resolution software for fraud work by testing it on your own closed rings, not on a demo file. The nine tests below can be run in a few weeks with a sample of claim files and two analysts.

Two of these tests are where tools differ most. Reading documents is the first: most of the identifiers that tie a ring together never reach a structured field, which is the subject of the eight documents a link analysis tool needs. Handling busy shared values is the second, and it is the same problem as hubs in a link map, covered in five tests a detected ring must pass.

Where Fraud Network Analysis fits, and where it does not

Fraud Network Analysis is not general-purpose entity resolution software. Its resolver works inside the claim files an SIU already holds, to join the people, phones, addresses, providers, cards and accounts that matter for rings, and it hands its work straight to the link map, the ring finder and the investigator's lead. If you need one customer record across policy administration, billing and marketing, that is a master data project, and a fraud team should not own it.

Where it fits is the gap between the two. The SIU's question is narrower and harder: is the "J. Ruiz" on a rehab center intake form the "Jay" who called the claims line about a different crash, and can I show a prosecutor why I think so? The resolver answers with a confidence, the documents behind it, and a person's review for anything uncertain. For how the merged caller then turns eleven claims into one ring, see Fraud Network Analysis.

Questions people ask

What is the difference between entity resolution and identity resolution?

Mostly the industry using the term. Identity resolution usually means joining one customer's records across marketing and service systems; entity resolution is the broader name for matching any kind of record, including businesses, addresses and phones. The method underneath, scoring candidate pairs and deciding with thresholds, is the record linkage described by the Census Bureau.

What is blocking in entity resolution?

Blocking limits comparisons to pairs that share something cheap to check, such as a surname or a date of birth, because comparing every record with every other is too slow on large files. The Census Bureau notes that classical matching might run ten blocking passes. Each blocking key decides which true matches can ever be found, so a fraud tool should block on phones and accounts as well as names.

What is the Jaro-Winkler score?

Jaro-Winkler is a string comparator that scores two strings between 0 and 1, allowing for insertions, deletions and transposed letters, and giving extra weight to agreement at the start of the string. In the Census Bureau's table it scores SHACKLEFORD against SHACKELFORD at 0.982, and JONES against JOHNSON at 0.832, which is why it should never decide a match on its own.

Does entity resolution need a Social Security number?

No, and in claim documents one is rarely present. Without a unique identifier, matching relies on combinations of name, date of birth, address and phone, which the Census Bureau calls quasi-identifiers. For fraud rings, phones, payout accounts and payment cards are often stronger than names, because they cost money to change.

Is entity resolution machine learning?

It can be, but it does not have to be. The classical Fellegi-Sunter model estimates match weights statistically, and the Census Bureau estimated its parameters largely with the EM algorithm, a form of unsupervised learning. Newer systems add learned comparators. Either way, the useful outputs for an SIU are the same: a score, the reasons, and a band for human review.

How many uncertain matches should an analyst expect to review?

It depends on the thresholds and the data, which is why both thresholds should be settings. On the fictional sample morning in Fraud Network Analysis, with a 24-month look-back of 212,406 claims and a review band from 0.75 to 0.92, fourteen pairs were waiting for the fraud analyst. Widening the band trades analyst time for fewer wrong merges.