Fraud detection software for banks falls, broadly, into three working approaches: a sampled manual review by trained analysts, a rules-based alert engine that flags transactions or applications against fixed thresholds, and an agentic system that re-reads every file and proposes a cause for a person to confirm. Each one is a real, defensible way to run a credit risk desk, and each one catches a different slice of the same loans. The honest comparison is not accuracy in the abstract — it is which of the three ever looks at the file that actually matters, on the vintage you actually have.
This guide tests the three approaches on one fictional sample vintage from Botminds' Early Payment Default Review: Lanternfield Bank, August, 87 early defaults out of 4,812 loans booked, $1,268,450 at risk. The research figures come from GAO, the Federal Reserve, the OCC and NIST, linked where they appear.
Three ways to review early payment defaults, tested on one vintage
A sampled manual review pulls a subset of early defaults — a risk-weighted or random sample, which we assume at 25 of the 87 on this vintage — and has an analyst read each file by hand: application, documents, verification, the decision record, the payment history. It is slow and it is thorough on the files it reaches, which is exactly the problem: it was never designed to reach all of them.
A rules-based alert engine runs every loan, not a sample, against fixed thresholds set in advance — a fraud score above a cut-off, a debt-to-income ratio outside a band, a device seen on more than one application. It is fast and complete in coverage, but it only ever finds what its rules were written to look for, which means a fraud pattern or a credit policy gap that nobody anticipated passes through unflagged, however many loans the engine processes.
An agentic full re-read, the approach Early Payment Default Review takes, has specialist agents read every file — all 87, not a sample — the way an underwriter would after the fact: re-reading the application and verification, linking loans by device and account across 180 days, replaying the credit decision under the policy version in force, checking what the bank's own onboarding did. It proposes a cause for every loan, with evidence and a confidence score, for a person to confirm. It is complete in coverage like the rules engine and detailed like the sampled review, and its limit is the one this whole series keeps returning to: a proposed cause is not a confirmed one until a person reads it.
The same vintage worked through each of the three approaches
| Approach | Files reached | Time to a result | What still has to happen after |
|---|---|---|---|
| Sampled manual review | 25 of 87, by design | About 6 weeks, estimated | Agreeing causes and fixes with owners, then writing the report |
| Rules-based alert engine | 87 of 87 scored | Minutes, same day | Turning individual alerts into a confirmed cause — the engine does not connect them |
| Agentic full re-read | 87 of 87 re-read | 1 min 52 s to propose a cause for each | Analyst confirms 79 together, reads 8 individually, within a 5-day review clock |
What a sampled manual review catches, and what it misses by design
On the files it reads, a sampled review is genuinely thorough — an experienced analyst reading an application end to end catches nuance a rule cannot be written for, and can ask the kind of open question ("does this employer phone number look right?") that a fixed threshold never will. What it misses is entirely a function of what it never reads. If the sample draws 25 of 87 at random, any particular loan from Ring R-0926 has about a 29% chance of being read, and the sample will on average contain five or six of the ring's 19 loans. That is enough to see several frauds, but not that they are one ring: the shared devices, accounts and pay-stub template only line up when a reader compares every file against every other, including the 62 the sample never opens. A sample built to size up a vintage default rate was never built to find a pattern that only becomes visible once several loans are compared side by side.
What a rules-based alert engine catches, and where it stays blind
A rules engine's strength is also its ceiling: it will find exactly what it was told to look for, on every loan, every time, with no sampling gap. If a rule exists for "device seen on more than one application in 30 days," it would have flagged several of Ring R-0926's 19 loans individually. What it would not have done on its own is connect those individual flags into one ring, assemble the shared bank accounts and the pay-stub template, or know that the rule driving the credit-band defaults (CP-26-07) had changed six weeks earlier and explained a different 21 loans entirely. A rules engine answers "does this loan match a known pattern," not "why did this loan default," and the gap between those two questions is most of the work a credit risk analyst still has to do after the alerts come in.
What each approach actually delivers on this vintage
Coverage and detail: 25 of 87 files read in full depth, with a person’s judgment on each one.
What it misses: 62 files never read. A pattern spread thin across the vintage is unlikely to surface.
Coverage and detail: All 87 scored the same day against fixed thresholds — full coverage, no sampling gap.
What it misses: Only finds what a rule was written for. Does not connect individual flags into a ring or trace a rule change.
Coverage and detail: All 87 files re-read, causes proposed with evidence, patterns like the ring and the rule found automatically.
What it misses: A proposed cause is not a confirmed one — still needs a person to read and sign every case.
What an agentic full re-read adds, and what it still cannot do
What an agentic re-read adds over the other two is coverage combined with connection: every file gets the detailed reading a sampled review gives only 29% of loans, and the link analyst agent does the cross-file comparison — devices, accounts, employers, document templates across the last 180 days — that a rules engine's per-loan scoring never attempts. On the sample vintage, that combination is what surfaced Ring R-0926 as one pattern of 19 loans rather than 19 separate flags, and what traced 21 of the credit-band defaults to one rule change rather than 21 individual credit misses.
What it still cannot do is decide. The agents propose a cause with a confidence score; "the cause and fix writer agent never confirms a cause" and "never sends a fix without the analyst," by the product's own design. The link analyst "never blocks an account or declines an application" and "never files a report — prepares the referral." The credit rule tracer "never changes a rule," and every proposal passes fair-lending review before the credit risk committee sees it. A full re-read changes how much ground gets covered before a person makes the call; it does not remove the person from the call.
Where each approach still needs a person, not a flag
All three approaches converge on the same handful of decisions that stay with a named person regardless of which tool found the evidence: confirming a cause, sending a fix to the team that owns it, deciding whether a ring goes to the BSA team as a suspicious activity report, and signing off on a credit policy change after fair-lending review. "Human roles and responsibilities in decision making and overseeing AI systems need to be clearly defined and differentiated," NIST's AI Risk Management Framework states, adding that "human-AI configurations can span from fully autonomous to fully manual" (NIST AI 100-1). The question a buyer should be asking is not whether a tool automates the decision — none of the three legitimately should — but how much of the work before the decision it removes, and how clearly it draws the line at where the automation stops.
What regulators expect you to be able to show, whichever you choose
Whichever approach a bank runs, examiners expect to see the same discipline behind it. "Well-run banks generally handle exceptions in a structured and disciplined manner to promote sound operations as well as compliance with applicable laws and regulations, including those related to consumer protection and fair lending," the OCC's Comptroller's Handbook on retail lending states, and "policy exception volumes and analysis should be a normal part of board and senior management reporting" (OCC, Retail Lending). A sampled review, a rules engine and an agentic re-read all have to be able to produce that reporting; the difference is how much of the vintage the reporting is actually based on.
For any tool that uses generative or agentic AI specifically, the current model risk guidance leaves a gap a buyer has to fill rather than rely on. "Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance," the Federal Reserve, OCC and FDIC's SR 26-2 states (SR 26-2), while still expecting "a banking organization's risk management and governance practices" to determine appropriate controls. A rules engine, built on traditional logic rather than a generative or agentic model, may sit more comfortably inside existing model-risk frameworks for exactly that reason — which is itself one factor in the choice, separate from how well each approach finds the cause.
Coverage on the sample vintage, and the scale of identity crime it sits in
90%+
confidence or more on 79 of 87 proposed causes, eligible for bulk confirmation on the sample vintage
1 min 52s
for six agents to re-read all 87 files, demonstrated on the sample vintage
42%
of all BSA reports filed in 2021 were identity-related, $212 billion in suspicious activity
What to ask a vendor to run on your own closed files before buying
The fairest test of any of the three approaches is not a vendor's demonstration vintage; it is a bank's own closed files, where the true cause of each early default is already known from a completed investigation. A pilot that re-runs last quarter's sampled review, a rules engine's alerts and an agentic re-read side by side on the same closed vintage shows, concretely, which loans each approach would have caught, which it would have missed, and how long each took — the exact comparison this guide ran on one fictional vintage, done for real on your own.
What to ask any vendor to run before buying
- Run it on a closed vintage, not a demo one
A tool that performs well on its own curated sample data tells you little about your loans, your channels and your fraud patterns.
- Compare against what your sampled review already found
Where the new approach agrees with a closed, confirmed cause, and where it disagrees, is the start of a real false positive and false negative rate.
- Ask what it would have missed, not just what it found
A rules engine’s coverage gap and a sampled review’s sampling gap are both measurable once the true causes are known.
- Time the whole path, not just the scoring step
A rules engine that scores in minutes still needs a person to turn the alerts into confirmed causes and routed fixes — measure that too.
- Ask what the tool never does on its own
Every limit — never blocks an account, never changes a rule, never confirms a cause — should be a specific, checkable sentence, not a general assurance.
What "live" should mean, and the quality checks it has to pass first
None of the three approaches should go live on a bank's production vintage without first passing the same checks a bank already applies to any system that touches credit decisions and fair-lending-sensitive outcomes: validated on closed files, reviewed by the model risk or credit risk function appropriate to the approach, and, for anything touching a credit rule, run through fair-lending review before a change reaches the committee. The product's own description ties going live to a list of quality checks completing, with a walkthrough run first on the bank's own early defaults rather than a demonstration vintage.
Six agents re-reading 87 files, step by step
The business case for closing the coverage gap, whichever tool closes it, is the subject of what fraud actually costs a bank. And because a confidence score is not the same thing as a measured accuracy rate, false positive rate in fraud detection sets out the four numbers worth asking any of the three approaches for before trusting its output at scale. To see the full re-read, the link map and the rule trace together on one vintage, start from Early Payment Default Review.
Questions people ask
What is the best fraud detection software for banks?
There is no single best answer across every bank; a sampled manual review, a rules-based alert engine and an agentic full re-read each cover different ground, and most mature desks eventually combine more than one. The right starting question is not which category is best in general but which gap costs your bank the most today — missed coverage, missed patterns, or missed speed.
Is rules-based fraud detection still worth using next to AI?
Yes, for what it is good at: fast, complete, predictable coverage against known patterns. Its limit is that it only catches what a rule anticipated, which is exactly where an agentic re-read's cross-file link analysis adds ground a fixed threshold was never written to cover.
How accurate is AI fraud detection compared with a sampled manual review?
Accuracy depends on measurement, not the category of tool. A sampled review is thorough on the files it reads and blind to the rest; an agentic re-read reaches every file but still needs its proposed causes checked against closed files to know its real accuracy. Neither should be assumed accurate without that measurement — see how to measure it.
How long does a sampled review of early payment defaults usually take?
There is no published industry benchmark. Our own estimate is roughly six weeks for one monthly vintage's sampled review of about 25 files, an estimate rather than a measured figure — a bank should time its own process rather than assume this number applies.
Can a fraud detection tool decide a loan is fraud on its own?
In a well-governed design, no. Early Payment Default Review's agents propose a cause with evidence and a confidence score; the credit risk analyst confirms or changes it. The system never blocks an account, contacts a borrower, changes a credit rule or files a suspicious activity report on its own.
What should a bank ask before switching from a sampled review to an agentic re-read?
Ask for a pilot on your own closed, confirmed files rather than a demonstration vintage, time the whole path from flag to confirmed cause rather than just the scoring step, and get a specific, checkable list of what the system never does on its own — see the checklist in this guide.