Using ChatGPT for AML compliance works for explaining concepts and drafting plain-language summaries, and fails where a BSA officer most needs it: the specific, current answer to a question about a real transaction. A general chatbot answers from patterns learned in training unless it is connected to the current text, it cannot show which version of a rule it relied on, and peer-reviewed studies of legal questions found public models wrong at least 58% of the time on verifiable facts about court cases. BSA rules also changed in ways a model trained earlier cannot know, from the October 2025 SAR FAQs to the April 2026 model risk guidance. An assistant built for the work, such as Botminds' BSA/AML Manual Search, retrieves the current passage, quotes it with its date, checks every sentence against it and sends the question to the BSA team when it cannot.
The product examples here come from a fictional sample credit union, Norrell Federal Credit Union, run through the solution. The research is linked where it is used.
One counter question, answered two ways
A general chatbot, answering from training: "Member wants to deposit $9,500 cash today and $9,500 tomorrow, do I file a CTR?"
An assistant that cites the manual: The same words, typed at the counter by a member service representative.
A general chatbot, answering from training: Answers at once, assuming no other cash that day.
An assistant that cites the manual: Asks the one fact that changes the answer: did the member bring in other cash today, at any branch, ATM or through someone else? Or reads it from the core with the asker's permission.
A general chatbot, answering from training: Often right on the arithmetic: two days, each under $10,000.
An assistant that cites the manual: No CTR for either day; splitting cash may be structuring, so refer it, and say nothing to the member about a referral.
A general chatbot, answering from training: No passage shown; any citation it adds has to be checked by hand.
An assistant that cites the manual: 31 CFR 1010.313(b) and 1010.311, the FFIEC manual's CTR section, 31 CFR 1010.100(xx) and the SAR FAQs of October 9, 2025, each with its date.
A general chatbot, answering from training: Not known.
An assistant that cites the manual: Quoted beside the rule, with version and effective date, and flagged where it is older than a source it relies on.
A general chatbot, answering from training: Generic advice, sometimes including how the reporting threshold works.
An assistant that cites the manual: Only the words the procedure allows, and never how to stay under the threshold.
A general chatbot, answering from training: A chat history on someone's screen.
An assistant that cites the manual: The answer kept with the sources as they stood that day, for the examiner.
Can ChatGPT answer BSA questions? Sometimes, and you cannot tell which times
A general chatbot can answer many BSA questions correctly, especially stable ones: what a CTR is, that the threshold is more than $10,000 in currency in one business day, what structuring means. The problem is not that it is usually wrong. It is that the answer gives no way to tell the right ones from the wrong ones, because it carries no passage, no date and no sign of doubt.
NIST calls the failure confabulation: "a phenomenon in which GAI systems generate and confidently present erroneous or false content in response to prompts." It explains why it happens. Generative models "generate outputs that approximate the statistical distribution of their training data," which "can produce factually accurate and consistent outputs" and can also produce outputs "that are factually inaccurate or internally inconsistent" (NIST AI 600-1, July 2024).
For a compliance answer the worst version is the confident citation. NIST notes that generative outputs "may also include confabulated logic or citations that purport to justify or explain the system's answer, which may further mislead humans into inappropriately trusting the system's output." A section number that looks right is the thing a busy reader is least likely to check.
Is ChatGPT reliable for AML compliance? What the studies measured
No study has measured chatbots on BSA questions specifically, but two peer-reviewed studies of legal questions give the closest evidence, and neither is reassuring. Using ChatGPT 4 and other public models on verifiable questions about federal court cases, Dahl and co-authors found that the models "hallucinate at least 58% of the time, struggle to predict their own hallucinations, and often uncritically accept users' incorrect legal assumptions" (Journal of Legal Analysis, 2024).
Retrieval helps but does not cure it. A preregistered evaluation of commercial legal research assistants built on retrieval-augmented generation found that while "hallucinations are reduced relative to general-purpose chatbots," the tools tested "each hallucinate between 17% and 33% of the time" (Journal of Empirical Legal Studies, 2025). The authors concluded that the providers' claims of hallucination-free answers were overstated.
What the research says about AI answers to legal questions
58%+
of answers hallucinated by public chatbots on verifiable questions about federal court cases
17–33%
hallucination rate of commercial legal research tools built on retrieval
12
risks NIST names for generative AI, confabulation among them
The gap between the two figures is the argument of this page. Grounding the answer in retrieved text cuts the error rate sharply; what remains has to be caught by checking each sentence against its source and by an assistant that is allowed to say it does not know.
Three BSA answers that changed after a model's training
Even a model that recalls the rules perfectly recalls them as of its training. BSA guidance changed in ways that make older answers wrong, and on some points the FFIEC manual, the text most often quoted online, has not caught up.
Three BSA questions whose answer changed in 2025 and 2026
| Question | The older answer still found in many sources | The current answer | Source |
|---|---|---|---|
| Must we document a decision not to file a SAR? | Yes: the FFIEC manual still says to document the reason for filing or not filing | No requirement or expectation to document it; a short statement will usually suffice where one is kept | SAR FAQs Q4, Oct 9, 2025 |
| Is a deposit just under $10,000 a reason to file a SAR? | Often treated as one on its own | Not by itself; the question is whether the activity is designed to evade the CTR | SAR FAQs Q1, Oct 9, 2025 |
| Does the 2021 interagency statement on model risk for BSA/AML systems apply? | Yes, it set expectations for monitoring models | Rescinded April 17, 2026; the revised guidance puts generative and agentic AI out of scope | SR 26-2; OCC Bulletin 2026-13 |
The first row is the trap that catches retrieval too. An assistant that finds and quotes the FFIEC manual's SAR section has cited a real passage accurately and still given an out-of-date answer. Why that happens, and how to see it coming, is the subject of the guide to FFIEC manual updates.
Is it safe to use ChatGPT in a bank? Three questions before anyone pastes a case
Using a general chatbot for public, conceptual questions carries little risk. Pasting a real case into one is a different act, and three questions should be answered first.
The first is confidentiality. No bank employee may disclose a SAR, or anything that would show one exists (31 CFR 1020.320(e)). A prompt that describes an investigation, a referral or a filing decision about a named customer can reveal exactly that, to a service outside the bank. Whether a given tool counts as a disclosure is a question for the BSA officer and counsel, decided before use, not after.
The second is the institution's own risk management. The agencies' 2018 statement on innovation, still cited by FinCEN in 2026, tells bank management to consider "information security issues, third-party risk management, and compliance with other applicable laws and regulations, such as those related to customer notifications and privacy" before relying on a new approach (joint statement, December 2018). A consumer chatbot account usually sits outside all three.
The third is where model governance stands. The April 2026 revised model risk guidance places generative and agentic AI outside its scope (OCC Bulletin 2026-13). That is not a permission. It means the institution has to set its own controls for these tools, and Treasury reported in December 2024 that some banks were hesitant to put large language models in front of customers because "inaccurate, inconsistent, or incomplete answers could lead to liability and reputational damage" (Treasury, AI in Financial Services).
How do you check an AI answer to a regulatory question?
Check it the way an examiner would check a memo: find the passage behind each claim, confirm the passage says it, and confirm nothing newer says otherwise. NIST's own action list for generative AI includes the same step, to "review and verify sources and citations in GAI system outputs" (NIST AI 600-1, MS-2.5-003).
Six checks for any AI answer to a BSA question
- Open every citation
Does the section exist, and does it say what the sentence claims? A real section number attached to the wrong rule is a confident error that is easy to miss.
- Find the date of the text
Which version of the regulation, FAQ or manual section was relied on? An undated quote cannot be shown to be current.
- Look for anything newer
FinCEN FAQs, rulings and agency bulletins can change the practical answer without changing the regulation.
- Separate the rule from the procedure
What the law requires and what your institution requires are different answers. Both matter; they should not be blended into one sentence.
- Ask what fact would change it
Most counter questions turn on one fact, such as other cash the same business day. An answer that never asked has assumed it.
- Check the status of anything proposed
A proposed rule, such as the April 2026 AML/CFT program proposal, is not in force and should never be the only basis of an answer.
What an AI compliance assistant that cites the manual does differently
An AI compliance assistant is a tool that answers staff questions from a defined set of sources, the regulations, guidance and the institution's procedures, and shows the passage behind each answer. The design choices that matter are the ones that make a wrong answer visible or stop it before it is shown.
In BSA/AML Manual Search, eight agents divide the work. A question reader marks whether a real member is involved. A fact finder asks back the one fact that would change the answer, or reads it from the core banking system with the asker's permission. A rules researcher searches 31 CFR Chapter X, the FFIEC manual and FinCEN rulings, FAQs and advisories passage by passage, keeping each passage's date; a procedure matcher finds the institution's own sections with their versions. An answer writer puts the verdict first and the rule beside the procedure. Then a citation checker reads every sentence back against its passage and removes anything the source does not say.
BSA/AML Manual Search asks back the deciding fact before any answer is written
On the sample question, the checker read all 11 sentences back against their passages and removed none, and the answer appeared at a confidence of 0.96. Every citation opens the passage itself, with the sentence relied on highlighted and the date of the copy used, so the reader can do the first two checks on the list in one click.
A BSA/AML Manual Search citation opens its passage: the check a reader can do in one click
Where a cited assistant still fails, and what it should do then
Retrieval narrows errors; it does not remove them. The legal research study's 17% to 33% came from tools that retrieved real sources. The failure that remains is usually a wrong passage, a passage read out of context, or a question the sources do not settle. The defence is an assistant that measures its own uncertainty and stops.
In the sample credit union's settings, an answer below a confidence of 0.85 is not shown at all; the question goes to the BSA team's queue with the passages found and the reason the agents stopped. One question in that queue, whether staff may confirm to a member's attorney that a CTR was filed, scored 0.71 and went to a person. Another was held because the institution's procedure and the FFIEC manual read differently. Those are the questions a general chatbot answers anyway.
The other limit is scope. An assistant that answers from rules and procedures does not decide whether to file a SAR, does not file anything and does not change a member's record. In the solution those decisions stay with a BSA analyst, the BSA officer and the compliance committee, which is the subject of earning trust in AI for compliance in banking.
What to ask before you buy an AI compliance assistant
The questions that separate tools are about sources and stopping, not about how fluent the answers sound. Ask a vendor to show, on your own procedures, what happens in each case below.
Buying questions for an AI assistant that answers BSA questions
- Which sources does it answer from, and how current is each copy?
Ask for the list with the sync date of each, and what happens when a sync fails.
- Does every sentence carry a citation to a passage, not just a document?
A link to the FFIEC manual is not a citation; a link to the paragraph is.
- Is each answer checked against its passages before anyone sees it?
Ask how many sentences are removed, and whether removals are logged.
- What does it do when it is unsure?
Look for a threshold and a queue to a person, not a softer-sounding answer.
- Does it keep your procedure separate from the rule?
And does it flag a procedure older than a source it relies on?
- How does it treat proposed rules?
They should be labelled not in force and never be the only basis of an answer.
- Where do member details go?
Member-linked questions need restricted history, masking, and a check that nothing reaches member-facing systems.
- Can you replay an old answer against the sources of that day?
That is what lets you show an examiner what staff were told, and on what basis.
For the economics of the same choice, set against what one question costs when the BSA team answers it by hand, see BSA compliance software and the cost of one regulatory question.
Questions people ask
What is the best AI for regulatory research in BSA?
The best tool is the one whose answers you can verify: it cites the passage, shows the date of the text, checks its sentences and declines when unsure. A preregistered study found that even retrieval-based legal tools hallucinated 17% to 33% of the time, so verification matters more than the model behind it.
Can AI answer compliance questions without a person checking?
Not for decisions with legal effect. NIST's generative AI profile lists verifying sources and citations as a core measure, and the 2026 model risk guidance leaves generative AI to each institution's own controls. A sound design lets AI answer and cite, and leaves filing, SAR decisions and procedure changes to named people.
Does FinCEN allow banks to use AI for AML compliance?
Nothing in the BSA rules forbids it, and the agencies' 2018 statement on innovation said that a pilot that fails is not, by itself, grounds for supervisory criticism. FinCEN's April 2026 proposed program rule names machine learning and generative AI as possible innovative approaches, but it is still a proposal.
Do AI legal research tools hallucinate?
Yes, measurably. In the Journal of Empirical Legal Studies evaluation, commercial legal research tools built on retrieval gave hallucinated answers between 17% and 33% of the time, fewer than a general chatbot but well above zero. Tools that let the user open each passage make those errors findable.
Should staff be told not to use ChatGPT for BSA questions?
A clear written rule beats a silent one. A workable rule allows general chatbots for public, conceptual material and forbids entering customer, account or investigation details, because 31 CFR 1020.320(e) bars revealing a SAR's existence. The rule belongs in the BSA program and in training, alongside an approved place to ask real questions.