AI control testing uses software agents to test internal controls the way a tester would: read the evidence, compare who prepared and who reviewed each instance, re-perform the arithmetic, and flag exceptions with the evidence they rest on. Compared with sampling by hand, its advantage is coverage: it can look at every instance of a control rather than the 25 a tester has time for. Compared with rules-based checks, its advantage is reach: it can reconcile, trace a Call Report line through a mapping rule, and read a leave roster, where a rule only tests the condition someone thought to write. None of the three should decide anything. Botminds' Regulatory Reporting Controls uses agents for the testing and leaves every conclusion to a named person.
Chance that a hand sample finds a single bad instance in a control's population
Can AI test internal controls?
It can perform the steps of a test; it cannot own the conclusion. A control test has mechanical parts (select the population, pull the evidence, compare names and times, re-foot and re-tie) and a judgment part (is this exception a deficiency, and how serious). Agents do the first kind well because they do not tire of the hundredth instance. The second belongs to management, and in a bank subject to Part 363 it ends in a management report signed by the CEO and CFO (12 CFR 363.2(c)).
The regulators' own use of AI follows the same split. GAO reported in May 2025 that the financial regulators using AI combined its outputs with examination reports and other supervisory information, and did not let it make decisions on its own or serve as their only input (GAO-25-107197). A bank's control testing should hold itself to no lower standard than its examiners do.
Three ways to test one quarter
Take a sample bank's third quarter: Tessaly Community Bank, fictional, with fictional people and figures, filing the FFIEC 041 for September 30, 2026. It decided four deficiencies that year: three in the quarter and one, a missing control, found in June. The table sets out which method would find each, and why.
Four issues in the sample bank's year, and which testing method finds each
| Issue | Sampling by hand | Rules-based checks | Agents |
|---|---|---|---|
| July loan reconciliation prepared and reviewed by the same person (1 of 3 monthly instances) | Found if July is the instance sampled: 1 chance in 3 for a one-instance quarterly test | Found, if a rule compares preparer and reviewer IDs on the sign-off log | Found one minute after the review, with the leave roster and the reviewer's entitlements read as well |
| Vendor master change approved by its own requester (1 of 22 changes) | Found if that change is among those sampled | Found, if a rule compares requester and approver | Found, with the change ticket cited |
| Manual journal entry of $1,240,000 approved the morning after posting | Found if the entry is sampled; the sample bank's random sample of 25 drew it as item 17 | Found, if a rule compares approval and posting times | Found in the same sample, and decided by the Controller as timing only, with no effect on the balance |
| No key control on the hop that classifies brokered deposits from partner programs | Not found: there is no instance to sample | Not found: no rule fires on a control that does not exist | Found by tracing the Call Report line hop by hop and naming the control on each |
What a 25-item sample can and cannot tell you
A sample estimates how often a control fails; it does not find the failures. If one instance in 250 is bad, a random sample of 25 contains it 25 times in 250, one time in ten. Testers know this, which is why sampling is paired with a tolerable rate of deviation and a conclusion about the population, not a list of every bad instance. That is a sound basis for an opinion on a control. It is a weak basis for a CFO who wants to know, before signing, whether any reconciliation under this quarter's report was self-reviewed.
The arithmetic of a hand sample, worked for the sample bank's July break
The sampling point applies to rules too, in a different way. A rule runs on every instance, so its coverage is complete, but only for the condition it tests. The sample bank's July break would have been found by a rule comparing user IDs; it would not have been explained by one. The rule cannot read that the reviewer was on leave from July 27 to August 7, that no backup was set, or that the only colleague who might have stepped in held posting rights in the loan subledger.
What does AI find that sampling misses?
Three kinds of problem, each for a different reason. The first is the rare single break: one self-review in twelve monthly instances, one self-approved change in 22, found only if the sample happens to draw it. Testing every instance removes the luck. The second is the missing control: a hop between the core system and a Call Report line that no control covers produces no instances at all, so neither a sample nor a rule can find it; tracing the line hop by hop can. The third is context from another system: whether the reviewer was on leave, whether a stand-in held posting rights, whether the evidence was created after the sign-off. A sample tester can find those things on the instances they draw. A rule finds them only if someone joined the systems in advance.
The July self-review, as a rule reports it and as agents report it
A rule on the sign-off log: RR-07 July: preparer ID = reviewer ID. Exception.
Agents on the same instance: Prepared and reviewed by Dana Ostrow, 14 minutes apart, same workstation, on a Sunday.
A rule on the sign-off log: None.
Agents on the same instance: Assigned reviewer on leave July 27 to August 7; no backup set; August and September reviewed independently.
A rule on the sign-off log: Not tested.
Agents on the same instance: Re-performed in full: ties, difference $0.00, every figure cited.
A rule on the sign-off log: Not addressed.
Agents on the same instance: Two independent people named; the preparer and a colleague with posting rights excluded.
A rule on the sign-off log: An exception in a queue.
Agents on the same instance: The owner's sub-certification held until the Controller decides; a classification proposed with its reasoning.
How does AI control testing work on a Call Report control?
It works on the evidence the control left behind, one instance at a time. For the July loan reconciliation, a checking agent read the general ledger sign-off log, found two entries for the same user 14 minutes apart, checked the reviewer roster and the August and September instances, and raised a flag citing each piece. A testing agent then re-performed July in full: the subledger balance of $2,481,659,357, plus a $246,857 payment batch in the subledger on July 31 and in the ledger on August 3, against a general ledger balance of $2,481,906,214. Difference: $0.00.
The checking agent's finding, every statement cited
Below its confidence threshold a finding shows as 'possible', never as a flag; the sample bank set the threshold for this agent at 0.80.
Two design rules make this testing rather than guessing. Every statement cites the evidence it rests on, so a reviewer can check the agent the way they would check a junior tester's work paper. And no agent closes its own flag or decides what an exception means; a person does, with a reason code, and that decision is what the auditor reads.
A sample tells you how often a control fails. Only testing every instance tells you which reconciliation under this quarter's report failed.
Is an AI control testing tool a model under model risk guidance?
Partly, and the 2026 guidance draws the line explicitly. The Federal Reserve, OCC and FDIC issued revised model risk guidance on April 17, 2026, which replaces SR 11-7, the 2011 guidance (SR 26-2). Its definition of a model leaves out simple arithmetic, of the kind found in spreadsheets, and rule-based processes and software with no statistical, economic or financial theory behind them. A rule that compares two user IDs, or a re-footing of a subledger, is not a model in that sense.
The agents are a different case, and the guidance leaves them outside its scope rather than inside it: a footnote places generative and agentic AI models beyond the guidance because they are new and changing fast. The sentence that follows matters as much: "a banking organization's risk management and governance practices should guide the determination of appropriate governance and controls for any tools, processes, or systems not covered" (Revised Guidance on Model Risk Management, 2026). The letter also says it is expected to be most relevant to banking organizations with over $30 billion in assets. For a community bank, that means the governance of an AI testing tool is the bank's to design, and it should look like the governance of any other control: versioned instructions, thresholds written down, results reviewed by a person.
Is AI control testing accepted by auditors and examiners?
We found no U.S. banking regulator guidance on AI for testing regulatory reporting controls specifically, so the question is whether the output meets the standards that already apply. Federal Reserve Governor Michelle Bowman made the general point in November 2024: the technology "does not operate outside the existing legal and regulatory framework" (Governor Bowman, November 22, 2024). An auditor deciding whether to use the bank's testing asks the questions it would ask of any tester's work: is it independent of the control owner, is it documented, can it be re-performed, and who reviewed it.
That favours testing that shows its work. A tool that reports "pass" for a reconciliation will be re-tested. A tool that shows the balances, the reconciling item, its source document and the trial balance it tied to gives the auditor something to inspect. Internal audit's position is the clearest: in the sample bank it reads the agents' results without relying on them, and tests six controls itself with its own samples. The OCC's booklet asks examiners to consider "independence of personnel performing reviews" when assessing quality control and quality assurance (OCC Comptroller's Handbook).
What to ask before you rely on AI control testing
Ask what an examiner would ask of a new tester. The checklist below works on a demonstration and on a pilot; run it on a quarter your team has already tested by hand, with your own results held back, and count what the tool finds that you found, what it finds that you missed, and what it misses.
Seven questions for any AI control testing tool
- Does every finding cite its evidence?
A log entry, a ticket, a balance with its source. A finding without a citation cannot be reviewed.
- Does it test every instance, or a sample?
If a sample, how drawn, with what seed, and is every flagged instance then tested in full?
- Does it show its arithmetic?
Re-performance should list every figure and where it came from, not a pass mark.
- Who can close a flag?
Never the agent that raised it, and never the owner of the control it concerns.
- What happens below its confidence threshold?
A low-confidence finding should be shown as possible and left for a person, not dropped and not raised as a flag.
- Are its instructions and models versioned?
So that a result from March can be reproduced in December, when the auditor asks.
- Does it ever decide?
Classification of a deficiency, acceptance of a reconciling item and the attestation itself stay with named people.
Where Regulatory Reporting Controls fits, and what it leaves to people
The solution uses all three methods in their places. Deterministic checks run on every instance: preparer against reviewer, approval time against posting time. Agents re-perform the 14 key controls on 25-item random samples with recorded seeds and every flagged instance in full, trace the top Call Report lines hop by hop, and propose a classification for each exception against the bank's severity guide. Internal audit keeps its own samples. In the sample quarter, 349 of 350 items passed and the one exception was a timing difference the Controller decided with a reason code.
Where it does not fit: it does not replace the external auditor's testing or internal audit's independent view, it does not test controls whose evidence lives only in someone's head, and it does not decide. The design rules are simple to state and checkable on screen: agents never sign, never close their own flags, never change a mapping and never decide a deficiency. How a decided exception becomes a classified deficiency is covered in material weakness vs significant deficiency, and the measures that show whether the testing is working in the KPIs that prove SOX controls automation.
Questions people ask
What is control re-performance?
Re-performance is doing the control again independently and comparing the result with what the control produced: re-footing a subledger, re-agreeing reconciling items to their sources and re-tying to the trial balance. It is stronger evidence than inspecting a sign-off, because it tests the work and not only the signature.
What is continuous control testing?
Testing triggered by the control itself running, rather than by a testing calendar: the check on a reconciliation runs when it is marked reviewed. In the sample bank, the July self-review was flagged at 19:07, one minute after the review was recorded.
How big should a control testing sample be?
Big enough to support the conclusion the tester needs, which depends on the control's frequency and the deviation rate the bank will tolerate; the bank should write its rule down before testing. Where every instance can be compared automatically, as with monthly reconciliations, a sample is unnecessary.
Can AI control testing produce false alarms?
Yes, which is why each finding should carry its evidence and a confidence score, and why findings below a threshold should be shown as possible rather than flagged. A false alarm that cites a log entry takes a minute to dismiss; one that does not takes an investigation.
Does AI control testing replace internal audit?
No. Internal audit's value is its independence from management, including from management's testing tools. It can use the results to plan its work, but its conclusions should rest on its own samples and its own review.