Compliance monitoring and testing, task by task: agents and people

Compliance monitoring and testing is the bank's own, continuing check that it follows the consumer rules: the monitoring watches for weaknesses across products and channels, and the testing proves it on actual transactions, such as a quarter of Regulation CC exception holds or Regulation E error claims. Split a single test into its tasks and a pattern shows: of about a dozen steps, only three or four need a person's judgement. The rest are pulling a population, counting it, fetching documents, reading two screens side by side and typing up the result. Botminds' Consumer Compliance Testing hands those steps to agents, each with limits written down, and leaves scope, the exception call, the cause and approval to the analyst and the compliance manager.

What is compliance monitoring and testing?

It is the part of a compliance management system that checks, on a schedule and at the transaction level, whether the rules are being followed. The FDIC's examination manual describes an effective monitoring system as "regularly scheduled reviews" of, among other things, disclosures and calculations, document filing and retention, and "posted notices, marketing literature, and advertising" (FDIC Consumer Compliance Examination Manual II-3, June 2019). For deposits, that list reads like a testing plan: hold notices, error-resolution letters, account disclosures, change-in-terms notices and the availability notice in the lobby.

The CFPB's examiners check the same scope from the other side. Their procedures ask how far monitoring reaches into calculation tools, consumer disclosures and notices, marketing, and the scripts staff use with customers (CFPB Supervision and Examination Manual, November 2025, Compliance Management Review). A program that tests the core's numbers but never reads what the customer was sent covers only half of that sentence.

What is the difference between compliance monitoring and testing?

Monitoring is the broad watch; testing is its sharpest tool. Monitoring includes complaint trends, management reports, quality control in the business, training checks and reviews of new products. Testing takes a defined population and checks each item, or a sample, against attributes drawn from the rule. The OCC puts the relationship plainly: "Transaction testing is an important component of monitoring." The same passage says a bank's "sampling methodology should incorporate appropriate sampling techniques" and that testing "may be conducted by compliance staff or within business lines" (OCC Comptroller's Handbook, Compliance Management Systems, version 1.0, June 2018).

Both sit apart from audit, which is independent and validates the program as a whole. Compliance testing vs audit covers that line and why examiners rate the two separately. This page stays on the testing side, where the hours are.

The tasks agents take, and the limits on each

The tasks that move are the ones with a right answer that can be checked. Pulling every hold placed between July 1 and September 30 has a right answer, and so does reconciling the count to the core's daily hold report. So does reading a notice and recording that it says "Amount delayed: entire deposit". Each of those can be done by an agent and checked by another agent or a person, and each leaves evidence.

The limits matter as much as the tasks. In Botminds' Consumer Compliance Testing, the agent that pulls a population never fills a gap: if the count is short, the test stops and a person decides whether a partial run is acceptable. The agent that reads notices decides nothing below a confidence floor, 0.95 by default, and cites both the rule and the record for every fail. The agent that drafts workpapers shows only the last four digits of an account and sends nothing without the analyst's confirmation. None of them approves a cause, moves money or closes an issue.

What the agents read, system by system

Every task in the grid above depends on reaching the evidence, and deposit evidence is scattered. The core holds the holds, deposits, balances, fees and statements. Document imaging holds the hold notices, disclosures, letters and the mail log. The dispute system holds Regulation E claims with their dates and letters; the contact center system holds the call notes that show when a customer first reported an error. Teller platform settings show which notice form each branch actually prints, and the general ledger gives the fee income a population can be reconciled to.

In Consumer Compliance Testing all of those connections are read only, and the only system the solution writes to is the issue register, where findings become issues for their owners. Data is kept to what a test needs: workpapers show the last four digits of an account, business owners see only their own issues, and testing never reads or shows whether a suspicious activity report exists. Those limits matter as much to a bank's information security review as the agents' accuracy does to its compliance review.

Which tasks stay with the analyst and the manager?

Four kinds of decision stay with people, because each is a judgement an examiner will hold a named person to. What to test and how often comes from the risk assessment. Whether an item is an exception is a reading of the rule against the facts, and the facts are sometimes ambiguous: a notice the branch cannot find is missing evidence until someone decides it is a missing notice. Severity weighs harm to customers. And the cause is the hardest call of all, because it decides what gets fixed.

The cause is also what examiners grade. The interagency rating system weighs every violation on its root cause, its severity, how long it lasted and how many consumers it reached (FFIEC Uniform Interagency Consumer Compliance Rating System, 81 FR 79473), and the OCC expects banks to run root cause analyses "to determine whether identified violations of law or regulation or failures to follow established policies and procedures are isolated events or systemic" (OCC, Compliance Management Systems). An agent can propose that six exceptions share one cause and show why. A person approves it.

Agents test every hold; the analyst decides what the exceptions mean

Isolated or systemic: who decides, and on what

The sample bank's hold test shows the call in both directions. Six of nine exceptions shared one cause, and the analyst confirmed them together, each keeping its own workpaper. The other three did not: one notice was mailed a business day late, one cited repeated overdrafts the account history did not support, and one notice could not be found at all. The analyst confirmed each as isolated with a reason code and reported it to the branch manager, with no finding. A pattern agent could have grouped them wrongly; the reason codes are how a manager, and later an examiner, sees that a person decided they did not belong together.

Can AI write compliance workpapers?

Yes, as drafts a person confirms, and only if every statement in them points to its evidence. The risk is not that a model writes badly but that it writes confidently about something that is not there. A peer-reviewed study of leading AI legal research tools found that each one hallucinated "between 17% and 33%" of the time on its test queries (Magesh et al., Journal of Empirical Legal Studies, 2025). A workpaper with an invented rule citation or a misread date is worse than no workpaper, because it looks finished.

Three design choices answer that risk without asking anyone to trust the model. Every attribute, exception and statement cites the rule text and the record it rests on, so a reviewer can check it in one click. Missing evidence shows as missing, never as a pass. And a second agent re-tests independently, so an error has to be made twice, by different models, to get through. The federal banking agencies' revised model risk guidance, issued April 17, 2026, is principles rather than rules, says that falling short of it will not by itself draw supervisory criticism, and is aimed mainly at banks over $30 billion (FDIC FIL-15-2026). A bank of any size will still be asked how it knows the agents are right, and a blind re-test is a measurable answer.

A blind second test, and who settles a disagreement

A blind re-test turns "is the agent accurate?" into a number the bank can track. In the solution's default setting, a second agent on a different model family re-tests every exception and a random 5% of passes, without seeing the first answer. On the sample bank's third quarter it re-tested 469 items and agreed on 468. The one disagreement shows why a person has to settle it.

A Regulation E claim for $486.20 was keyed in the dispute system on Monday, August 31, and provisionally credited on Monday, September 14: business day 9 by the dispute system's clock, inside the 10 business days the rule allows (12 CFR 1005.11(c)). The second agent read the call notes and found the customer had first reported the error by phone on Thursday, August 27. An oral notice starts the clock. Counting from the call, with Labor Day skipped, September 14 was business day 11. The analyst decides; the second agent never does.

The one disagreement in 469 blind re-tests, laid out for a person

Where the analyst's days go, before and after

Moving tasks changes the shape of the work more than its total. On the sample bank's estimate, reading items took about a third of a sampled quarter's 31 business days and the manager's review about an eighth. With agents doing the reading, the design target is about 9 days, and the largest share left is approval: the decisions the program exists to produce.

The cost side of that shift, and how it compares with paying an outside firm, is worked through in compliance testing services, in-house or agents. The point here is narrower: the hours that disappear are the ones nobody would choose to spend.

How a test changes without breaking

Rules change, products change and so do the agents, and each is a way for a test that passed last quarter to test the wrong thing this quarter. In Consumer Compliance Testing a test is written as one plain sentence, and an agent turns the sentence into attributes, each cited to the section it checks. A change to the sentence is compiled the same way and dry-run on last quarter's items, so the manager sees which results would change before approving it. Every version is kept with who changed it, when and why.

The same discipline applies to the agents themselves. A new model or prompt version re-runs last quarter first, and the version of each agent is recorded against every result, so an examiner asking how a hold was tested in August can be shown the script, the attribute and the agent that read it. At the sample bank, the hold test's third version took effect on July 1, 2025 with the new Regulation CC amounts.

What a compliance monitoring and testing framework must now say

A written framework that assumed people did every step needs a few new clauses once agents do some of them. None of them is exotic; each turns an informal habit into a rule an examiner can read. The GAO's review of AI in financial services sums up the balance: the technology "presents both benefits and risks" (GAO-25-107197, May 2025). The framework is where a bank writes down which risks it has closed and how.

How would agents test Reg CC holds, start to finish?

The sample bank's third quarter shows the whole chain. 1,512 exception holds were placed; the count matched the daily hold report; every one was read beside its notice; nine failed. Six of the nine were large-deposit holds at two branches that held the whole deposit, because both branches' teller platforms still printed a notice form retired on July 1, 2025. The other 39 branches held only the excess on all 214 of theirs. The look-back found 21 more such holds since July 2025: 27 customers in all, three of whom paid a $34.00 overdraft fee while their funds were held.

The analyst confirmed the six in one step, each keeping its own workpaper, and recorded the other three as isolated errors for their branch managers. The compliance manager approved the finding and its cause. An issue opened for deposit operations with refunds of $102.00 to sign and a re-test of November's holds before it can close. Each screen is on the Consumer Compliance Testing page; the arithmetic of the first hold is in transactional testing of Reg CC availability.

Questions people ask

Which compliance monitoring and testing tasks can agents do?

The ones with a checkable answer: pulling and reconciling populations, fetching documents, reading a notice or letter against the core record, recomputing dates and amounts, and drafting workpapers with citations. Choosing what to test, calling an ambiguous exception, approving a cause and closing an issue are judgements to keep with people.

Who decides a compliance exception?

The analyst who ran the test makes the call, and the compliance manager reviews it before it becomes part of a finding. With agents, the analyst confirms or clears each exception with a reason code; an agent's result is a proposal until then.

What is a compliance monitoring and testing framework?

It is the written plan for how the bank checks its own compliance: what is monitored, which tests run and how often, how populations and samples are set, who reviews results and how findings are tracked to closure. The OCC expects the plan and schedule to be risk-based, documented and updated at least once a year.

How does monitoring differ from audit?

Monitoring is management's own, frequent and less formal check, often run by compliance or the business. Audit is periodic, more formal and independent of both, and it tests whether the whole program works. The FDIC treats the two as complements and expects many banks to need both.

Does monitoring have to include transaction testing?

Not for every rule at every bank, but for high-volume deposit rules it is the core of it. The OCC calls transaction testing "an important component of monitoring", and examiners ask about the sampling method behind it.

How do you know the agents tested correctly?

Measure it. A blind re-test by a different model, on every exception and a random share of passes, gives an agreement rate by script; every disagreement goes to a person. On the sample bank, 468 of 469 re-tests agreed.