Control testing automation is software that tests controls as they run, comparing who prepared and who reviewed each instance, re-performing reconciliations and tie-outs, and keeping the evidence, instead of a person sampling them by hand weeks later. For a bank's regulatory reporting controls, the date that decides whether it was worth it is the fiscal year-end: Part 363's annual report is due within 90 days for public institutions and 120 days for the others, with no extensions, and at $5 billion and above it carries management's assessment of internal control as of the last day of the year. Automation that goes live in the fourth quarter tests one quarter of twelve months. The plan below gets Botminds' Regulatory Reporting Controls testing on live quarters well before that, with the rules of the test written first.
Year-end readiness: ten things in place before control testing automation goes live
- A control library with owners, reviewers and backups
Every key control with its owner, its named reviewer and a named backup for leave. In the sample bank, the one self-review came in the week the backup column was blank.
- The evidence each control produces
For each control, the documents and log entries that prove it ran, and the system that holds them.
- A written severity guide
How the bank decides control deficiency, significant deficiency and material weakness, before the first exception arrives.
- Sample sizes and selection method
Per control frequency, with random selection and a recorded seed. Monthly and quarterly controls tested on every instance.
- What counts as an exception, and who decides
A late approval, a self-review, a difference over a stated amount. The decider is a named role, never the tool.
- Internal audit's own lane
Agreed with the head of internal audit: what they will test themselves, and that they read the automated results without relying on them.
- Read-only access to the systems controls run in
General ledger, regulatory reporting system, core, tickets, access management, document store.
- A baseline quarter
The last quarter filed by hand, re-run through the new process, to set the starting values of every measure.
- A parallel quarter
One live quarter with both processes running, so differences can be explained before the old one stops.
- The auditor's view of the outputs
The external accountant told, before year-end, what evidence and test results the bank will be able to show.
What is control testing automation?
It is the part of a controls programme where a machine does the checking a tester would do: pull the population, select a sample or take every instance, re-perform the control's arithmetic, compare it with what was signed, and record the result with its evidence. It differs from evidence collection, which only gathers proof that a control ran, and from monitoring dashboards, which count activity. A test produces a pass or an exception, and an exception goes to a person.
For Call Report controls, the tests are concrete. Does the loan subledger agree with general ledger accounts 1400–1490 at month-end after the reconciling items? Was the reconciliation reviewed by someone other than its preparer? Was each manual journal entry over the bank's threshold approved before it was posted? Was each mapping change approved and tested by a second person? The OCC's examination procedures test the same things from the other side: they sample report schedules and work papers and, where software maps accounts to lines, a sample of coded items (OCC Comptroller's Handbook, "Regulatory Reporting").
When is the Part 363 report due, and what does that set?
Within 120 days after the fiscal year-end for an institution that is neither a public company nor the subsidiary of one, and within 90 days for one that is (12 CFR 363.4(a)(3)). For a calendar-year bank closing December 31, 2026, that is April 30, 2027 or March 31, 2027. The rule is blunt about time: "No extensions of time for filing reports required by § 363.4 shall be granted", and the FDIC's guidelines add: "An institution's failure to timely file is considered an apparent violation of part 363."
The Part 363 calendar for a December 31, 2026 year-end
Mar 312027
Part 363 Annual Report due: public institutions and their subsidiaries (90 days)
Apr 302027
Part 363 Annual Report due: all other covered institutions (120 days)
$5B
assets at the start of the fiscal year from which management assesses internal control and the auditor attests
The deadline sets the go-live date backwards. Management's assessment is as of year-end, but an assessment built only on fourth-quarter testing has nothing to say about whether the controls held in March or August, and an exception found in the first week of January has almost no time to be remediated before the report. A bank planning for the 2027 year-end should aim to run in parallel in the second quarter of 2027 and live in the third. For the 2026 year-end, it is late to change the testing approach; the useful work this winter is the baseline and the control library.
What controls should be tested first?
Start where a failure would move a reported line most, and where people, not systems, carry the control. The OCC's materiality test points at capital: its booklet warns that a small adjustment can be material when capital is thin enough for it to change the bank's prompt corrective action category. Its procedures point at manual adjustments, mapping and independent reconcilements. Put together, a first wave of eight to fifteen controls can cover the top lines of Schedule RC and RC-R.
A first wave of controls for control testing automation, and why each goes first
| Control | Why first | Test |
|---|---|---|
| Call Report tied to the general ledger trial balance | Every line rests on it; a break here is a break everywhere | Re-tie a sample of lines each draft, every line at final |
| Loan and deposit subledger to general ledger reconciliations | The largest balances on Schedule RC, and controls prepared and reviewed at month-end under time pressure | Every monthly instance: re-foot, agree reconciling items, compare preparer and reviewer |
| Manual journal entries above a threshold | A manual entry can move a balance across lines or periods | Random sample with a recorded seed; approval time against posting time |
| Mapping changes in the regulatory reporting system | One rule moves every account mapped to it, every quarter after | Every change: ticket, approver, tester, version |
| Allowance and capital inputs | Small errors here can be material through capital | Re-tie inputs to their sources; check the reviewer |
| Pre-filing variance review | The last detective control before the CFO signs | Every line above the bank's threshold explained and reviewed |
A control testing plan, written before the first test
A control testing plan states, for each control, the population, the sample size and how it is drawn, what counts as an exception, who decides an exception, and when a failed control is re-tested. It has to exist before the tool runs, because a plan written after the results invites choosing the rule that fits the result.
None of the texts that govern these controls, Part 363, the Call Report instructions or the OCC's booklet, sets a sample size for management's own control testing. The OCC's examination procedures refer examiners to the Comptroller's Handbook booklet on sampling methodologies for judgmental and statistical samples. What a bank needs is a rule it applies consistently and can defend. The sample bank in this series, Tessaly Community Bank (fictional, with fictional people and figures), set two: 25 items for daily and per-transaction controls, and every instance for monthly and quarterly controls. A monthly reconciliation has three instances a quarter, so testing all of them is cheaper than arguing about a sample of one.
A flagged instance re-performed in full, every line of arithmetic cited
Two live quarters before year-end: the go-live plan
The plan has three stages: a baseline on a quarter already filed, a parallel quarter with both processes running, and a live quarter in which the manual sampling stops. Each stage has an exit test, and the stages are sized so that the last one finishes before the fourth quarter.
Control testing automation go-live, sized to finish before the fourth quarter
- Rules3 wk
- Connect1 wk
- Baseline2 wk
- Both processes run; differences explained1 quarter
- Automated testing only; internal audit tests on its own1 quarter
The baseline matters more than it looks. Re-running the last filed quarter through the new process shows what the old process missed, which is the case for the change, and it shows the bank what its exception rate looks like before anyone is under pressure to explain it. The measures to baseline are set out in SOX controls automation for regulatory reports: the KPIs that prove it.
Who owns control testing?
Three groups test, for different purposes, and automation changes the work of each without merging them. The OCC puts it in terms of lines of defense: "The effectiveness of internal controls is assessed through the bank's risk reviews (often second line of defense) and audit program (third line of defense)." It also separates quality control, done by an independent party before filing, from quality assurance, done independently of preparer and reviewer after filing (OCC Comptroller's Handbook).
Who does what in control testing, before and after automation
Today
Runs the control, then answers an evidence request weeks later and signs a sub-certification from memory.
With agents
Runs the control; evidence is captured as it completes; can sign only when the control has no open flags.
Still theirs: Running the control and signing for it.
Today
Samples by hand, chases evidence and tracks signatures in a spreadsheet.
With agents
Reads the exceptions with their evidence, assigns independent reviewers, decides each deficiency with a reason.
Still theirs: Every decision on severity, and the recommendation to the CFO.
Today
Tests a handful of controls with its own samples, months after the quarter.
With agents
Reads the automated results without relying on them, and records its own tests in its own lane.
Still theirs: Its independence, its samples and its conclusions.
Today
Signs on the strength of a tracker of emails.
With agents
Signs when every sub-certification is in and everything behind the declaration is listed and open.
Still theirs: The attestation.
How do you implement continuous control monitoring?
By deciding, for each control, which event triggers a check, and running the check then rather than on a testing calendar. GAO's standards distinguish ongoing monitoring, performed "as part of the normal course of operations" and which "may include automated tools", from separate evaluations, which look at controls "at a specific time" (GAO Green Book, paras. 16.05–16.06). Continuous control monitoring moves as many tests as possible from the second kind to the first.
The triggers are usually events the systems already write down: a reconciliation marked reviewed, a journal entry posted, a mapping version released, an access certification completed. In the sample bank, the self-review in July was flagged at 19:07, one minute after Dana marked the reconciliation reviewed, because the check ran on that event. The quarterly separate evaluation still has a place, for controls with no system event and for internal audit's independent view.
Where Regulatory Reporting Controls fits in the plan
The solution supplies the testing, the evidence and the calendar; the bank supplies the rules. Its testing agent re-performs key controls on the bank's samples and every flagged instance in full, shows its arithmetic and marks each item pass or exception. It never concludes for internal audit and never re-performs on data it cannot cite. A deficiency agent proposes a classification against the bank's severity guide, with likelihood, magnitude and compensating controls written out; the Controller decides.
Deficiencies proposed against the bank's severity guide, decided by the Controller
In the sample quarter that came to 349 of 350 sample items passing across 14 key controls, one timing exception decided with a reason, and four deficiencies in the year, none above a control deficiency. Those are results on fictional data and show what the output looks like, not what a bank will find. How the testing compares with sampling by hand on the same quarter is set out in AI control testing vs sampling by hand.
Questions people ask
How long does it take to automate control testing?
For regulatory reporting controls at a community bank, plan on about six weeks to set up and baseline, then one parallel quarter and one live quarter: roughly seven months before the old process stops. Most of the first six weeks goes on the control library and the written rules, not on technology.
What sample size should we use for control testing?
Part 363, the Call Report instructions and the OCC's booklet do not fix one for management's own testing; the bank sets a rule by control frequency and applies it consistently. Many risks shrink once monthly and quarterly controls are tested on every instance, which automation makes cheap, leaving samples for daily and per-transaction controls.
Can we start control testing automation mid-year?
Yes, and you should rather than wait for January. The assessment is as of year-end, and evidence that controls ran in the second and third quarters strengthens it. Re-running a quarter already filed is a cheap way to start.
Does the external auditor have to approve the tool?
No approval is required, but the auditor decides how far to rely on the results. Tell them before year-end what the tool tests, how samples are drawn and what evidence it keeps, and ask what they would want to see.
Should internal audit use the same tool?
It can read the results, but it should test with its own samples and record its own conclusions. The OCC asks examiners to consider the independence of the people performing reviews, and internal audit's value lies in being outside management's testing.