SOX control testing where you talk to the agents, see the evidence behind every cell and sign the workpaper yourself
Every sample cited, re-performed and signed before your auditor asks.
See one case, screen by screen ↓SOX control testing is the test of whether a key control over financial reporting is designed and operating effectively: a population is tied out, a sample drawn, each attribute checked against evidence, exceptions evaluated and the result documented in a signed workpaper. SOX Control Testing adds seven agents that sample, test, re-perform blind, sweep the whole population when a sample fails and draft the cited workpaper, while the SOX tester and the program manager decide and sign.
How to choose AI solutions for back-office and enterprise work →
Updated
A SOX programme tests hundreds of key controls a year, and most of them are still tested by hand. The tester asks the ERP team for a population report, shows it is complete and accurate, picks a sample from the methodology table, emails control owners for evidence, then ticks five or six attributes per item in a spreadsheet with tick marks and cross-references. The evidence comes back as screenshots, PDFs and exports — often the wrong period or the wrong version — and the chasing takes days.
The risk sits in what the sample cannot tell you. One exception in 25 items says nothing about the other 1,821, so teams extrapolate and either under-classify the deficiency, until the auditor finds more, or over-classify it. The manager's review is a re-read of the same evidence, and review notes go back and forth for days. Where the external auditor cannot rely on internal audit's work, it tests the control again, and the cost shows up in the fee and in late Q4 surprises.
Estimated split of one test of operating effectiveness, by hand and with the solution. The 16-hour total is the published average; the ≈ 3-hour total is the design target.
Seven agents sample, test, re-perform blind and sweep the population; the tester and the programme manager sign.
Kestrel & Vane Group is testing P2P-07 for its FY2026 interim period, 1 January to 30 September: invoices of $50,000 or more must be approved in the ERP workflow by someone whose delegated limit covers the amount, before payment is released. Lena Marsh is the tester, Tomás Ferreira the reviewer. Here is how the work goes, screen by screen, in the working solution.
Interim testing closes in 16 days and Lena has 18 controls this cycle. Her home screen lists what only she can do — P2P-07 has 1 exception and 1 disagreement, O2C-12 has a review note from Tomás, R2R-04 has 2 evidence requests 3 days overdue — and what the agents did overnight, step by step, from the sampling agent's re-run of the population at 06:21 to the drafted workpaper at 07:11.
Overnight, the agents: “Deficiency agent swept 1,846 invoices for attribute C: 4 found, $251,470; drafted DEF-26-014.”
Lena asks the assistant. The answer names the invoice — INV-2026-118463, Brightwell Industrial Supply BV, $61,240.00 — the approver, Owen Halloran, Plant Finance Manager, Tilburg, and his $25,000 single-invoice limit under DoA matrix v7, §4.2 row 9. It says the reviewer agent re-performed the cell blind and agrees, points to the workflow change behind it, cites both sources and opens the cell beside the answer.
“The cause is workflow change CHG-26-0412 of 3 Jun. I have opened the cell.”
Attributes A to E run across the top — approval present, approver on the DoA matrix, amount within the limit, approved before payment, evidence complete — and the 25 sampled invoices run down the side. Each of the 125 cells carries its one-line evidence snippet. Above the grid: a population of 1,846 invoices, $214.6M, tied to the ledger; a random sample with seed 20261002; 24 pass and 1 exception; 24 of 25 re-performed cells agree; 4 of 1,846 with the same break.
The sampling agent tied report ZAP_INV_APPR to the AP subledger, account 2100, with a $0.00 difference before it drew the 25.
Opening the cell shows the evidence the agent read, with the figures highlighted: the invoice total of $61,240.00 and the approver's limit of $25,000, over by $36,240.00. Below it are the rule (DoA matrix v7 §4.2), the testing agent's reasoning at confidence 0.98 and its sources. The approval came from the workflow, not a manual override. The reviewer agent, working from the source documents on its own, reached the same result.
Cells below the 0.90 confidence floor always go to a person.
The reviewer agent re-performed samples 3, 9, 17, 21 and 24 — every exception, topped up at random to 20% of the sample — before it could see the testing agent's answers. It agrees on 24 of 25 cells. On sample 21, attribute E, the testing agent said pass at 0.96; the reviewer agent says not complete at 0.94: the invoice cites PO 4500-88213 version 3, issued 2 Jul 2026 with a price change, and the file holds version 2. Both reasonings sit side by side, and the cell waits for Lena.
“The reviewer agent never resolves its own disagreement. Whatever you choose is recorded with your name and reason.”
Lena chooses “Request PO version 3 from Hannah Brooke's team”. Request REQ-2026-0931 goes to AP Shared Services, the PO arrives, and the evidence agent and the reviewer agent re-check version 3. The cell passes, marked “Resolved by Lena Marsh · PO 4500-88213 v3 received 14 Oct, reviewer agent re-checked”, and 25 of 25 re-performed cells now agree.
“PO v3 received and re-checked · 25 of 25 cells agree.”
The deficiency agent checked attribute C on all 1,846 invoices: 4 were approved above the limit, $251,470 in total, all by the same approver for cost centre 4410, from 22 Jun to 26 Sep. All four are three-way matched, goods received, prices per contract — no misstatement. It traced the cause to change CHG-26-0412 of 3 Jun 2026, which routed tier 2 approvals for the Tilburg plant to a role group without checking its limit, and opened a question on IT change control ITGC-04.
Drafted evaluation: potential exposure $2.71M, 8.6% of $31.5M performance materiality; compensating control P2P-11 tested effective.
DEF-26-014 is drafted with condition, criteria, cause and effect, the sweep, the root cause timeline and the AS 2201 evaluation. The deficiency agent proposes a control deficiency, not significant, at confidence 0.88, and says why. Only the SOX program manager can classify: Tomás picks reason code CL-01, agree with the proposed classification, writes his note and signs. Remediation goes to the control owner: correct the routing, retrospective approval by the Division CFO, retest 30 days after the fix.
The deficiency agent never classifies a deficiency. A changed classification needs its own reason code.
The workpaper writer drafted TW-P2P-07-FY26 with 9 sections and 31 citations; Tomás's review note on section 5 asked how the reviewer agent picked the five samples. With every cell resolved, Lena signs as preparer: 24 of 25 pass with 1 exception, 25 of 25 cells agree on re-performance, DEF-26-014 proposed as a control deficiency. Tomás then signs as reviewer — the preparer and the reviewer must be different people.
“I performed or reviewed every attribute, the evidence supports each result, and the conclusion reflects the test.”
Ravi asks how completeness of the P2P-07 population was established and whether report parameters were checked. The workpaper writer drafts the answer from the record: report ZAP_INV_APPR run on 1 Oct 06:02, its $214,612,388.40 agreeing to account 2100 with no difference, 1,846 rows agreeing to the ERP document count, five rows traced, parameters checked against the control description. Lena sends it. Ashdown Pryce plans to rely on 248 of 412 key controls.
The workpaper writer never sends anything to the external auditor.
The same test of P2P-07, seen by the four people who carry it — what their cycle looked like, and what it looks like now.
Seven agents sample, test, re-perform blind and sweep the population; the tester and the programme manager sign.
Rolls forward the risk and control matrix, finds design changes in the year's change tickets and policy updates, and proposes the test plan — new, changed and retired key controls, each with its source.
Runs the population report with the control's parameters, ties its count and total to the ledger, traces five rows to source, picks the sample size from the methodology table and draws at random with a recorded seed.
Sends evidence requests to control owners, reads what comes back and checks the period, version, signature and completeness of every file.
Checks every attribute of every sample against the evidence and records pass or exception with the passage it relied on, its reasoning and a confidence.
Re-performs every exception, topped up at random to 20% of the sample and at least 5 items, working from the same sources independently, and says so on the grid when it reaches a different answer.
Sweeps the whole population for the failed attribute, sizes the exposure, traces the root cause to the system event behind it and drafts condition, criteria, cause, effect and the likelihood and magnitude evaluation.
Drafts the workpaper and the answers to the external auditor's questions, with a citation behind every claim.
Preparer and reviewer sign. The agents propose; a named person decides.
Ask in plain words, or tell the assistant what to do next. Every answer cites its sources and opens the right place in the test.
Why did sample 17 fail?
Sample 17 is INV-2026-118463, Brightwell Industrial Supply BV, $61,240.00. It was approved on 15 Jul 09:12 by Owen Halloran, Plant Finance Manager, Tilburg, whose single-invoice limit is $25,000 under the DoA matrix v7, §4.2 row 9. The reviewer agent re-performed it blind and agrees. The cause is workflow change CHG-26-0412 of 3 Jun.
Is this exception isolated or systemic?
Not isolated, but contained. The deficiency agent checked attribute C on all 1,846 invoices: 4 were approved above the limit, $251,470 in total, all by the same approver, all for cost centre 4410, from 22 Jun to 26 Sep. None is misstated: all four are three-way matched. Potential exposure is $2.71M, 8.6% of performance materiality.
Which other controls depend on the same approval workflow?
Three: P2P-11, the monthly review of invoices over $50,000, which is the compensating control and tested effective; ITGC-04, changes to production, which let CHG-26-0412 through (evidence request REQ-2026-0929 is with Mateo Silva); and P2P-06, purchase-order approval, which uses the same role groups.
Draft the status note for the audit committee
Interim testing is 69% complete (286 of 412) against a 70% plan. 14 exceptions on 12 controls, each followed by a sweep of its population. 7 control deficiencies, none significant; DEF-26-014 covers 4 invoices, $251,470, approved above limit after a workflow change, no misstatement. Ashdown Pryce plans to rely on 248 key controls. It is in Reports for the CAE to approve.
13 screens from the working solution, on its sample data. Pick one to see it large.
The SOX program manager sets how the programme runs. Testers see the settings read-only, and every change is versioned and on the audit trail.
| Setting | Default | Choose from |
|---|---|---|
| Sample sizes (methodology v4.2, table 3) | Many times a day: 25 at low or moderate risk, 60 at high risk | Annual 1 / 1 · quarterly 2 / 2 · monthly 2 / 5 · weekly 5 / 15 · daily 20 / 40 · many times a day 25 / 60, each editable |
| Selection method | Random with recorded seed | Haphazard selection is not allowed; a re-draw needs reason code RD-01 or RD-02 |
| Re-performance rate | Every exception, topped up at random to 20% of samples, at least 5 | Set by the SOX program manager |
| Sweep the whole population when a sample fails | On | Set by the SOX program manager |
| Reviewer agent works blind | On | Cannot be switched off |
| Testing agent confidence floor | 0.90 | Below it, the cell goes to a person |
| Materiality | Overall $42.0M · performance $31.5M · clearly trivial $2.1M | Set each year, agreed with the external auditor |
| Evidence SLA | 5 business days, escalation on day 8 | Reminders on days 5 and 7; escalation to the control owner's manager |
Every figure is labelled: a target the solution is built to, an estimate, a typical published result, or a proven one.
“demo” = seen in the working solution, on its sample data · “target” = the design goal, measured in the live solution · “estimated” = our estimate · “typical” = published figures (KPMG, The 2025 SOX Survey — 16 hours per control for the test of operating effectiveness, up from 12, and an average programme cost of $2.3M, up from $1.6M; KPMG, Trends in material weaknesses — 238 of 3,307 annual reports by mid-2025). PCAOB AS 2201 and AS 2605 are named as context only. People, companies, suppliers and figures named on this page are fictional — characters and sample data in the working solution.
SOX control testing is the test of whether a key control over financial reporting is designed and operating effectively: a population is tied out, a sample drawn, each attribute checked against evidence, exceptions evaluated and the result documented in a signed workpaper. SOX Control Testing adds seven agents that sample, test, re-perform blind, sweep the whole population when a sample fails and draft the cited workpaper, while the SOX tester and the program manager decide and sign.
The sampling agent ties the population to the ledger and draws the sample from your methodology table with a recorded seed. The evidence agent requests and checks the evidence, and the testing agent checks every attribute of every sample, pinning the passage it relied on to the cell with its reasoning and confidence. You see the result as one grid, and can ask about any cell in plain words.
The deficiency agent checks the same attribute across the whole population — in the demo, 1,846 invoices rather than 25 — sizes the exposure, dates the first and last occurrence and traces the root cause to the system change behind it. It then drafts condition, criteria, cause, effect and the likelihood and magnitude evaluation for a person to decide.
A second agent, the reviewer agent, re-performs every exception and a random top-up to 20% of the sample, at least five items, without seeing the testing agent's answers. Where it disagrees, both reasonings are shown side by side and the cell goes to the tester; the reviewer agent never resolves its own disagreement.
No. The deficiency agent proposes a classification with its reasons. The tester submits the deficiency, and only the SOX program manager can classify it as a control deficiency, significant deficiency or material weakness, with a reason code and a signature.
The auditor opens shared workpapers read-only, sees the reliance checks for each one — objectivity, population tested, every cell cited, re-performance, preparer and reviewer signatures — re-performs any cell in the browser and asks questions that are answered with citations and sent by a person. How far to rely on internal audit's work stays the auditor's decision.
Yes. Sample sizes by frequency and risk, the selection method, the re-performance rate, the confidence floor, materiality, evidence SLAs, reason codes, roles and notifications are settings that the SOX program manager edits. Changes are versioned and on the audit trail.
Yes. Agents never conclude a test, sign, classify a deficiency or send anything to the external auditor. Cells below the confidence floor go to a person, every override needs a reason code, the tester signs as preparer only when every cell is resolved, and a different person signs as reviewer.
The Agentic Solution Engine builds and deploys it from your requirements and documents — your risk and control matrix, testing methodology, delegation of authority and a past workpaper — and it goes live once every quality gate has passed. We will walk you through it on one of your own key controls first.
Evidence checked before the auditor sees it, and the request list closed before fieldwork.
1-min demoContinuous Controls MonitoringControl failures found in a day, not at year-end, and every fix proved by a re-test.
1-min demoJournal Entry TestingEvery entry tested for override, not a sample, with each conclusion cited to its support.
1-min demoWe'll run SOX Control Testing on one of your own key controls, with its population and last year's workpaper.