Continuous auditing tools vs samples: one coverholder, three audits

Continuous auditing tools are software that extracts an auditee's data as it is produced, runs automated tests against rules on every record, and raises the exceptions for an auditor to investigate, so assurance arrives close to the event instead of at a periodic audit built on a sample. In delegated authority, the data is the delegate's bordereaux and files, the rules are the binding authority's limits, and the auditee is an MGA or coverholder writing on the insurer's paper. The choice for a carrier is not sampling versus technology but three practical approaches: an annual sample audit, the semi-annual on-site review most US MGA laws require, or whole-book testing of every risk and claim with a smaller, better-aimed sample of files. Botminds' MGA & Coverholder Audit is an agentic solution of the third kind: it tests every record on a delegate's bordereaux against the binder, then picks the files and drafts the findings for the auditor to sign.

The comparison below uses one fictional coverholder from the solution's sample data, Bayline Coastal Underwriters, a commercial property MGA with 2,318 risks and 412 claims in six months. The research on continuous auditing comes from the academic literature, linked where it is cited; the counts are from the sample audit, and the timings are our illustration.

What are continuous auditing tools?

Continuous auditing tools are systems that test populations automatically and repeatedly, so the auditor's attention goes to exceptions rather than to finding them. The idea is older than most of the software sold under the label. In 1991 Miklos Vasarhelyi and Fern Halper described a system built at AT&T Bell Laboratories for the internal audit organisation: "CPAS monitors key operational analytics, compares these with standards, and calls the auditor’s attention to any problems that may exist" (Vasarhelyi and Halper, "The Continuous Audit of Online Systems", reprinted in Continuous Auditing, Emerald, 2018).

The tools sold today fall into three groups: audit analytics packages that run scripted tests on extracted data, monitoring modules inside an organisation's own systems, and newer agent-based tools that also read documents, such as bordereaux in many layouts, binders and claim files, rather than only structured tables. For delegated authority the third matters, because the evidence that settles a delegate's exception usually sits in a document: the referral log, the signed application, the claim note.

Continuous auditing vs continuous monitoring

Continuous monitoring is management checking its own controls; continuous auditing is the independent check that those controls, and the results, are right. In a delegate relationship both exist. The delegate monitors its own binding system, and the insurer's delegated authority team audits what the delegate reports. The line blurs in practice: one of the best-known pilots, at Siemens, was a continuous auditing system whose job was "continuous monitoring of business process controls" running on top of the company's enterprise system with read-only access (Alles, Brennan, Kogan and Vasarhelyi, in Continuous Auditing, Emerald, 2018).

For a carrier the distinction is about ownership, not software. The delegate's monitoring is the delegate's control; the carrier cannot rely on it without testing it. A continuous tool owned by the carrier and run on the delegate's data is audit evidence the carrier can use.

What each audit would find in one coverholder's book

In the Bayline book, 60 of 2,730 records carry an exception against the binder. A random sample of 36 files can be expected to include 0.8 of them. The risk-weighted 36 that the solution drew after testing every record found exceptions in 12 files, and the whole-book tests themselves listed every exception with its record and clause before a single file was opened.

The difference is not that the sample auditor was careless. It is arithmetic: rare breaches are rarely drawn. What the sample audit does well, and the whole-book test cannot, is look at the files with judgement; the third approach keeps that and points it at the files that failed. Why random samples behave this way is set out in the audit sampling methods guide.

When would each audit see a breach?

A sample audit sees a breach at the next audit, if at all; a continuous approach sees it when the data that contains it arrives. Take one of Bayline's over-limit risks, bound on 14 June 2026 with no referral. Under an annual audit cycle, the earliest anyone at the carrier would see it is the next audit. Under the semi-annual review the sample audit used, it surfaces at the October visit, about four months later. Tested when the June bordereau lands, it surfaces within weeks, while the policy can still be referred, restructured or cancelled.

How continuous auditing works on bordereaux

It works in five steps. The tool reads each bordereau as the delegate sends it, in whatever layout, into one shape. It reads the limits from the binding authority and its endorsements. It runs each test on every row: territory, class, insured value per location, aggregates, screening date against bind date, issue date, remittance date, payment against settlement authority, reserve reviews. It lists every exception with the row and the clause. And it hands the auditor the exceptions, grouped by test, with the files that need opening.

Financial statement auditing has just written rules for this kind of work. The PCAOB's 2024 amendments on technology-assisted analysis, approved by the SEC and effective for audits of fiscal years beginning on or after 15 December 2025, "Specify auditor responsibilities when identifying items that require further investigation when performing tests of details" and "Specify auditor responsibilities for evaluating the reliability of external information provided by the company under audit" (SEC order approving the amendments, Federal Register, August 2024). Neither binds a delegated authority audit, but both name the two duties any continuous tool leaves with the auditor: deciding what each exception means, and judging whether the data the auditee provided can be trusted.

What continuous testing of the bordereaux cannot see

It cannot see anything the delegate left out or falsified, because it tests what the delegate reports. The Markel case shows how sharp that limit is. Before the insurer's audit, a colleague saw the bond files being amended under the direction of one of the men running the business; asked about it, that man said he was bringing the files into line with the bordereaux (Markel v Surety Guarantee Consultants, [2008] EWHC 1135 (Comm)). The bordereaux understated the insurer's exposure on bonds written over the limits, and the files had been made to agree. A whole-book test of those bordereaux would have returned a clean book.

The PCAOB counts this among nonsampling risks: an auditor can apply a procedure to every item and still miss a material misstatement (AS 2315.11). The remedy is to bring in data the delegate does not author. Premium received in the trust or fiduciary account against premium declared on the bordereau. Documents actually issued against risks declared. Claim payments in the bank against payments on the claims bordereau. A continuous tool that cannot ingest those is testing the delegate's story against itself.

making sure that the files tied up with the bordereaux

The explanation given for amending the bond files before the insurer's audit, as recorded in Markel v Surety Guarantee Consultants, High Court, 2008

Alarm floods, and the judgement that cannot be written as a rule

The most common practical failure of continuous auditing is not missed exceptions but too many of them. The Siemens pilot's authors put it among their key findings: they identified "the management of audit alarms and the prevention of the alarm floods as critical tasks" and found that "“formalizability” of audit procedures and audit judgment is grossly underestimated" (Alles, Brennan, Kogan and Vasarhelyi, 2018). An alarm on every late screening, every rounding difference and every expired endorsement buries the three breaches that matter.

Two design choices keep alarms useful in a delegate audit. Exceptions are grouped by test and theme, so 37 late sanctions screens arrive as one pattern with 37 rows rather than 37 alerts, and a theme seen at earlier audits is flagged as a repeat. And clearing is a person's act: an exception explained by an approved referral or a corrected bordereau line is cleared with the reason beside it, not filtered out by a rule nobody reviews.

Fourteen tests on every risk and claim, grouped so the patterns show

Exceptions are never cleared by an agent: the bar and tavern risk is shown as cleared because a person recorded the approved referral.

Is continuous auditing worth the cost?

Not automatically, and the field's own researchers say so. Writing on the economics of continuous assurance, Alles, Kogan and Vasarhelyi argued that "the key driver of CA is the demand for it" and that "there is no guarantee that CA is either cost effective—the only way of enhancing efficiency—or actually has to be continuous" (Alles, Kogan and Vasarhelyi, "Feasibility and Economics of Continuous Assurance", in Continuous Auditing, Emerald, 2018). A delegate whose business changes little month to month does not need minute-by-minute testing.

What the delegated authority case does need is every record tested at least once per audit period, because the breaches that matter are rare and the law already demands a review twice a year in most US states (NAIC Model #225, §5C). The cost question is then simple to frame: auditor days per delegate with and without whole-book testing, times the number of delegates. What one coverholder audit costs works through that sum term by term.

What to look for in continuous auditing tools for delegated authority

Ask what the tool tests, against what, and who decides. A demonstration on the vendor's sample data shows the interface; a test on one of your own delegates' closed periods shows whether it finds what your last audit found, and what it missed. The questions below separate tools built for delegate audits from general analytics with an insurance label.

Where Botminds' MGA & Coverholder Audit fits, and where it does not

It fits the third approach: every risk and claim on each period's bordereaux tested against the binder, a risk-weighted sample with random picks, file tests that cite their pages, findings drafted for the auditor, and actions tracked until the exception stops coming back. In the fictional Bayline audit, 14 tests ran on 2,730 records in 11 minutes, and the sample of 36 was drawn from the results that evening. The whole book can be re-run when new bordereaux arrive, and a later run that finds an exception an action was meant to fix reopens the action.

It does not do the visit, decide ratings, clear exceptions or close actions; people do. It does not make a false bordereau true: its premium and remittance tests use the trust account statements the delegate provides, and a carrier that suspects the delegate's own statements needs evidence from the bank or other third parties. And like any continuous tool, it is worth its cost only where the testing it replaces was real work. For most carriers with more than a handful of delegates, the weeks per audit suggest it was. A practical way to find out is the eight-week plan in the MGA audit checklist, run on one delegate first.

Questions people ask

What is continuous auditing?

Continuous auditing is auditing in which tests run automatically on complete populations of data, soon after the data is produced, and the auditor works from the exceptions. The concept goes back at least to a 1991 paper on a system built at AT&T Bell Laboratories; the 2018 Emerald volume Continuous Auditing collects the research since.

What are continuous auditing techniques?

The usual ones are rule-based tests on every record, reconciliations between independent data sources, analytics that compare results with expectations, and alerts routed to an auditor. In delegated authority the most valuable is reconciliation against data the delegate does not author, such as bank statements against bordereaux.

Can delegated authority be audited continuously?

The bordereau tests can: each month's bordereau can be tested against the binder when it lands. File-level judgement, interviews and the statutory on-site review still happen at intervals; continuous testing changes what the auditor brings to them, not whether they happen.

Does continuous auditing replace the on-site MGA review?

No. US MGA laws based on the NAIC model require an on-site review of underwriting and claims operations at set intervals. Continuous testing makes that visit better aimed, because the auditor arrives knowing where the exceptions are.

Is continuous auditing more expensive than sampling?

It moves the cost. Setting up the data and the tests costs more at the start; each audit after that costs fewer auditor days. Whether it pays depends on how many delegates and records there are, which is why the field's own researchers warn that continuous assurance is not automatically cost effective.

What is the difference between continuous auditing software and audit management software?

Audit management software organises the audit: plans, workpapers, findings and actions. Continuous auditing software produces evidence by testing data. A delegated authority team needs both functions; the question is whether they share one record of each test, file and finding.