A risk based audit approach directs an audit's effort to the areas, units and items where the risk of an error or breach is highest, instead of spreading the same tests evenly over everything. The auditor first judges how likely each area is to go wrong and how well the auditee's own controls would catch it, then decides what to test, how much and when. In a delegated authority audit the approach works at three levels: which delegates to audit first, which parts of each delegate's business to focus on, and which policy and claim files to open. The third level is where the data now does most of the work. Botminds' MGA & Coverholder Audit is an agentic solution that scores every risk and claim a delegate reports, picks the files most likely to fail plus a random set, and writes down why each was chosen; the auditor reviews the choice, rates each finding and signs the report.
The examples come from the solution's fictional sample audit of Bayline Coastal Underwriters, a commercial property MGA, by its carrier Harrowgate Specialty. The audit-risk framework is the PCAOB's, written for financial statement auditors; it is used here for its logic, not as a rule for delegate audits.
One risk based delegate audit: what agents prepare, what people decide
Agent
Whole-book testerTests every risk and claim against the binder’s limits and lists each exception with its record and clause.
Never: Clears an exception
Agent
Risk-weighted samplerScores every record on the carrier’s weights, takes the exceptions that need a file, then the highest scores, then random picks from a recorded seed.
Never: Drops the random picks
A person decides
DA auditorReviews the sample memo, adds or removes files with a reason; the sample locks when fieldwork starts.
Agent
File testerRuns each file’s test steps against the documents and cites the page for every result.
Never: Passes a step without a cited passage
Agent
Report drafterDrafts each finding and proposes a rating from the carrier’s matrix, marking repeats.
Never: Issues the report
A person decides
DA auditor and head of DAThe auditor sets every rating and signs; High findings go to the head of delegated authority before the delegate sees them.
What is a risk based audit approach?
It is an approach that sizes the audit to the risk: more work where errors are likely and costly, less where they are not, with the reasoning written down. The PCAOB's sampling standard expresses the idea as a model in which the overall audit risk is the product of four risks: the inherent risk that something goes wrong, the control risk that the auditee's controls do not catch it, the risk that analytical procedures and other substantive tests miss it, and the risk that the detailed tests miss it. "AR = IR x CR x AP x TD", in the standard's notation (PCAOB AS 2315, appendix).
The standard adds a caution worth keeping: "The model is not intended to be a mathematical formula including all factors that may influence the determination of individual risk components". It is a way of thinking about where assurance comes from. For a delegate audit, its most useful lesson is that the detailed file tests are the last line, not the only one, and that other tests can carry much of the load.
The audit risk model, read for a delegate audit
Inherent risk
IR: how likely a breach is
Classes and territories with high exposure, such as Tier 1 wind counties; new lines; rapid growth; complex limits.
Set by the auditor from the binder and the book
Control risk
CR: will the delegate’s controls catch it
Referral logs, sanctions screening before bind, authority checks in the binding system; weak where last audit found them weak.
Judged from the last audit and the visit
Other substantive tests
AP: will whole-book tests miss it
Field tests on every record of the bordereaux: limits, territories, dates, payments against authority.
Agents run them; the auditor reviews each exception
Tests of detail
TD: will the file tests miss it
The sampled files, each step against a cited page. Smaller and better aimed when the other three are known.
Agents test; the auditor judges
Risk based audit selection criteria: how the files are chosen
Files are chosen for the signals that predict a breach, with some files taken in full regardless of score and some taken at random regardless of score. The PCAOB states the first rule plainly: "The auditor should examine those items for which, in his judgment, acceptance of some sampling risk is not justified" (AS 2315.21). In a delegate audit those are the records a whole-book test has already flagged and that only the file can settle: a risk over a limit, a claim paid over authority. The remaining picks are scored.
The table shows the weights in the fictional sample audit. They are the carrier's settings, not the solution's: a carrier can try other weights on the sample screen before the sample locks, and the memo records whichever were used.
Risk based audit selection criteria in the sample audit, and why each predicts a breach
| Signal | Weight | Why it points at a breach |
|---|---|---|
| A whole-book exception that needs the file | 40 | The data already shows a break; only the file says whether it was approved, referred or explained |
| Within 10% of a binder limit | 15 | Values near a limit are where a limit is most often misread or rounded |
| A process raised at the last audit | 15 | A closed action is a promise; the file shows whether the fix held |
| Tier 1 wind county or parish | 10 | Concentrated catastrophe exposure, where aggregates and pricing matter most |
| Top 5% of the book by size | 10 | The largest risks and claims carry the most money per error |
| Bound or paid in the last three days of a month | 5 | Month-end pressure, when screening and referrals are most often done after the fact |
| Underwriter or adjuster new since the last audit | 5 | New staff, new habits |
| Linked to another sampled record | 5 | A claim on a sampled policy, or a policy behind a sampled claim, tests both halves at once |
Take one file to see the arithmetic. Palmetto Shores Condominium Association, a six-storey condominium in Gulf Shores, Alabama, scores 70 out of 100, rank 4 of 2,318 risks: 40 for a whole-book exception (its insured value of $17.4 million is 16% over the $15 million per-location limit), 10 for a Tier 1 wind county, 10 for being in the top 5% of the book by premium, 5 for an underwriter who joined in May, and 5 for a linked claim in the sample. The file then fails its second test step: the referral log for June has no entry for it.
Why this file is in the sample, and what it proves
What is a risk based audit plan, one level up?
A risk based audit plan decides which delegates get audited when, and with what focus, before any file is chosen. The same logic applies at that level: premium at stake, last opinion, open actions, repeat findings, new delegates and lines, and how the delegate's bordereaux have looked since the last visit. Where the law sets a minimum clock, such as twice a year for each MGA under the NAIC model, risk decides the order, the depth and the extra visits, not whether a delegate is reviewed at all. The rule state by state is in the MGA compliance audit guide.
Regulators are asking for exactly this kind of steering. The PRA's 2026 priorities for insurers expect delegated authority business to grow in the London Market and warn that, without proper oversight, the growth adds to the risk of inadequate pricing and reserving (Prudential Regulation Authority, Insurance supervision: 2026 priorities). An audit plan that ranks delegates on fresh data from their bordereaux is one answer to that.
Risk based audit methodology for one delegate, in six steps
The methodology for a single delegate audit runs in six steps, and the order matters more than the tooling. Understand the delegate: the binder, its schedules and endorsements, the classes and territories written, staff changes, and the last audit's findings and actions. Assess risk by area: where breaches are most likely (inherent risk) and where the delegate's controls are weakest (control risk), area by area, from underwriting authority to premium remittance. Test the whole book where the data allows: every field the bordereaux carry, against every limit the binder sets, so the auditor knows where the exceptions are. Choose the files: the exceptions only a file can settle, then the highest risk scores, then random picks, with each choice recorded. Test and evaluate: each file against its documents; if exceptions are more frequent than the risk assessment assumed, revise it and extend the testing, as the PCAOB's sampling standard expects when results contradict the planning assumptions. Report and follow up: rated findings that cite their clauses and files, actions with owners and dates, and the next audit's risk assessment starting from what this one found.
The fifth step is the one most often skipped. A risk based approach that never revises its assessment when the files disagree with it is a fixed sample with a better story.
Can AI select audit samples?
Yes, provided the selection is explainable, recorded and owned by the auditor. A model that picks files without saying why cannot be defended to internal audit or a regulator; one that writes each file's reasons, keeps random picks beside its choices, and locks the sample when fieldwork starts can be. The responsibility does not move with the work. EIOPA's opinion on AI governance is explicit that the insurer remains ultimately responsible for the AI systems it uses, including those a third party built (EIOPA Opinion on AI governance and risk management, August 2025).
US insurers face the same expectation through the NAIC's model bulletin on AI systems, which many jurisdictions have adopted and which asks insurers to govern AI they buy as well as AI they build; what it requires is set out in the NAIC AI bulletin requirements guide. For an audit, the practical test is simple: could the auditor explain to an examiner why each file was chosen, and show that a file outside the model's view still had a chance of being picked?
The line between agents and the auditor, task by task
The line falls between preparing evidence and deciding what it means. Every task that can be checked against a document or a rule can be done by an agent, provided it cites its source; every task that weighs one consideration against another, or commits the carrier to something, stays with a person. The table draws the line for one audit.
Who does each task in a risk based delegate audit
| Task | Agents | The auditor | Why the line is there |
|---|---|---|---|
| Build and send the information request | Draft from the scope and binder; chase; check each item is readable | Approve before it is sent | Only the auditor can agree scope with the delegate |
| Test the bordereaux | Run every test on every record; list exceptions with record and clause | Clear an exception, with a reason | Clearing is a judgement that something is acceptable |
| Choose the files | Score, select, write each file’s reasons | Add or remove files with a reason; the sample then locks | The sample is the auditor’s; the memo shows any change |
| Test each file | Run each step against a cited page; mark a missing document as missing | Override a step, with a reason | A missing page is a fact; whether it matters is a judgement |
| Write the findings | Draft condition, criteria, effect, recommendation; propose a rating; mark repeats | Set the rating; edit the finding | Severity weighs context the data does not hold |
| Issue the report | Assemble it | Sign it; High findings approved by the head of delegated authority first | The opinion is the carrier’s, given by a named person |
| Follow the actions | Chase owners; review closure evidence; reopen when the exception returns | Accept closure | Closing an action says the risk is gone |
Who signs an audit report?
A named person at the carrier: in the sample audit, the senior delegated authority auditor signs and the head of delegated authority approves High findings before the draft reaches the delegate. The signature is the point at which the carrier satisfies itself, which is what regulators ask of it. When an attorney asked New York's insurance department what its semi-annual MGA review must include, the department's counsel did not give a list. The answer was a standard.
[T]he review must be sufficient to satisfy the insurer that the MGA is acting in a prudent and proper manner, and that the insurer does not relinquish its responsibilities.
New York Office of General Counsel, opinion 07-10-08, October 2007, on the semi-annual MGA review
A risk based approach helps meet that standard only if the insurer can show its reasoning. A sample memo that records the weights, the random seed and each file's reasons does that; so does a finding that cites its clause and its files. An agent that drafted the finding does not weaken the signature. A signature on a finding nobody can trace would.
Three people the risk based approach has to satisfy
Today
Picks thirty files by experience and cannot show the files left out.
With agents
Starts from the scored book and the reasons for each pick; spends the time on the failures and the visit.
Still theirs: The sample, every rating, the signature
Today
Takes the sample on trust and re-performs a few files.
With agents
Reads the weights, the seed and each file’s reasons; can re-run the draw.
Still theirs: Its independent view of the method
Today
Sees a rating, not the pattern behind it.
With agents
Approves High findings with the repeat history and the evidence one click away.
Still theirs: Approval of High findings; renewal decisions
Where a risk based approach goes wrong
It goes wrong when the model's view of risk becomes the only view. A model weighted on prior findings keeps looking where it has looked before; a new delegate has no history to score; and some breaches never appear in the data at all. Three guards address those. Keep random picks, drawn from a recorded seed, so every file has a chance. Test a new delegate's whole book early, before its history exists. And keep at least one test outside the delegate's own reporting, such as premium in the bank against premium on the bordereau, because a risk score built on a false bordereau ranks the wrong files with confidence.
The other failure is cosmetic risk-weighting: a "risk based" label on a sample that is in practice whatever was easy to pull. The cure is the same memo that internal audit reads. How a random sample compares, and what it misses, is in the audit sampling methods guide; how many random picks are enough is in the audit sample size calculator.
The finding is drafted; the rating is the auditor's
The claim's score explains why it was picked: the whole-book test had already found the payment over authority.
Questions people ask
What is the difference between risk based auditing and compliance auditing?
A compliance audit checks conformity with rules, often against a fixed list; a risk based audit decides where to look, and how hard, from an assessment of what could go wrong. A good delegate audit is both: the rules from the binder and statute are the tests, and risk decides which delegates, areas and files get the most attention.
What are the steps in a risk based audit approach?
Understand the auditee and its rules; assess inherent and control risk by area; decide which tests to run on the whole population and which on samples; choose the sample, recording why; test; evaluate the results against the risk assessment; report and follow up. If the results show more errors than expected, the risk assessment is revised and testing extended.
Does a risk based sample replace random sampling?
It should not. Random picks let every file have a chance of selection and test whether the risk model is blind somewhere. In the sample audit, 6 of the 36 files are random, drawn from a recorded seed, and found no exceptions, which supports the whole-book tests' picture of the rest of the book.
Can a risk score be wrong?
Yes. It ranks files on the signals it is given, so it misses breaches nobody anticipated and anything absent from the delegate's data. That is why the scores are the carrier's own settings, the reasons are recorded, and the auditor can add files the model did not choose.
Who is responsible if an AI tool picks the wrong files?
The insurer. Regulators in Europe and the US expect insurers to remain responsible for AI systems they use, including ones bought from vendors. In practice that responsibility runs through the auditor who reviews and locks the sample.
How often should the risk weights be reviewed?
At least once a year, and after any audit where a breach was found outside the sample. A weight that never moves the selection, or a signal that never coincides with a breach, is a candidate for removal; a breach the weights missed is a candidate for a new signal.