Questionnaire automation is the use of software to answer recurring questionnaires from an approved answer library: it reads each incoming question, finds the closest answer already given, updates it and routes it to the person who must approve it. For an insurer the questionnaires that matter are rating agency reviews, state examination request lists and auditor support lists, and the people who must approve are senior: the chief underwriting officer, the chief reserving actuary, the chief risk officer, the CFO. Their trust is the whole return. If owners rewrite every draft, the software has added a step; if they sign drafts they have not read, it has added a risk. Botminds' Rating Agency & Exam Response is built for the middle path: every figure shows its filed source and what the last answer said, every reused paragraph shows who approved it and when, and owners approve paragraph by paragraph. This guide is about getting from installation to that point in two questionnaire cycles.
What an owner sees on one sample answer
10
figures in one drafted answer, each tied to a filed source, page and line
91%
how far the question matched the answer approved and sent in May
2
figures changed since the last answer, marked for the owner
1
paragraph with no earlier answer, drafted from a board pack and labelled new
What is questionnaire automation, for an insurer?
For an insurer, questionnaire automation is three jobs in one: reuse approved wording, refresh every figure from the filed statements, and check the result against what every other audience has been told. Generic tools do the first. The second and third are what make an insurer's questionnaires different, because the answers are mostly statutory figures that change every quarter and the readers, rating agencies and examiners, compare them. The cost of one questionnaire shows that those two figure steps hold about half the hours.
The output is a draft, never a submission. The product decision that follows from that, and the subject of this guide, is how owners come to treat the draft as their own work: reviewed, corrected where needed, and signed.
How do you get experts to trust automated answers?
By showing them the evidence they would have gathered themselves, and by making the draft easy to disagree with. Trust in automation fails in two directions, and the U.S. National Institute of Standards and Technology describes both in its generative AI profile: experts who distrust the system on principle, and users who, over time, lean on it and rate its output above other sources, which NIST calls automation bias.
Humans bring their unique perspectives, experiences, or domain-specific expertise to interactions with AI systems but may not have detailed knowledge of AI systems and how they work.
NIST AI 600-1, Generative Artificial Intelligence Profile, July 2024, §2.7
From there NIST draws its other warning: human experts may be unnecessarily "averse" to these systems and so give up their benefits. In a reserving or underwriting team both happen, often in the same person over a year: sceptical in the first cycle, too comfortable by the third. A rollout has to address both, which means two kinds of evidence: proof that the draft is right where it is right (sources, matches, approvals), and friction where it might be wrong (flags that block approval until a person decides).
The American Academy of Actuaries' professionalism committee makes the same point in its own terms: actuaries remain responsible for the decision whether and how to rely on AI tools (American Academy of Actuaries, 2024). An owner who signs a drafted reserve answer is relying on it, and should be able to say why.
What evidence moves each topic owner?
Each owner has a different test, and it is usually the test they already apply to a junior colleague's draft. The roles below are the owners of a typical rating questionnaire; what each needs to see is our summary of what makes a draft signable.
The owners of a rating questionnaire, and the evidence each needs before signing a draft
Today
Receives the forty-page questionnaire and is asked for “your section”; rewrites last year’s paragraph on rate adequacy from the rate monitor.
With agents
Receives only their paragraphs, each figure with its source and the value the last answer used, and approves paragraph by paragraph.
Still theirs: what the company says about pricing, and on what basis
Today
Checks every reserve figure against Schedule P before signing, because drafts often carry board-paper numbers.
With agents
Sees each reserve figure named to its filed source and page; a figure that differs from Schedule P arrives flagged, with both statements quoted.
Still theirs: which reserve figure is the company’s position
Today
Pulls the same figures out of the annual statement, Schedule P and the RBC report for every request.
With agents
Maintains each key figure once, with its page and line, and sees every answer that uses it.
Still theirs: the filed figures themselves
Today
Rewrites risk management answers so they agree with the ORSA summary report.
With agents
Co-signs risk answers drafted from the approved ORSA language, with changes since the last answer marked.
Still theirs: risk appetite and stress results
Today
Asks, before signing, whether anything contradicts what the company has told anyone.
With agents
Signs only when every owner has approved and the consistency report is clean, with the report in the pack.
Still theirs: the signature and the release
Two owners deserve early attention. The chief reserving actuary is usually the most sceptical and the most consequential, because reserve figures are the ones most often restated between a board paper and Schedule P. The chief risk officer sees the whole risk picture; the NAIC's examiners' handbook advises examiners to interview the CRO as early as possible for that reason (NAIC Financial Condition Examiners Handbook, 2026, Exhibit Y), and a rollout can borrow the logic: win the CRO and the chief actuary first, and the other owners follow their lead.
How do you roll out questionnaire automation? Two cycles
Roll it out over two questionnaire cycles: a shadow cycle on a questionnaire you have already answered, then a live cycle in which owners approve real drafts. The plan below is our recommendation; the weeks are indicative and depend on how many past answers and filed statements are loaded.
Adoption over two cycles: shadow first, then live
- Load past answers, filed statements, topic owners2 wks
- Draft last year’s questionnaire again1 wk
- Owners compare drafts with what they sent2 wks
- Tune owners, tolerances, look-back1 wk
- First draft1 day
- Owners approve paragraph by paragraph2 wks
- Liaison reviews the whole1 wk
- CFO signs; answers filed for next year1 wk
- Review: what owners changed, and why2 days
The shadow cycle is the trust-building step, because owners can compare the draft with an answer they already signed. Every difference is informative: either the draft is wrong and the settings need tuning, or the original answer was wrong, which happens more often than teams expect when figures were copied by hand. The live cycle then starts from evidence rather than a demonstration.
What is a shadow cycle?
A shadow cycle runs the system on a questionnaire the company has already answered and sent, using the statements that were current at the time, without sending anything. Owners receive the drafts and mark each paragraph: same as sent, better than sent, or wrong. The liaison counts the marks by owner and by type of error.
Three things come out of it. A baseline: how many paragraphs owners would have signed as drafted. A settings list: owners missing for some topics, a tolerance that is too tight or too loose, a filed source the refresher was not allowed to use. And a list of past inconsistencies the company did not know it had, which is often the most persuasive result for a CFO.
How do owners review drafted answers?
Paragraph by paragraph, with the reasons on the page. In Rating Agency & Exam Response each paragraph shows which earlier answer it reused, who approved that answer and when, how similar the question was, and which figures were refreshed, confirmed or contradicted. Owners can edit the wording, and every edit is kept as a tracked revision; a paragraph the consistency guard has flagged cannot be approved until the flag is resolved.
A resolved flag: one number for every audience
Labelling matters more than polish. A paragraph marked "new, drafted from the second-quarter board pack" invites a careful read; one that looks like every other paragraph invites a skim. That is the practical defence against automation bias: make the risky parts look different.
Who should own each questionnaire topic?
One named person per topic, the same person across rating agencies, examiners and the auditor. Rating questionnaires and exam request lists overlap heavily, and an owner map that differs by audience produces two answers to one question. The NAIC's corporate governance disclosure regulation offers a ready list of the critical risk areas a board oversees, from risk management and the actuarial function to investments, reinsurance, compliance, financial reporting and market conduct (NAIC Model #306, § 5E); it makes a sensible first draft of a topic map.
Topic owners, tolerances and the sign-off chain as settings
Gaps are useful. In the sample, IT controls, cybersecurity and AI governance had no owner because they appear in the examiners' request list but not in rating questionnaires; the router left those items unassigned and told the liaison rather than guessing. A rollout should expect a few such gaps and decide them before the live cycle.
How do you measure adoption of questionnaire automation?
Measure what owners do with drafts, not how many people log in. The measures below come from the system's own history and need no survey.
Seven measures of adoption, read from the answer history
- Signed as drafted
Share of paragraphs approved without edits. Rising over two cycles is trust; 100% from the start is a warning sign of skimming.
- Edits by type
Wording changes versus figure changes. Figure changes point to settings or sources; wording changes are normal.
- Flags raised and how resolved
Contradictions found, and how many owners resolved with the recommended fix versus another choice.
- On-time sign-off
Owners approving by their date, which shows whether the drafts reach them in reviewable shape.
- Unassigned items
Questions that waited for the liaison because no owner existed, falling as the topic map fills in.
- First draft time
Days from receipt to a full draft, against the one-day design target.
- Follow-up questions after release
Agency or examiner questions about figures, the outcome measure that matters most.
When is questionnaire automation the wrong fit?
When the questionnaires are few and short, the figures rarely change, or no one owns the answers. A small insurer answering one brief questionnaire a year may do better with a well-kept document and a checklist. A team without named topic owners should fix that first, because a router cannot route to nobody. And security questionnaires and sales RFPs are a different job, better served by general answer-library tools.
Where it fits, the payoff compounds. Each signed answer becomes the next cycle's starting point, with its history, so the second year starts from a better library than the first. To try the shadow cycle on your own last questionnaire, start from Rating Agency & Exam Response; four measures of a draft gives the liaison a scorecard for the live cycle.
Questions people ask
How long does it take to roll out questionnaire automation?
Plan on two questionnaire cycles: a shadow cycle of about six weeks on a questionnaire you have already answered, then a live cycle on the next one. Most of the shadow time is loading past answers and filed statements and letting owners compare drafts with what they sent.
What training do topic owners need?
Very little on the software and more on the review habit: read the flagged and new paragraphs closely, check that each figure's source is the one you would have used, and edit rather than rewrite. An hour with their own answers from the shadow cycle teaches more than a general demonstration.
What if an owner disagrees with the recommended fix on a flag?
They choose another option or write their own, and the choice is recorded with the answer. The guard recommends, for example, the filed figure over a board-paper figure, but the decision belongs to the owner because only they know why a basis changed.
Should owners see the whole questionnaire?
Usually not. Owners review their own paragraphs, with the related questions from other audiences shown beside them; the liaison reads the whole submission, and the CFO reads the consistency report. Sending a forty-page questionnaire to every owner is a common reason drafts come back late.
How do you stop owners from rubber-stamping drafts?
Make the risky parts look different and make some decisions unavoidable. New paragraphs are labelled, changed figures are marked with the old value, and a flagged paragraph cannot be approved until someone resolves the flag. Watch the share of paragraphs signed without any edit; a sudden jump is worth a conversation.