ChatGPT can be used in insurance underwriting for drafting, summarising and organising, but an answer from a general chatbot to "does this risk fit our binding authority?" should not be relied on until it passes four tests: it cites the clause, it reads the clause correctly, it uses the binder as amended on the effective date, and it tests every condition at every location without assuming a missing detail. The research gives reason for care. A peer-reviewed study of AI legal research tools built on retrieval found that each of them gave a wrong or wrongly sourced answer on roughly a sixth to a third of queries (17% to 33%), and cites earlier work showing general-purpose models wrong on legal queries between 58% and 82% of the time (Magesh et al., Journal of Empirical Legal Studies, 2025). Botminds' Binder Guidelines Search is built for this one question: an MGA underwriter asks in plain words, and the answer comes back within authority, refer or outside, with the deciding clause quoted from the version in force.
The examples use a fictional MGA and a fictional sample binder. The research and regulatory sources are real and linked where they appear.
Four tests a binder answer must pass before an underwriter relies on it
Cited
Does every line of the answer quote the clause it rests on, with page and document?
How it fails
A limit stated from memory, or a clause number that sounds right but says something else.
How to test it
Ask for the clause text, then open the binder at that page and compare word for word.
Correct
Is the verdict what a careful senior underwriter would reach from those clauses?
How it fails
The right clause, misread: per policy read as per location, a minimum read as a maximum.
How to test it
Score against answers your own underwriters wrote down before the test.
Current
Is it the binder as amended on the risk’s effective date, and only by signed endorsements?
How it fails
The January wording after a June endorsement; a carrier bulletin treated as if signed.
How to test it
Include questions whose answer changed mid-year, and one that turns on an unsigned change.
Complete
Was every condition tested at every location, with nothing assumed?
How it fails
A yes built on a missing roof year; a schedule passed in total that fails at one site.
How to test it
Leave one detail out of a question. The right answer is a question back, not a verdict.
Can ChatGPT be used for insurance underwriting?
Yes, for work where a person checks the output before it matters: summarising a broker's submission, cleaning a schedule of values into columns, drafting a covering email, listing questions to ask the insured. European insurers are already doing this. EIOPA's 2026 survey of 347 undertakings in 25 countries found "nearly two-thirds of undertakings already actively using the technology", mostly on internal processes (EIOPA, February 2026).
The same survey names the problem: insurers ranked inaccurate outputs as the leading risk of generative AI, ahead of cybersecurity and data protection. For an MGA underwriter, an inaccurate output has a specific shape: a confident "within authority" on a risk the binder routes to the carrier, or a "refer" on a risk the binder allows. The first is binding outside authority; the second costs the broker days and sends the carrier work it did not need.
Can ChatGPT read a binding authority agreement?
It can read the text you give it; reading a binder correctly is a different task. A binding authority is not one document. The fictional sample binder used across this series is a 64-page agreement, 20 pages of schedules, a 38-page rating guide and four signed endorsements, with a fifth awaiting signature: 212 pages and 148 clauses in all. The answer to one question about one storage risk turned on four of them.
Four features of binders make them hard for a general chatbot. Limits are cross-referenced: clause 3.1(a) caps a location at $10 million, but clause 3.1(b) lets a $10 million to $15 million location in under a loss limit, priced from a section of a separate rating guide. Endorsements change clauses from a date, so the right wording depends on the risk's effective date, not the day of the question. Conditions apply per location while the schedule arrives in totals. And a chatbot is built to answer: given a gap, it fills it. None of these is exotic; each is a normal Tuesday at an MGA.
How often do AI tools get legal documents wrong?
Often enough that a citation alone proves nothing. The 2025 study in the Journal of Empirical Legal Studies, by researchers at Stanford and Yale, ran a preregistered set of 202 legal research queries through commercial tools built on retrieval-augmented generation and through a general-purpose model. It counted an answer as a hallucination when it was wrong, or when it was misgrounded: its key statements cited a source that does not say what the answer claims, or a source that does not apply.
What a peer-reviewed study found when it tested AI legal research tools
17–33%
of answers hallucinated by each commercial legal research tool built on retrieval
58–82%
average hallucination rate on legal queries for general-purpose models, in earlier work the study cites
202
preregistered legal queries in the study’s benchmark
Two findings matter for underwriters. Retrieval helped: the tools hallucinated less than general-purpose chatbots did, so a tool that reads your binder beats one answering from memory. But it did not solve it, and a tool that retrieves real text is exposed to exactly the error the study's definition singles out: a real source, cited for something it does not say. In a binder, that is clause 3.1(a) quoted against a location that clause 3.1(b) lets in, or the January named storm deductible quoted against a November risk.
A quoted clause proves the tool found a real sentence. It does not prove the sentence decides your risk.
Who is responsible if AI gives a wrong underwriting answer?
The insurer, and through the binder the MGA; never the software. EIOPA's opinion on AI governance is explicit that an insurer stays ultimately responsible for every AI system it uses, built in-house or bought from a provider (EIOPA Opinion on AI governance, August 2025, ¶3.11). In the United States, Model #225 §6 treats whatever the MGA does for the insurer as the insurer's own act (NAIC Model #225), so a risk bound on a wrong AI answer is the insurer's risk.
US regulators have also set expectations for insurers' use of AI, including due diligence on AI acquired from third parties; what the NAIC AI bulletin requires of insurers covers that ground. For an MGA the practical consequence is simple. The carrier will want to know, at audit, how each "can we write this?" was answered, by whom, on which clause and which version. A chat transcript is a poor answer to that question; a log of every check, with the clause returned, is a good one.
How do you test an AI underwriting answer on your own binder?
Build a small test set from your own binder before you trust any tool, and score every answer on the four tests. The study above shows the method: write the questions in advance, decide the right answers before running them, and mark both the verdict and whether the cited source supports it. Twenty well-chosen questions tell you more than a demonstration on the vendor's documents.
Ten questions to put to any AI tool before it answers binder questions at your MGA
| Question type | Example on the sample binder | What a safe answer does |
|---|---|---|
| Exactly at a limit | A location at $10,000,000 total insured value | Within authority: the clause says "shall not exceed", so equal passes |
| A permitted way in | A location at $11.6M | Refer as submitted, but names clause 3.1(b) and the $10M loss limit route |
| Changed mid-year | A first-tier risk with a 2% named storm deductible, effective 1 November | Applies the 5% minimum that an endorsement made effective on 1 June |
| Not yet in force | A frame apartment risk in a coastal county after a carrier bulletin | Says the bulletin is coming, does not apply it until the endorsement is signed |
| A missing detail | A schedule with no roof year for one site | Asks for it; gives no verdict |
| Per location, not total | Three sites, $20.9M in total, one over the location limit | Finds the one site; does not pass the schedule on its total |
| A shared limit | A first-tier risk when the aggregate is at 96% | Shows the aggregate as close, with the amount already used |
| A referral trigger inside the limits | A risk with a $120,000 loss three years ago | Refer under the loss-history criterion, even though every limit is met |
| Outside, not refer | Frame construction over $2.5M | Outside: the binder excludes it |
| No clause answers | A question the binder is silent on | Says no clause answers it, rather than guessing |
EIOPA points the same way for any AI system an insurer uses: "The undertaking should use metrics, including, where appropriate, fairness metrics, to measure the performance (e.g. accuracy, recall etc.) adapted to the AI system in question." For binder questions the metric that matters most is the share of answers that pass all four tests at once. A tool that is right but uncited cannot be audited; one that is cited but stale is wrong with a footnote.
What ChatGPT does well for underwriters, and where to stop
General chatbots are good at language around a decision and weak at the decision itself. Drafting a polite decline, summarising a ten-page survey, rewriting a referral note in plainer English, listing questions for a broker: all useful, all checked by the person who sends them. The line to draw is where an output becomes a statement about authority. "This risk is within our binder" is a statement the carrier will hold the MGA to, and it should come from a check that can show its clauses.
Personal lines desks face the same problem with eligibility questions from agents; why an underwriting chatbot's cited answer needs a second check covers that side. Delegated underwriting adds two things a carrier's own desk does not have: a contract with another company, and a carrier auditor who will read the answers months later.
How Binder Guidelines Search is built to the four tests
Binder Guidelines Search is not a general chatbot with a binder attached; it is a set of agents that each do one step and show what they did. In the sample check, the risk reader turned the underwriter's question and the broker's schedule into twelve details; the binder reader took the version in force on 1 November 2026 with endorsements 1 to 4 and pulled 31 clauses; the authority checker tested twelve conditions at three locations; the fits-if finder looked for a permitted alternative; the answer cited four clauses. The whole run took 8.1 seconds.
What each step of Binder Guidelines Search did: the version in force, the bulletin set aside
Each test maps to a rule the agents cannot break. Cited: every line of the checklist names its clause, quoted in the binder's own words, and limits are never paraphrased. Correct: each condition is tested by rule against the clause, at each location, and the verdict follows from the lines. Current: the binder reader uses the version in force on the effective date and never an unsigned endorsement or a bulletin. Complete: "within authority" is shown only when every condition is met and known. The design target is that 100% of answers cite the clause and the version in force; that is a target, to be measured on each customer's own binder.
Where it does not fit matters as much. It does not judge whether a risk is good; it does not price outside the binder's rating guide; it does not bind, apply an alternative or send anything. When no clause answers a question, it says so.
Every check, kept the way the carrier's auditor will ask for it
Questions to ask any vendor of an AI underwriting tool
Before an AI tool answers binder questions at your MGA
- Show me the clause, not a summary of it
Every limit quoted in the binder’s words, with page and document.
- Which version did you use, and why?
The effective date should decide it, and unsigned endorsements should be visible but unused.
- What happens when a detail is missing?
The right behaviour is a question back; a default value is a guess.
- Do you test per location?
Ask for a three-site schedule that passes in total and fails at one site.
- Will you run my test set?
Twenty questions from your binder, answers agreed in advance, scored on cited, correct, current and complete.
- Who can see the log?
The carrier and its auditor should be able to read the checks for their binder, and only theirs.
- What will it never do?
Bind, send a referral, or offer an alternative the binder does not name.
A buyer comparing tools side by side will find an AI underwriting assistant tool tested against PDF search and the referral desk useful next; the fourth test, current, has a guide of its own in which version of the binder applied on the day. To run the test set on the sample binder, start from Binder Guidelines Search.
Questions people ask
Is ChatGPT accurate for underwriting questions?
Not reliably enough to decide authority on its own. A 2025 study in the Journal of Empirical Legal Studies found even legal tools built on retrieval hallucinated on 17% to 33% of queries, and general-purpose models did worse. Accuracy on your own binder is something to measure, not assume.
Is it allowed to use ChatGPT in underwriting?
We know of no US or UK rule written specifically for AI in binder lookups, so general expectations apply: EIOPA says insurers remain "ultimately responsible" for AI they use, including third-party systems, and US regulators expect governance and third-party due diligence. Check your binding authority and the carrier's data rules before pasting its terms into any outside tool.
Does retrieval-augmented generation stop hallucinations?
It reduces them but does not stop them. The JELS study found hallucinations "reduced relative to general-purpose chatbots" yet still between 17% and 33% for each retrieval-based legal tool it tested. Its definition counts an answer that cites a real source for something the source does not support as a hallucination too.
What is a misgrounded answer?
One that cites a real source that does not support it: the source is misread or does not apply. In a binder, quoting the January clause for a November risk after a June endorsement is misgrounded, even though the quotation is accurate.
Can an AI tool bind a risk under our binder?
It should not. Binding is an act of authority the binder gives to the MGA and the MGA gives to named underwriters. Binder Guidelines Search never binds, never applies an alternative and never sends a note or referral; the underwriter does.