An AI clause library is a clause library you can question in plain words: an underwriter types "have we agreed a sanctions clause without the US reference?" and an AI model finds the clauses and writes the answer. The value is speed; the risk is that language models produce fluent answers that no wording supports. For insurance wordings, where a wrong precedent becomes a concession at the renewal table and a dispute at claim, the only safe design answers from the carrier's own bound wordings, cites a source for every sentence, and says plainly when no precedent exists. Botminds' Clause Library & Precedent Search is built on that rule: no citation, no sentence.
This guide sets out what independent researchers have measured about AI legal research tools, the three kinds of answer such a tool can give, what an AI should do when the book holds no precedent, and a forty-question test a wordings team can run on any AI clause library before buying it. Figures from the solution itself come from its fictional sample book of wordings.
Even tools built to cite real sources got legal questions wrong or misattributed
What is an AI clause library?
An AI clause library joins three things that can each fail on their own: a store of clauses, a search that understands the question, and a model that writes the answer. Most clause libraries already have the first, a set of approved standard clauses. Adding AI usually means adding the second and third: a search that matches meaning rather than exact words, and a model that summarises what it found.
For an insurer, the store is the part that decides whether the answer is worth having. A library of approved standard text can only say what the carrier prefers. The question a broker's draft raises is different: has this carrier bound this variation before, for whom, with whose approval, and what happened at claim? That answer lives in bound wordings, endorsements, referral notes and claim files, and an AI layered over a library of templates cannot reach it, however good the model. Folders, contract tools and cited search, compared on one broker question, shows how much the store matters.
The second and third layers decide whether the answer can be trusted. Search that matches meaning will find "may expose" when asked about "would expose", which is useful, and will also find a 2019 draft that was never bound, which is not. A model that writes fluent prose will write a fluent sentence whether or not a source supports it. That is the measured problem.
Do AI legal research tools hallucinate? What Stanford researchers measured
Yes, and the tools that cite real sources do it less than general chatbots, but not rarely. In the first preregistered test of commercial legal research tools built on retrieval, Stanford researchers ran over 200 legal queries and found that each tool gave hallucinated answers on roughly a sixth to a third of them. The best-performing system answered 65% of queries accurately; another gave "incomplete answers (refusals or ungrounded responses ...) on more than 60% of our queries" (Magesh et al., Stanford, 2024). The study was published in the Journal of Empirical Legal Studies in 2025.
General chatbots did worse on legal questions in an earlier study by the same group. Testing public models on questions about US federal court cases, Dahl and colleagues found that "LLMs hallucinate at least 58% of the time, struggle to predict their own hallucinations, and often uncritically accept users’ incorrect legal assumptions" (Dahl et al., Journal of Legal Analysis, 2024). The last finding matters most for wordings. An underwriter who asks "when did we first drop the US reference?" has assumed it was dropped; a model that accepts the premise will find a date.
Neither study tested insurance wordings, and a carrier's bound book is a narrower and cleaner corpus than all of US case law. That cuts both ways: a narrow corpus means fewer wrong sources to retrieve, and also more questions to which the true answer is "nothing here". A tool that is good on the first and bad on the second will invent precedents for exactly the requests a carrier has never agreed, which are the requests that matter most.
Grounded, ungrounded, misgrounded: the three answers you can get
The Stanford study gives buyers the vocabulary they need. A response is grounded "if the key factual propositions in its response make valid references to relevant legal documents"; ungrounded "if key factual propositions are not cited"; and misgrounded "if key factual propositions are cited but misinterpret the source or reference an inapplicable source". A hallucination is then precisely defined: "A response is considered hallucinated if it is either incorrect or misgrounded" (Magesh et al.).
Translated to wordings, the three look like this. A grounded answer says the US reference was removed once, in a named 2022 wording, and links to that clause. An ungrounded answer says the same thing with no link, which may be right and cannot be checked on a renewal call. A misgrounded answer links to a real wording that does not say what the sentence claims: a broker's draft that was refused, a quote that never bound, or a wording from a class the underwriter is not entitled to see. Misgrounded is the dangerous one, because the citation makes it look checked.
The study also counted refusals as incomplete rather than wrong. That is the right way to score a wordings tool. An answer of "no precedent in the bound book for marine cargo since 2021" is complete when it is true, and a buyer should test for it deliberately.
What should an AI do when no precedent exists?
It should say so, name what it searched, and send the request to the person who decides new wordings. The Stanford researchers built false-premise questions into their test and scored a refusal as correct when it "mentions the fact that no pertinent sources were found" (Magesh et al.). A precedent tool should treat "never seen in our book" as an answer, not a failure.
Three checks before an AI clause library may show an answer
1Is there a bound wording, standard form or approval note in the book that carries this clause family, class and wording?
NoSay so: "No precedent in the bound book." Name what was searched, offer to widen the class or years, and route the request to the head of wordings as never seen.
YesGo to check 2.
2Does each sentence of the draft answer point to a passage that actually says it?
NoRemove the sentence. An answer with fewer sentences is better than one with an unsupported sentence.
YesGo to check 3.
3Is every cited source a final, bound wording (not a draft or a quote) from a class the reader may see?
NoDrop the source and re-check the sentence it supported; a draft cited as a precedent is the misgrounded answer in wordings form.
YesShow the answer, each sentence with its citation, and the steps taken to find it.
On the solution's fictional sample book, a broker's 14-clause renewal draft came back with one clause the book had never seen. The answer did not guess at a nearest match; it marked the clause as never seen, which under the carrier's settings means the head of wordings must approve it before it is offered. That is what the refusal is for: it turns a gap in the evidence into a decision for the right person, rather than a confident sentence from the wrong one.
The refusal rate is therefore a quality measure, read alongside accuracy. A tool that never says "nothing found" on a book where many requests are genuinely new is not more capable; it is inventing. A tool that refuses most questions is not safe either; its search is failing. The forty-question test below measures both.
What is a cited answer, sentence by sentence?
A cited answer is one in which every sentence points to the passage that supports it, and a sentence with nothing to point to is removed before the answer is shown. In the solution's sample, the answer to "have we bound a cargo sanctions clause without the US reference?" had seven sentences, and all seven were cited to bound wordings; the screen shows the count. A citation must land on one of three kinds of source: a bound wording or endorsement that was finally issued, the carrier's standard form, or an approval, referral or refusal note. Drafts and quotes are evidence of what was asked, never of what was agreed.
How the answer was found: each step on screen, with its time and its sources
The steps panel is as important as the citations. An underwriter who can see that the question was read as "sanctions, marine cargo, 2021 to 2026" can tell at once if the tool misread it, for example by searching the wrong class, before relying on the answer. Showing the interpretation is how a reader catches the false-premise problem the Dahl study describes.
How do you check an AI precedent answer? A forty-question test
Check it against your own book, with questions whose answers you already know, including questions whose true answer is "never". The test below takes a wordings specialist about two days to build from a sample of bound wordings and referral notes, and it scores any AI clause library, ours included, on the three measures the research uses.
A forty-question test for any AI clause library, built from your own bound wordings
- Ten questions with one known precedent
A variation bound once, for one insured, with an approval note. The answer must find that wording and cite the clause, not the policy.
- Ten questions whose true answer is "never"
Variations your book has never agreed. Score a plain "no precedent found", naming what was searched, as correct. Any named precedent is a hallucination.
- Five refused requests
Requests the carrier declined. The answer must say they were refused, by whom and when; a tool that only reads bound wordings will call them "never seen", which is incomplete.
- Five near-duplicates
Pairs of wordings that differ by one word ("would" and "may", a regime added). The answer must cite the right one.
- Five drafts that never bound
Seed the test with broker drafts and quotes. Any answer that cites a draft as a precedent is misgrounded.
- Five questions across classes the user may not see
Log in as a class underwriter and ask about another class. The answer must not show the wording.
- Score each answer three ways
Accurate (correct and every sentence cited), incomplete (refused, or correct but uncited), hallucinated (wrong, or citing a source that does not say it). Two specialists score independently.
The two-scorer instruction is there because people disagree too, more than most teams assume, and an AI should be judged against the spread between readers rather than against one reader's view. How to measure agreement between two wordings specialists sets out the arithmetic.
Who is responsible when an AI clause library is wrong?
The carrier and the person who relies on the answer, not the tool. English courts have said so plainly about lawyers. In a 2025 judgment on fictitious citations put before the court, the President of the King's Bench Division wrote that freely available generative AI tools "can produce apparently coherent and plausible responses to prompts, but those coherent and plausible responses may turn out to be entirely incorrect" (R (Ayinde) v London Borough of Haringey [2025] EWHC 1383 (Admin), para 6). The duty to check sat with the professional who used them.
Insurance supervisors take the same line on underwriting tools. New York's Department of Financial Services told insurers in 2024 that they, not their vendors, remain responsible for understanding the tools used in underwriting and pricing (NY DFS Insurance Circular Letter No. 7 (2024)). The practical consequence for a wordings tool is narrow and strict: an answer the underwriter cannot check is an answer the carrier cannot defend.
AI is now routine in financial firms, and hallucination is a named risk
86%
of UK financial firms responding are using AI
18
median AI systems in production at responding insurers
Top 3
GenAI risks such as hallucination rank among the highest-rated risks today
Can AI replace a wordings specialist?
No, and a well-built AI clause library is designed so that it cannot. What it replaces is the fetching: reading thousands of wordings to find the clause, lining up versions, chasing the approval note and the claim outcome. What stays with people is deciding whether the carrier should agree a change now, which depends on appetite, the insured, the reinsurance behind the risk and the claims history, and on authority the carrier gives to named people.
Clause Library & Precedent Search: agents fetch and cite, people decide and sign
Agent
Wording readerReads every bound wording, endorsement and standard form and names each clause by family and version.
Never: Never treats a draft as bound until the final wording is filed.
Agent
Precedent finderTurns the plain question into family, class, years and markets, and ranks the bound versions that answer it.
Never: Removes any uncited sentence; never shows a class the user may not see.
Agent
Difference markerMarks the changed words against the version before, the standard and the broker's draft.
Never: Never judges whether a change is acceptable.
Agent
Outcome linkerAttaches approvals, refusals and claim outcomes to the version used.
Never: Never shows claims counsel's privileged notes outside claims.
A person decides
Class underwriterReads the cited precedents and decides, within line authority, what to offer.
Never: A version refused before or never bound goes to the head of wordings first.
There is a second reason the specialist stays. The quality of the answers depends on the quality of the book underneath: whether endorsements were filed with the wordings they amend, whether refusals were kept, whether a claim was linked to the version argued. Those are curation decisions a wordings team owns, and they are the decisions that make the AI's citations worth reading.
What to ask an AI clause library vendor, and where ours does not fit
Ask what the answer is built from, what happens when nothing is found, and what the tool may never do. The table gives the questions and the answer a buyer should want; we have filled in our own.
Questions for an AI clause library demonstration
| Ask | The answer to look for | Clause Library & Precedent Search |
|---|---|---|
| What does it read? | Your bound wordings, endorsements, refusals, approvals and claim outcomes, not only templates. | All six sources, with volumes and refresh times shown in settings. |
| What happens when nothing matches? | A plain "no precedent", naming what was searched, and a route to the decider. | Marks the clause never seen; the head of wordings approves before it is offered. |
| Can I see why each sentence is there? | A citation per sentence to a passage, and the steps the search took. | No citation, no sentence; steps and times on screen. |
| Can it cite a draft? | Never as a precedent; drafts are evidence of what was asked. | A draft is not treated as bound until the final wording is filed. |
| Who sees claims material? | Underwriters see an outcome line; privileged notes stay with claims. | Outcome line only outside claims, by setting. |
| What may it do on its own? | Find, compare, cite and draft; never send, edit or sign. | Agents never send to a broker, edit a wording, change a standard form or sign. |
Where it does not fit: a carrier with few bound wordings in machine-readable form, or none of its referral and refusal history kept anywhere, will get thin answers until that history is loaded, and building the library from bound wordings comes first. It is not a drafting tool for new wordings, and it does not decide appetite. For reinsurance treaties, where the question is usually how two contracts differ rather than what was agreed before, the reinsurance treaty comparison guide covers the different test. To see cited answers on a sample book, start from Clause Library & Precedent Search.
The answer and the precedent stack: six bound versions, every sentence cited
Questions people ask
Can AI search insurance precedents reliably?
It can find them quickly; whether it reports them reliably depends on the design. Published tests of legal research tools found wrong or misattributed answers on roughly a sixth to a third of queries even when the tools cited real sources. A precedent tool for wordings should answer only from bound wordings, cite every sentence and say when nothing matches.
How often do AI legal tools make up citations?
In Stanford's preregistered study of commercial legal research tools, each tool hallucinated, meaning it gave an incorrect answer or cited a source that did not support it, on roughly 17% to 33% of more than 200 queries. General chatbots tested on US federal case-law questions did worse, at 58% to 88% depending on the model.
What is a misgrounded answer?
One that cites a real source that does not support the statement, or cites an inapplicable source. In wordings, the common form is an answer that cites a broker's draft or a quote as if it were a bound precedent. It is more dangerous than an uncited answer because the citation makes it look checked.
Is a clause library with AI the same as ChatGPT on our wordings?
No. A general chatbot given a folder of wordings will answer from whatever it retrieves and will accept the premise of the question. An AI clause library built for precedent should restrict sources to final bound wordings, standard forms and approval notes, enforce who may see which class, remove uncited sentences and refuse when nothing matches.
Who is liable if an underwriter relies on a wrong AI answer?
The carrier remains responsible for the tools it uses in underwriting; New York's regulator said so in its 2024 circular on AI in underwriting and pricing. That is why the answer must show its sources and steps: the underwriter can only be expected to rely on what they can check.
How do we test an AI clause library before buying?
Build about forty questions from your own book, including ten whose true answer is "never" and five drafts that never bound, and have two specialists score each answer as accurate, incomplete or hallucinated. A tool that never says "no precedent found" on genuinely new requests is inventing answers.