AI legal research hallucinations are answers that are wrong, or that cite a source which does not support them, and they cluster where the law is hard: the wrong jurisdiction, an outdated version, a false premise, unclear text. AI can do most of a fifty-state legal research task accurately when the state's text is clear: find the section, quote it and do the arithmetic. It becomes unreliable when the text is unclear, because language models tend to answer anyway and often do not know they are wrong. The safe design treats "the passage does not settle this" as an answer in its own right. The state is flagged, both readings are shown, and a person decides. Botminds' State Lending Law Research works this way. Its state readers never interpret beyond the text, return both sections where two conflict, and send ambiguity to a person rather than guessing. Where the rule library holds no passage, the solution says so and stops.
Where the agents stop and pass the question to a person
Agent
Restate the questionThe research agent restates the question with the loan facts it depends on, so a false premise shows before any state is read.
Never: answer a question it has not restated
Agent
Find the passageOne reader per jurisdiction searches statutes, administrative code and regulator guidance for the facts asked.
Never: answer from memory when the library holds no passage
Agent
Test whether the text settles itClear text gives an answer. Two conflicting sections, an undefined term or a judgment word give a flag, with both readings.
Never: interpret beyond the text
A person decides
The analyst triages the flagsFlagged states are read first. Some are settled by a regulator FAQ; the rest go to counsel with the passages attached.
A person decides
Counsel decides and signsCounsel chooses the reading, records why, and the decision is kept with the answer for the next time the question is asked.
How often do AI legal research tools hallucinate?
In the only preregistered public test of commercial legal research tools, between one answer in six and one in three. Researchers at Stanford put more than 200 legal queries to three tools that retrieve sources before answering. They counted a response as hallucinated "if it is either incorrect or misgrounded", where misgrounded means the answer cites a source that does not support it. The best tool's answers were "accurate (i.e., correct and grounded) for 65% of queries"; the other two managed 41% and 19%, and the tools gave incomplete answers "18%, 25% and 62% of the time" (Magesh et al., 2024).
AI legal research hallucinations in three commercial tools: answers by outcome
Tool A
- Accurate · 65%
- Incomplete · 18%
- Hallucinated · 17%
Tool B
- Accurate · 41%
- Incomplete · 25%
- Hallucinated · 34%
Tool C
- Accurate · 19%
- Incomplete · 62%
- Hallucinated · 19%
Two findings matter most for state lending law. The study found that "hallucination rates are slightly higher for jurisdiction and time specific questions", which is what every fifty-state question is. And the tool with the fewest hallucinations after the best one, Tool C, got there mostly by not answering: 62% of its responses were incomplete. A low error rate bought with refusals is not useful to a compliance team either. The aim is a tool that answers the clear states and says precisely why it will not answer the others.
Why do AI legal research tools hallucinate?
Mostly because they read the wrong source or believe the question. The same study traced "many failures" to "poor retrieval", and found tools citing authority that was inapplicable "because the authority is for the wrong jurisdiction, wrong statute, wrong court, or has been overruled". It also documented sycophancy, "the tendency of chat systems to assume the veracity of a premise even when false". One tool described a Missouri statute as valid although the state's supreme court had struck it down in the very case the tool cited.
Why legal AI goes wrong, and what that looks like on a fifty-state question
| Cause the research found | On a state lending-law question | The control that catches it |
|---|---|---|
| Wrong jurisdiction | A neighbouring state's cap, or a federal rule, reported as the state's answer | One reader per jurisdiction, searching only that state's library |
| Overruled or outdated authority | Last version's late-fee amount, or a rule a court has enjoined | Sections kept by version; the answer dated "as of" |
| Misgrounded citation | A real section cited for a condition it does not contain | A separate checker reads each answer back against its passage |
| False premise accepted | "Since Texas caps late fees at $15…" built into the answer | The question restated with its facts before any state is read |
| Unclear text resolved by guess | A yes or no on whether a state rule binds a national bank | A flag with both readings, routed to counsel |
Can AI do fifty-state legal research accurately?
It can when the text is clear and the system is designed to check itself, and it is dangerous when the text is unclear and the system answers anyway. The first four rows of the table have mechanical fixes: the right library, the right version, a second reader, a restated question. The last row does not. No amount of retrieval turns a judgment word into a lookup, so the only safe output is a flag.
Computer-science research has a name for that behaviour. Abstention, "the refusal of large language models (LLMs) to provide an answer, is increasingly recognized for its potential to mitigate hallucinations and enhance safety" (Wen et al., Know Your Limits, 2024). In legal research, refusing to answer is not enough. The question still needs an answer, so the system should say why it stopped, show both readings and pass the state to someone who can decide. That is the difference between Tool C's refusals and a useful flag.
Three kinds of unclear text a fifty-state reader meets
Unclear text in lending law tends to fall into three kinds, and each has a recent, real example. The first is a judgment word, where the statute requires an assessment rather than a reading. The second is a contested phrase, where courts disagree on the meaning. The third is a condition that depends on facts outside the text. An AI reader can recognise all three. It cannot resolve any of them on its own authority.
Real phrases a reader should flag rather than resolve
| Phrase | Where it appears | Why the text alone cannot settle it |
|---|---|---|
| "a practical assessment of the nature and degree of the interference" | Cantero v. Bank of America (2024), the standard for whether a state law binds a national bank. | It asks for a weighing of effects, case by case. The Court declined to set a bright line. |
| loans "made in" such State | DIDMCA section 525, the state opt-out from rate exportation. | Whether a loan is made where the bank is, or also where the borrower is, is before the full Tenth Circuit. |
| a law "similar in principle to this chapter" | Fla. Stat. § 516.02(2)(c), the exception for loans made to Florida residents from another state. | Whether another state's consumer finance law is similar enough is a judgment, not a lookup. |
| the rate set "by written agreement of the parties to the contract" | Washington's general usury limit, RCW 19.52.020(1), which reads the T-bill rate in the month before that agreement. | Which month applies, and whether a later adjustment restarts it, depends on the lender's documents, not the statute. |
Cantero shows the problem most clearly. A unanimous Supreme Court rejected the lower court's test because it "did not conduct the kind of nuanced comparative analysis required by Barnett Bank, but instead distilled a categorical test that would preempt virtually all state laws that regulate national banks" (Cantero). An agent asked whether a state's rule binds a national bank, and answering yes or no, would be doing what the Court rejected.
A court applying that standard must make a practical assessment of the nature and degree of the interference caused by a state law.
Cantero v. Bank of America, N.A., 602 U.S. 205 (2024)
When the rule library holds no passage, the answer is "no passage"
The simplest case is also the most important. If the reader finds no section that answers the question, for example because a state has no specific late-fee rule for the product, it should say so and stop, not fill the gap from what other states do. In the solution this is a written limit on the research agent: "Outside the rule library, says so and stops". A state with no fixed cap is a different thing, an answer backed by a passage. In the sample run, seven jurisdictions set no fixed cap beyond what the contract states, and each of those answers carries the passage that says so.
The difference matters to a product team. "This state sets no cap; the fee is as agreed" lets them set their own fee there. "We found nothing" means someone must look further, perhaps at a regulator's guidance or an attorney general's position, before the state goes in the table. A tool that merges the two produces a confident answer that may be wrong.
Two sections that disagree: return both
Where two sections apply and say different things, the reader should return both and flag the conflict. Picking one is an interpretation, and that belongs to counsel. Conflicts happen more often than tables suggest. A general usury statute and a consumer finance act may both appear to cover a loan. A statute and a later regulator rule may express a cap differently. A fee rule may be amended while an older section still refers to the earlier amount. The solution's state readers have the limit written into their role: "Where two sections conflict, returns both and flags it."
The cited section, the plain answer and the checker's verdict side by side
Flags are a measure of quality, not of failure
A system that never flags anything has either been given easy questions or is guessing, so the flag rate should be measured as a quality signal. During a pilot on past questions, count how often the agents flagged a state, how often counsel agreed the flag was needed, and how often an unflagged answer turned out to need judgment. The last of these is the dangerous number, because it counts the guesses that got through. Test it on questions your counsel already answered sets out how to measure all three.
What an agent should do when the text does not settle the answer
- Name the reason
No passage, conflicting sections, a judgment word, a contested phrase, or a fact outside the text.
- Show the competing text
Both passages, highlighted, so the reviewer starts from the source, not from the agent's summary.
- Answer the rest of the question
Flag one state; do not hold back the other fifty that are clear.
- Route it to the right person
Factual conditions to the analyst or the business; interpretations and preemption to counsel.
- Keep the decision with the answer
Counsel's reading and reason travel with the saved answer, so the next person does not ask again.
- Re-raise it when the text changes
A decision taken on one version of a section is reopened when the section is amended.
Who answers the flagged question
Flagged questions go to people, and the solution keeps those people named. The analyst decides what is saved and sent, and counsel reviews every saved answer before it leaves compliance. As a Federal Reserve examiner put it, "the institution remains responsible for ensuring that the output from vendor-provided systems and services meets regulatory requirements" (Consumer Compliance Outlook, 2019). Routing ambiguity to a named person is how that responsibility is kept in practice.
The answer map filling in, with each agent's step shown as it runs
This is the strongest argument for agents over a chat assistant. A chat assistant gives one answer for the whole country. A set of state readers gives fifty-one answers, and each one can be clear, flagged or "no passage" on its own terms. Is ChatGPT good for legal research? sets out that contrast, and the State Lending Law Research page shows the full set of agents and the limits on each.
Questions people ask
How do I stop AI from guessing on legal questions?
Build the stop into the system rather than the prompt. Require a passage for every answer, check every answer against its passage with a separate step, and give the agents a way to return "unclear" with the competing text. Then measure how often they use it.
Who should resolve an ambiguous state statute?
In-house counsel, or outside counsel for a novel question. The reading and the reason should be kept with the saved answer, and the question should be reopened when the section is amended.
Does retrieval-augmented generation stop hallucinations?
It reduces them without stopping them. Legal research providers had described retrieval as "eliminating" or avoiding hallucinations, and the Stanford study found those claims "overstated": the retrieval-based tools it tested still hallucinated on one answer in six or more. Retrieval decides what the model reads, not whether its answer matches it.
What is a misgrounded citation?
A citation to a real source that does not support the statement it is attached to. The Stanford study counts it as a hallucination alongside a plainly incorrect answer. It is the harder kind to catch, because the cited section exists and a quick check that it exists will pass.
How do I measure an AI tool's hallucination rate on my own work?
Re-ask questions your counsel has already answered and count, state by state, citations that do not hold, caps that do not match on clear text, and confident answers where counsel had needed judgment. Fix the measures before the first question, as the Stanford researchers did by preregistering their test.