Legal contract review AI tools find, extract, summarize and compare clauses in contracts, and the better ones cite the text they relied on. They are fast and broadly accurate on clauses that say one thing; where they fail is the clause that can be read two ways, because a language model asked for an answer gives one, confidently. Peer-reviewed studies of legal AI show the risk: even retrieval-based legal research tools gave hallucinated answers to between about one in six and one in three of the queries tested. So the question to ask of any tool is not only how often it is right, but what it does when the words allow two meanings: does it pick one, or does it say so? Botminds' Investment Management Agreement Review is built to say so. Its agents quote every clause, re-read each one independently, write both readings when the text allows two, and send the choice to counsel; they never resolve an ambiguity themselves.
The worked example is a real-shaped clause from a fictional sample firm, Quillfield Capital, whose new client, Marlow County Retirement System, signed a side letter that rewords a credit limit in its investment guidelines. The research is cited from peer-reviewed journals and conference papers.
When a contract review agent routes, asks or stops
Both readers agree, both at 90% or more
Routes to the owning team, whose lead still accepts or returns it.
Agent routes · person accepts
Both readers agree, either below 90%
A person reviews the obligation before it routes.
Person first
The two readers disagree
Goes to Legal with both readings and their consequences. This setting cannot be switched off.
Counsel decides
The words allow two readings
Both readings are written out, a rule is drafted for each, and Legal chooses. The agents never choose.
Counsel decides
Text the agents cannot test
Goes to a person; no rule is drafted from a guess.
Person first
What legal contract review AI tools do well, and where they guess
On the easy majority of clauses, contract review AI does the reading a junior lawyer or paralegal would do, faster: it finds the termination clause, extracts the notice period, flags the assignment provision and lists the fee terms. On a separate-account bundle, most obligations are of that kind. In the sample firm's Marlow bundle of five documents and 112 pages, the agents found 41 obligations and two independent readers agreed on 40 of them.
The guessing happens on the forty-first. Contract language is built from exceptions, cross-references and words whose meaning depends on another document; the ContractNLI study of 607 annotated contracts found such features, including negation by exception, among the reasons machine reading of contracts is hard (Koreeda and Manning, Findings of EMNLP 2021). A tool asked for one answer will produce one. Whether it should have produced two is a design choice, and buyers rarely test it.
How often do AI legal tools hallucinate?
Often enough that every answer needs a check. In the first preregistered evaluation of commercial AI legal research tools, Magesh and colleagues found that tools built on retrieval reduced hallucinations compared with a general chatbot but still produced them in roughly 17% to 33% of queries (Journal of Empirical Legal Studies, 2025). An earlier study of general-purpose models on legal questions found far higher rates, and found that the models were poor at predicting their own hallucinations and tended to go along with a user's mistaken legal premise (Dahl, Magesh, Suzgun and Ho, Journal of Legal Analysis, 2024).
What two peer-reviewed studies found about legal AI
17–33%
of queries answered with a hallucination by retrieval-based legal research tools
58%+
hallucination rate of public general-purpose models on legal questions, at least
0
of these studies tested investment management agreements; your own test is the only figure that transfers
Neither study measured contract review, and both measured tools that have changed since. The lesson that transfers is structural: a model's confidence is not evidence that it is right, a model is poor at knowing when it is wrong, and a user's framing pulls the answer toward the user's assumption. Each of those is a reason to design for disagreement rather than hope for accuracy. How to measure a tool on your own documents is set out in a seeded test of contract obligation tracking software.
A clause that reads two ways: the 15% limit
Take the sample firm's new mandate as it runs through the solution. Schedule A §2.2 says securities rated below investment grade "shall not exceed 15% of the Account's market value at the time of purchase". The side letter, §3.1, says the account "shall not hold more than 15% of its market value in securities rated below investment grade". Both are 15%. They are different rules.
Two readings of one limit, and what each would make the firm do
Reading A: tested at the time of purchase: Only purchases are checked against 15%. If downgrades push the account above 15%, nothing must be sold.
Reading B: a continuous holding limit: The account may never hold more than 15%. Drift from downgrades is a passive breach that must be cured.
Reading A: tested at the time of purchase: The side letter adds nothing new: "hold" restates the Schedule.
Reading B: a continuous holding limit: "Shall not hold" is a different and stricter test, and the later side letter governs where inconsistent.
Reading A: tested at the time of purchase: Pre-trade check only.
Reading B: a continuous holding limit: Pre-trade and end-of-day check, with a cure workflow.
Reading A: tested at the time of purchase: Account at 13.8%: no action.
Reading B: a continuous holding limit: Account at 13.8%: a 1.2-point drift would open a cure.
Reading A: tested at the time of purchase: The Extraction agent, at 61%.
Reading B: a continuous holding limit: The Verification agent, at 72%; counsel read the same wording as B in three earlier agreements.
A tool that resolves this alone has made a legal decision with a trading consequence. Reading A leaves the portfolio manager free to hold downgraded bonds; Reading B obliges a sale when drift crosses the line. The agents' job is to put both in front of counsel with the evidence, including how counsel read the same pair of wordings before, and to have both compliance rules drafted so that whichever reading counsel picks can go live the same day.
Investment Management Agreement Review: both readings, both rules, and the choice left to counsel
Why a second reader must not see the first answer
A second reading is only a check if it is independent. A model shown the first model's answer tends to agree with it, the same way the Dahl study found models going along with a user's mistaken premise. In the sample firm's design, the Verification agent re-reads each clause without seeing the first answer, on a different model, and the two readings are compared; it never averages two answers, and a disagreement always goes to Legal with both readings.
On the 15% clause the two readers disagreed, and that disagreement is the signal the design exists to catch. On the tobacco exclusion in the same bundle they agreed, at 95% and 94%, so it routed automatically to compliance coding, where the rule analyst still checked it. Independence is what turns two readings into evidence: two readers that agree because one copied the other prove nothing.
Confidence is a routing signal, not a verdict
A contract review tool's confidence score is useful for one thing: deciding who looks next. The sample firm's settings send an obligation straight to its team only when both readers agree and both are at least 90% confident; below that, a person reviews it first. The threshold is a setting, adjustable from 75% to 99%, with each change versioned on the audit trail, and the rule that both readers must agree is locked on.
Confidence cannot replace a legal decision even when it is high. In the sample MFN check, one comparable account's lower fee was carved out because it was a founding-investor rate, and the MFN agent was 93% sure of that reading; counsel still confirms the carve-out before it is final, because it is part of a fee review that moves money, and every fee review needs counsel's approval. A tool that treats 93% as permission to decide has confused a probability with an authority.
What a useful answer from a contract review tool looks like
A useful answer has five parts: the conclusion, the words it rests on, where those words are, how sure the reader is, and what is still for a person to decide. Most tools give the first; the better ones give the second and third; few give the last two. Asked "Why 32 bps and not 35?", the sample firm's MFN agent answers that Marlow's side letter §4 promises the lowest fee given to any Core Plus Fixed Income separate account of equal or smaller size, that Harlow Valley Teachers' Retirement Fund, at $1.31 billion, has paid 32 basis points since July 1, 2026, that a cheaper account is carved out as a founding-investor rate, and that the gap is $426,000 a year, citing the side letter, the other account's amendment and the billing figure.
Asked the follow-up, "Why is Pemberton carved out?", the same agent says the account pays 31 basis points on a founding-investor rate, that §4(c) excludes founding-investor and performance-based fees, and that it is 93% sure of that reading, so Legal confirms the carve-out before it is final. That last clause is the part buyers should listen for. An answer that ends with a decision left to a named person is an answer designed to be checked.
Five parts of an answer a reviewer can check
- The conclusion
In one sentence, in the reader's words.
- The quoted words
The clause itself, not a paraphrase of it.
- The place
Document, version, page and section, clickable to the passage.
- The certainty
How sure the reader is, and whether a second reading agreed.
- What is left to a person
The decision the tool did not make, and who makes it.
What examiners will ask about agreements read by AI
The SEC's examination priorities for fiscal 2027 describe registrants using what the Division calls "super intelligence (SI)" for drafting, compliance monitoring and client communications, and say examiners will look at how these technologies influence outcomes and whether firms still meet obligations that apply whatever technology is used (SEC Division of Examinations, FY2027 priorities, a staff statement, not a rule). For an agreement review that means the obligations do not change: the firm must still follow its agreements, and its compliance policies must still be reasonably designed and implemented.
In practice an examiner will want what the firm would want anyway: for each obligation, the clause it came from, who read it and how, what was decided where the reading was uncertain, by whom and why. The SEC's 2019 fiduciary interpretation is a reminder of why the last part matters: an adviser's obligations depend on the scope of the relationship, which is set by the agreement's words (84 FR 33669). A clause read the wrong way changes the scope the adviser is held to.
Who may decide what: the separation of duties, enforced
Best AI tools for legal contract review: seven tests before you buy
No regulator or independent study ranks contract review products for investment management agreements, and the published benchmarks were built on other kinds of contract. The best tool for a firm is the one that passes the firm's own tests on its own documents. The seven below put the ambiguity question first.
Seven tests for any legal contract review AI tool, and what a pass looks like
| Test | How to run it | A pass |
|---|---|---|
| Two readings | Plant a clause that reads two ways, such as a limit "at purchase" against "shall not hold" | The tool flags it with both readings and does not pick one |
| Citation | Check ten answers against the document | Every answer quotes text that is on the cited page |
| Override | Add a side letter that changes a term and says it governs | The side letter's term is returned and the original marked as replaced |
| Independent check | Ask how the second reading is produced | A separate reading that does not see the first, with disagreements sent to a person |
| Wrong premise | Ask a question that assumes a term the contract does not contain | The tool says the contract does not say that |
| Language | Include a clause signed in another language | The original is kept and cited beside any translation |
| Authority | Ask the tool to approve, change a fee or contact a client | It cannot; only a named person can |
A tool that passes these is designed to be supervised. A tool that fails the first is designed to be trusted, which is a different product, and the research above says trust is not yet warranted for any of them.
Where Investment Management Agreement Review fits, and what it leaves to counsel
The solution is a contract review tool for one kind of contract: an asset manager's investment management agreements, guidelines, fee schedules, side letters and amendments, in the language each was signed in. It reads, quotes, re-reads independently, compares side letters with the agreement, writes both readings where the text allows two, drafts compliance rules and dry-runs them, tests most-favored-nation clauses against billed fees, and routes each obligation to its team. It does not negotiate, draft agreements or give legal advice.
A clause signed in German, read in German, with the English beside it
What it leaves to counsel is everything that is a decision: which reading of an ambiguous clause applies, whether a carve-out holds, whether a fee change is approved, and whether a register is complete. In the sample firm, counsel decides each with a reason code, the person who prepared an item can never approve it, and every decision is on an append-only audit trail. A liability clause is the hardest case of all, and how to test a hedge clause shows why it belongs with counsel. To run the seven tests on your own agreements, start from Investment Management Agreement Review.
Questions people ask
Can AI replace lawyers for contract review?
Not for decisions. AI tools do the first reading quickly and cite the text, but peer-reviewed studies still find confident errors in a meaningful share of legal answers, and a clause that reads two ways needs a lawyer's choice. The useful design is AI reading and checking, with counsel deciding.
Is ChatGPT good for legal contract review?
A general chatbot can summarize a contract, but a 2024 study in the Journal of Legal Analysis found public general-purpose models hallucinated on legal questions at least 58% of the time and tended to accept users' incorrect legal assumptions. For contract obligations, use a tool that quotes and cites every answer and is tested on your own documents.
What accuracy should a contract review AI tool have?
No regulator or independent study sets a figure for investment management agreements. Measure recall and precision by clause type on a seeded bundle of your own, and set the pass line by consequence: restrictions, fee terms and overrides matter more than low-stakes deliverables.
Is it safe to upload contracts to AI contract review tools?
It depends on where the data goes and who sees it. Side letters and other clients' fees are confidential; the sample firm limits side letters to legal, the relationship team and the lane leads who own their obligations, shows other clients' names in an MFN check only to legal, billing and internal audit, and keeps data in each entity's region.
Do contract review AI tools work on agreements in other languages?
Some do, and the test is whether the obligation is cited to the original text rather than to a translation. The sample firm's agents read German, Dutch, French and Japanese documents in the language they were signed in and show the original beside the English.
What should a contract review tool do when it is unsure?
Say so and route the clause to a person. In the sample firm, an obligation routes automatically only when two independent readers agree and both are at least 90% confident; otherwise a person reviews it, and any disagreement goes to Legal with both readings.