Side letter compliance with AI: a second model re-checks every call

Side letter compliance means two things: making every most-favored-nation (MFN) eligibility call correctly, with the clause behind it, and keeping every promise the side letters contain. Both depend on reading the letters right. AI can now read side letters, in their own languages, faster than any legal team, but the research on AI contract review says it still errs, and an error in an MFN call goes straight into an investor's packet. The control that answers that is old: four eyes, made independent. In Botminds' Side Letter Management, one agent works out each investor's eligibility for every term on its ballot, a second agent on a different model re-derives each call without seeing the first answer, and any disagreement holds the packet and goes to fund counsel with both reasonings side by side. People approve, send and confirm; the agents never do.

The examples come from a fictional fund run through the solution, Quillfield Capital Partners IV, L.P., in its October 2026 MFN election season.

What is side letter compliance?

Side letter compliance is doing what the side letters say, for each investor, and being able to prove it. It has a front end and a back end. The front end is the MFN election: after final close, every investor with an MFN right must be offered exactly the terms its clause allows, no more and no fewer. The back end is the obligations: every term signed or elected becomes a report, a notice or a restriction that someone must deliver on time.

The legal stakes are contractual first, and regulatory where investors are misled about them. In the US, misleading investors about side letter terms falls under the anti-fraud rule for pooled vehicles, Rule 206(4)-8; in the EU and UK, preferential treatment must be described to investors before they invest. The rules on side letters in private equity sets out each regime; this guide is about getting the calls right.

Can AI read side letters?

Yes, with limits that should shape how it is used. Side letters are short contracts, but they are full of conditions, cross-references to the LPA and carve-outs, often in Dutch, German, French or Japanese, and a misread "not" or a dropped condition changes who may elect a term.

The research benchmark built for this work is telling in its own design. CUAD, a dataset of more than 13,000 annotations on contracts made with dozens of legal experts, frames the task as finding the clauses a person must review, and its authors were candid that the best models of the day, though promising, were far from finished (Hendrycks et al., CUAD, NeurIPS 2021 Datasets track). The benchmark was framed around review by a person, and so should the use be. In the solution, the Intake agent reads each letter in its signed language, splits it into provisions cited to clause and page, and sends any provision it reads below 0.90 confidence to a legal operations analyst, with the original text beside the English.

That threshold is itself a setting with evidence behind it. In the sample firm, a change to 0.92 has been proposed because 6 of the last 40 provisions read between 0.90 and 0.92, in Japanese and French letters, needed a correction; it waits for counsel or compliance to approve it. A threshold that is never measured is a guess.

How accurate is AI contract review?

Good enough to draft and to find, not good enough to decide unchecked. The most careful public test is a preregistered evaluation of commercial legal research tools by Magesh and colleagues, which found that the leading products produced hallucinated answers in roughly a sixth to a third of queries, despite vendor claims that retrieval removed the problem. Because the products are closed, the authors note, nobody outside can test those claims systematically; their own test gave a short verdict:

“We demonstrate that the providers' claims are overstated.”

Magesh, Surani, Dahl, Suzgun, Manning and Ho, Journal of Empirical Legal Studies 22(2), 2025

No study has measured AI accuracy on side letters or MFN eligibility specifically, and no regulator publishes an error rate for it. A manager should therefore measure its own: take a past season's ballots, decided by counsel, and run them through any tool before it is trusted with a live one. The sample firm does that for its Checker agent with a test set of 2,480 past calls with known answers; the proposed new version scores 99.4% on it, against 99.1% for the current one, and waits for counsel or compliance to approve the change. Those are figures on fictional sample data, and the method, not the number, is the point.

What is a four-eyes check, and why use a second model to check the first?

A four-eyes check is a control in which a second person, independent of the first, verifies a decision before it takes effect. It works only if the second reviewer does not simply agree with the first. For AI, that means the checker must work out the answer itself, from the same sources, without seeing the first answer, and should not share the first model's blind spots.

The last condition is harder than it sounds. A large study of more than 350 language models found that models from different developers still fail in the same way surprisingly often:

“larger and more accurate models have highly correlated errors, even with distinct architectures and providers”

Kim, Garg, Peng and Garg, 'Correlated Errors in Large Language Models', ICML 2025 (arXiv 2506.07962)

On one benchmark the authors report that when two models both err, they give the same wrong answer 60% of the time. A second model is therefore a real check, not a perfect one. It catches the errors the first model makes alone, and misses many it shares, which is why the design matters as much as the model: the checker works blind, from the sources rather than the first answer, and every disagreement goes to a person rather than to a tie-break by a third model.

A disagreement between the two agents, as counsel sees it

On most ballots the two agents agree on every call; the value of the check is in the few where they do not. In the sample season, the Checker agent agreed with all 43 calls on a Dutch pension fund's ballot. On a Japanese insurer's ballot it agreed with 37 of 38, and the one difference is a good example of what a second reading is for.

Neither agent is obviously wrong. The disagreement exposes an ambiguity in the investor's own clause, which is exactly the kind of call that should be made by counsel, recorded once, and applied the same way to every letter with the same words. A single-pass system would have made the call silently, one way or the other, and the investor might have discovered the choice only when it disputed it.

A held ballot in Side Letter Management: both readings, side by side

Who approves an MFN eligibility call?

Fund counsel, always; the agents propose and check. In the sample firm counsel approves every packet with one of three reason codes: AP-01, eligibility agrees with the MFN clause; AP-02, an override recorded with counsel's reading; AP-03, conditions attached and accepted. An override of an agent's call needs its own reason: OV-01, the agent misread the clause; OV-02, the investor's status changed; OV-03, counsel's interpretation, applied as precedent.

The codes do two jobs. They make every decision explainable to the investor whose ballot it shaped, and they turn counsel's rulings into a body of precedent: once counsel decides what "same regulatory status" means in one letter, the agents apply that reading to every letter with the same words, and show it as a ruling rather than their own conclusion. In the sample season counsel overrode an exclusion, making a term electable for a fund of funds with reason OV-01, and that override is on the audit trail with her note.

What a second check cannot catch

A second model checks the reading, not the inputs. If the investor register omits an affiliated vehicle, both agents will aggregate the same wrong commitment; that is why the Investor agent holds the packet when a status or affiliate is missing rather than guessing. If a translation is wrong in a way both models share, both will misread the same clause; that is why the original language stays beside the English and uncertain readings go to a person. And if the clause is genuinely ambiguous, agreement between two models is not an answer; it is two guesses that happen to match.

Three controls sit outside the models for those reasons: counsel's approval of every packet, the CFO's co-signature on any elected term that moves money, and the rule that the person who prepares a packet can never approve it. The SEC's fiscal 2027 examination priorities, which call these tools "super intelligence" (SI), say examiners will review firms' policies and procedures for monitoring their use, including in back-office operations (SEC Division of Examinations, FY2027 priorities, October 23, 2026). A design in which no model's output reaches an investor without a named person's approval is easy to explain to that review.

One investor's ballot with the Checker agent's agreement on every line

What should you ask a side letter compliance vendor?

Ask how the tool is checked, not only how it reads. Any product can show a ballot that looks right; the questions below test whether a wrong one would be caught before an investor sees it.

Where Side Letter Management fits in side letter compliance

The solution covers the reading, the calls, the second check, the packets, the reading of returned forms and the obligations, with people at every point where something binds the fund or reaches an investor. It does not replace counsel's judgment on an ambiguous clause, and it does not negotiate side letters. Testing the fees a fund actually charged against the LPA and the elected terms is done by Botminds' LPA Compliance.

How each call is built, and what makes it defensible to a disputing investor, is set out in the most favored nation clause in private equity. To see the two agents work on your own letters, start from Side Letter Management.

Questions people ask

Is it safe to put side letters into an AI tool?

Only into one that keeps them confidential: letters readable on a need-to-know basis, held in the region the investor's terms require, and not used to train shared models. In the sample firm, EU investors' letters stay in the EU region and the platform administrator cannot read side letter text.

Does a second model make the first one unnecessary to check?

No. It catches errors the two models do not share and turns their disagreements into questions for counsel. Shared errors, wrong inputs and ambiguous clauses still need the people and controls around the models.

How many calls does a person still review?

Counsel approves every packet, but reviews in detail only the calls the agents flag. By day 49 of the sample season, three ballots were held for a disagreement; the others went to counsel with every call agreed and cited.

Can an investor see why a term was excluded?

Yes, and it should. In the sample, excluded terms stay on the investor's ballot, greyed, each with the clause that excludes it, so the investor's counsel can check the reasoning rather than ask for it.

Does using AI change who is responsible for side letter compliance?

No. The adviser remains responsible for what it tells investors and for keeping its promises; the tool changes who does the reading, not who is accountable for the result.