WorkflowInvestment Operations & Middle Office

Evaluate · AI on legal agreements

AI for legal agreements: a maker-checker design for CSA terms

AI for legal agreements can extract the terms of an ISDA credit support annex, but a single reading, however good, is not data a collateral system should run on. ISDA and CSA Data Extraction uses two independent AI readers and three people, and treats every disagreement as the most useful thing the AI produces.

Updated · 13 min read

See one case, screen by screen →
Watch with transcript →

AI for legal agreements can read an ISDA Master Agreement and its credit support annex and pull out the terms that matter, from the threshold and minimum transfer amount to eligible collateral, haircuts and termination events. The question for a collateral or legal team is not whether it can read, but what a reading is worth before anyone acts on it. Research on contract datasets shows models reading contracts unevenly, courts show that even careful human readers can find two meanings in one clause, and regulators expect a person to review what AI produces. The design that answers all three is maker-checker: one reader extracts, a second reads independently, any disagreement goes to a person, and a lawyer confirms. Botminds' ISDA and CSA Data Extraction is built that way, with two AI readers that must agree, an analyst who verifies, legal who confirms the terms in force, and system owners who approve every change.

Maker, checker, person, lawyer, owner: how a CSA term becomes data

  1. Agent

    Extraction agent (maker)

    Reads the annex, Schedule and amendments in the language signed and records each term with document, paragraph and page.

    Never: resolves a disagreement

  2. Agent

    Checker agent

    Reads the same clauses blind, with a second model family and a different method, and agrees or shows both readings side by side.

    Never: sees the first reading or picks a winner

  3. A person decides

    Documentation analyst

    Verifies terms both readers agree on, and reads every disagreement with both passages in front of her.

    Never: confirms what she verified

  4. A person decides

    Derivatives lawyer

    Confirms the terms in force, with a reason code and an attestation that the clause and the amendment chain were read.

  5. A person decides

    System owner

    Approves the load to the collateral system or the risk engine; the value is pushed, versioned and read back.

    Never: approves a term she confirmed

The roles in ISDA and CSA Data Extraction. Nothing is ever auto-confirmed, whatever the readers' confidence; agents never verify, confirm or approve.

The examples on this page come from a fictional sample firm, Quillfield Capital, with 2,400 executed ISDA, CSA, GMRA and GMSLA agreements, as it runs through the solution. Research is linked where it appears; preprints are labelled as not peer reviewed.

Can AI extract terms from legal agreements? Yes, and the research says how carefully

Yes, with caveats the research makes specific. CUAD, a contract review dataset built with dozens of lawyers and more than 13,000 expert annotations, found transformer models at an early stage on the task of pointing reviewers to the clauses that matter, with results that depended heavily on model design and training data (Hendrycks et al., CUAD, NeurIPS 2021). ContractNLI, built on 607 annotated contracts, asked systems whether a contract supports, contradicts or is silent on a statement, and found existing models did badly, with features of legal drafting such as negation by exception among the reasons (Koreeda and Manning, Findings of EMNLP 2021).

Work closer to derivatives points the same way. A 2025 preprint describes CDMizer, a template-driven framework that uses a language model with retrieval to turn over-the-counter derivative contracts into the industry's Common Domain Model, and argues that a template-driven approach "guarantees syntactic correctness, schema adherence, and improved scalability" compared with generating the structure freely; the same paper proposes a separate model-based check of "the completeness and accuracy of structured representations" (Mridul et al., arXiv 2506.01063, not peer reviewed). An earlier preprint examined which provisions of the ISDA Master Agreement are amenable to automation at all (Clack and McGonagle, arXiv 1904.01461, not peer reviewed). Neither publishes accuracy on credit support annex terms, and no regulator or independent study does. The test that counts is yours, and how to replay an AI tool on CSAs you already abstracted sets it out.

Why one reader is never enough: two available meanings

Because legal text can carry two readings that a careful reader would each defend, and a single reader, human or machine, reports only one. The clearest example is a court's. In 2019 the Court of Appeal had to decide whether a 1995 English-law credit support annex obliged the bank to account for negative interest on cash collateral. Each side argued that the drafter "could have said specifically what the other contends for, had it been intended", and the court accepted that both readings were available before deciding for the bank (State of the Netherlands v Deutsche Bank AG [2019] EWCA Civ 771).

“We have thought carefully about these interpretations and can accept that each is an available meaning of the words used.”

Court of Appeal, State of the Netherlands v Deutsche Bank AG, 2019

An AI reader that returns one value with a high confidence score hides the second meaning. A second reader that works independently is the cheapest way to surface it: if both read the clause the same way, the meaning is probably settled; if they differ, the difference is exactly where a person should look. The disagreement is not a failure of the system. It is the system working.

How does maker-checker work with two AI readers?

It works only if the two readings are independent. In ISDA and CSA Data Extraction the Extraction agent reads first and records each value with its document, paragraph and page. The Checker agent reads the same clauses without seeing the first reading, using a second model family and a different method, and reports either that both readings agree or both passages side by side. A checker that is shown the first answer can simply agree with it, which turns two readers into one.

One AI reader with spot checks, against two independent readers and three people

One reader, spot-checkedMaker-checker in ISDA and CSA Data Extraction
Where errors are found

One reader, spot-checked: In the sample a person happens to check, or when a dealer disputes a call.

Maker-checker in ISDA and CSA Data Extraction: Wherever the two readers disagree, on every term of every agreement.

A confident wrong answer

One reader, spot-checked: Goes to the system unless sampled.

Maker-checker in ISDA and CSA Data Extraction: Must also fool an independent reader using a different method.

A clause with two meanings

One reader, spot-checked: Recorded with one of them.

Maker-checker in ISDA and CSA Data Extraction: Shown with both passages and both readings to a person.

Amendments in conflict

One reader, spot-checked: Whichever was read last wins.

Maker-checker in ISDA and CSA Data Extraction: The Amendment agent stops and asks legal.

What reaches a system

One reader, spot-checked: The model’s value.

Maker-checker in ISDA and CSA Data Extraction: A value verified by an analyst, confirmed by legal and approved by the system owner.

Evidence for an auditor

One reader, spot-checked: The output.

Maker-checker in ISDA and CSA Data Extraction: Both readings, the clause, three names and the reason codes, on an append-only trail.

The right-hand column describes the solution's design and its settings, which are locked: both readers must agree, and nothing is auto-confirmed.

What happens when the readers disagree?

The term stops and goes to a person with both passages. The documentation analyst sees what each reader extracted, the page each relied on and the clause itself, in the original language with the English beside it, and decides which is right or whether the clause needs legal. In the sample estate, the analyst's queue showed the Asia Bond Fund's English-law variation margin annex held because the readers transcribed its notification time differently, and a German collateral annex where both readers agreed on 22 of 26 terms and four were for her to read first, including a haircut outside the firm's guideline.

Some disagreements are not between readers but between documents. A side letter signed by only one party set a lower minimum transfer amount than the signed amendment to the sample firm's repo agreement for its Short Duration Bond ETF. The Amendment agent does not choose between them; it stops and asks legal, because which document governs is a legal judgment, not a reading.

The analyst's review: agreed terms in one step, disagreements read first

Review screen for Priya Natarajan, documentation analyst: 46 agreements in review, 32 to verify and 14 with legal. The selected German annex between Brennmoor and the Euro Credit UCITS fund shows four steps, read by two agents, analyst verifies, legal confirms, owners approve loads. The Extraction agent reports reading the German annex in the original with the English beside it: 22 of 26 terms agreed by both readers at 95% or above, 4 to read, including a haircut outside the UCITS guideline. Buttons: verify 22 agreed terms; go to the haircut. A note says legal confirms after the analyst, who cannot confirm what she verified. A table lists each election, its value, clause and the readers' agreement.
ISDA and CSA Data Extraction on the fictional sample estate. Agreement between the two readers lets the analyst verify quickly; it never lets a term skip her.

When must a person confirm? Every time

Every extracted term is verified by a person and confirmed by legal before any system uses it. Confidence thresholds decide how much attention a term gets, not whether it gets any. In the solution's settings, terms both readers agree on at 95% or above are shown as agreed but still verified; below 85% an analyst re-reads the page; and "nothing is ever auto-confirmed" is locked, not a setting a user can relax. Changes to the thresholds need a second approver.

Review rules: both readers must agree, and nothing is auto-confirmed

Settings, review rules and thresholds, shown to an administrator: both readers must agree, on and locked, with any disagreement going to a person; terms agreed at or above 95% shown as agreed but still verified by a person; below 85% an analyst re-reads the page; intake auto-matches documents at or above 90%; scans with OCR legibility below 92% are treated as unreadable; nothing is ever auto-confirmed, locked, every value verified by an analyst and confirmed by legal before any load. Changes need a second approver.
ISDA and CSA Data Extraction's review rules, as configured in the fictional sample firm. The percentages are settings; the two locked rules are not.

Three people, three questions: verify, confirm, approve

The person who verifies a value, the person who confirms it is in force and the person who changes a system answer different questions, and the design keeps them different people. Verification asks whether the value matches the page. Confirmation asks which document governs today: whether the amendment is executed by both parties and effective, whether a protocol letter applies. Approval asks whether the owner's system should change before its next cycle, with the call the change creates in view. The detail of who sits in each seat is in who reads an ISDA CSA agreement.

What do regulators say about human-in-the-loop review?

They expect it, and they are watching how firms use AI in back offices. FINRA's 2026 oversight report says summarisation and information extraction is the most common generative AI use among its member firms, lists the risk of AI agents acting without human validation and approval, beyond their intended authority, or in ways hard to trace, and describes effective practices including keeping prompt and output logs, tracking which model version was used, and human-in-the-loop review of model outputs (FINRA, 2026 Annual Regulatory Oversight Report, December 2025). ESMA warned in 2024 of over-reliance on AI that neglects human judgment (ESMA, 30 May 2024).

The survey evidence shows where firms are. In the Bank of England and FCA's 2024 survey of UK financial firms, operations and IT was the largest area of AI use, at around 22% of use cases, and only 2% of use cases involved fully autonomous decision-making (Bank of England and FCA, November 2024). In the US, the SEC's examination division says it will prioritise firms' policies for monitoring what it calls "super intelligence", including its use in back-office operations (SEC, Fiscal Year 2027 Examination Priorities, October 2026). A design in which a person signs every term is the one that answers those questions without argument.

What to ask any AI for legal agreements before you buy

The questions that separate a demonstration from a control are about what happens after the model reads. The checklist below is the one we would use on any tool, including ours.

Questions to put to any AI for legal agreements before it touches collateral terms

  1. Is there a second, independent reader?

    Ask whether the checker sees the first answer, and whether it uses a different model or method. A checker that sees the answer is a rubber stamp.

  2. Can anything be auto-confirmed?

    Ask what happens to a term read at 99% confidence. If the answer is that it goes straight to a system, a person is not in the loop.

  3. Does every value cite its page?

    Document, paragraph and page for every term, in the original language for local-law annexes, so a reviewer checks the source and not the summary.

  4. How are amendments and protocol letters applied?

    By effective date, with conflicts stopped for legal, or by whatever was read last.

  5. Who can verify, confirm and approve?

    Three different people, enforced by the system, with administrators unable to approve.

  6. What does the tool write to my systems?

    Nothing without the owner’s approval; earlier values kept as versions; every write read back.

  7. How is a new model version tested?

    On a fixed set of agreements your team checked by hand, before anyone approves it for use.

  8. What is logged?

    Every reading, question, confirmation and approval, with the model version, on a trail that cannot be edited.

Our checklist. The last two items reflect FINRA's described effective practices on model version tracking and output logs (2026 report).Source: FINRA, 2026 oversight report

Where AI for legal agreements fits, and where it does not

AI fits the reading, comparing and sizing; it does not fit deciding. In ISDA and CSA Data Extraction the agents read every clause twice, apply amendments in date order, compare every term in force with the collateral system, the risk engine and the counterparty database each night, and size each difference in dollars. They never verify, confirm or approve, and the agent that loads a corrected value writes only after legal's confirmation and the owner's approval.

It is also not a negotiation tool or a search engine for every clause in every document; drafting and negotiating the terms belongs to ISDA Negotiation, and finding any clause across a documentation estate to Derivatives Documentation Search. What maker-checker adds here is narrower and, for collateral, more important: a term that reaches a margin call has been read twice, checked by a person, confirmed by a lawyer and approved by the system's owner. How a team moves from first trial to trusting that output is the subject of rolling out an ISDA data extraction tool.

Questions people ask

What is the best AI for legal agreements?

The one that performs on your own documents under your own controls. No regulator or independent study publishes accuracy for derivatives collateral terms, so a vendor's benchmark on general contracts says little about your annexes. Replay any tool on agreements your team has already abstracted and score it field by field.

Can AI review legal documents without a lawyer?

It can read them, but someone has to own the conclusion. For collateral terms the conclusion moves money, so a lawyer should confirm which document governs and what it says. FINRA lists human-in-the-loop review of model outputs among the effective practices it observed.

What is maker-checker in document extraction?

A control in which one party produces a result and a second, independent party checks it before it is used. With AI, the maker and checker can both be models, as long as the checker does not see the maker's answer, and a person resolves every disagreement.

Should two AI models read the same contract?

For terms that drive payments, yes. Two independent readings surface the clauses where meaning is uncertain, which is where both machine and human errors cluster. Where they agree, a person can verify quickly; where they differ, the person reads both passages.

Is AI-extracted contract data accurate enough for a collateral system?

Only after a person has verified it, legal has confirmed it and the system's owner has approved the load. Accuracy without that chain is a claim about a model; with it, the value has three named people behind it and a clause anyone can check.

ISDA and CSA Data Extraction.
See it on your own files.

An agentic solution from Botminds. Reads ISDA, CSA, GMRA and GMSLA agreements into structured terms for collateral and risk teams Book a demo, or explore how it works.