WorkflowInvestment Operations & Middle Office

Decide · going live

ISDA data extraction tool: how legal and collateral teams trust it

An ISDA data extraction tool earns trust by showing its disagreements, not by hiding them. Run it read-only beside your current abstracts, show legal every difference, and let it near the collateral system only once override rates fall below a level you set in advance, which is how ISDA and CSA Data Extraction goes live.

Updated · 13 min read

See one case, screen by screen →
Watch with transcript →

An ISDA data extraction tool reads executed ISDA Master Agreements, Schedules, credit support annexes and their amendments, and returns the terms in force as data a collateral system and risk engine can use. Buying one is the easy part; getting a derivatives lawyer to confirm its output and a collateral operations lead to let it change her system is the hard part. Trust comes from three things a team can see: every value cited to its clause, every disagreement between readers shown to a person, and a record of how often people overrule the tool, measured before it writes anything. The rollout that builds that trust starts read-only beside the abstracts you already have, and moves to loading systems only when the override rate is below a level the firm chose in advance. Botminds' ISDA and CSA Data Extraction goes live in exactly those stages, with an analyst, a lawyer and each system's owner signing every change.

Four stages to go live, each ending at a checkpoint the firm sets in advance

What runs
Set up and back-testSetup
Read-only beside current abstractsShadow
Loads for one fund type or dealerFirst loads
Whole estate, nightly comparisonLive
  • Set up and back-testSetup
  • Read-only beside current abstractsShadow
  • Loads for one fund type or dealerFirst loads
  • Whole estate, nightly comparisonLive
The checkpoint to move on
Back-test errors explainedCheck 1
Override rate below the set levelCheck 2
Every load read back cleanCheck 3
Version tested on the golden setOngoing
  • Back-test errors explainedCheck 1
  • Override rate below the set levelCheck 2
  • Every load read back cleanCheck 3
  • Version tested on the golden setOngoing
Stages of ISDA and CSA Data Extraction's rollout, drawn equal because each lasts as long as its checkpoint takes; the levels are the firm's settings, not ours. First-time setup connects systems read-only, loads the estate, maps each election to system fields and runs a back-test on agreements the team abstracted by hand.

The examples come from a fictional sample firm, Quillfield Capital, with 2,400 executed ISDA, CSA, GMRA and GMSLA agreements, as it runs through the solution. Regulators' expectations are linked where they appear.

What should an ISDA data extraction tool do before anyone trusts it?

Four things, before any accuracy claim. It should cite every value to its document, paragraph and page, in the original language for local-law annexes, so a reviewer checks the source rather than the tool. It should read every term twice, independently, and show disagreements rather than resolve them. It should apply amendments and protocol letters in effective-date order and stop when documents conflict. And it should not write to any system without a lawyer's confirmation and the system owner's approval. A tool that cannot do these is asking to be trusted; one that can is asking to be checked, which is a much easier request.

These are also the requirements most worth putting in writing. The design behind them, two readers and three people, is set out in a maker-checker design for AI on legal agreements, and how to test a tool before buying it in replaying it on CSAs you abstracted.

What should go live first?

The comparison, not the writing. The first thing to switch on is a nightly report of where the agreements in force differ from what the collateral system, the risk engine and the counterparty database hold, sized in dollars, with nothing written anywhere. It costs the operations team nothing to read, it surfaces the drifts that matter most on day one, and every item on it is a test case for the tool: either the agreement says what the tool claims, or the reviewer finds out it does not.

In the sample estate one nightly sweep compared 62,410 terms in force with three systems and found 37 differences worth $41.2 million at stake, nine of them over $1 million. A team that starts there has a reason to engage with the tool that is about its own systems, not about the tool.

Run it read-only beside your current abstracts

For the shadow stage, the tool reads everything and changes nothing, and every difference from your existing abstracts goes to a person. Three outcomes are possible for each difference: the tool is wrong, the abstract is wrong, or the clause is genuinely ambiguous. Record which. The tool's error rate is one result; the abstracts' error rate is another, and in the sample firm's back-test the cited pages showed more errors in the hand abstraction, 31, than agent errors, 14, all of which the two readers' disagreement had flagged.

The shadow stage is also when the review rules are set. In ISDA and CSA Data Extraction they are settings changed only with a second approver: terms both readers agree on at 95% or above are shown as agreed and still verified, an analyst re-reads the page below 85%, intake matches a document to its agreement automatically only at 90% or above, and both "both readers must agree" and "nothing is ever auto-confirmed" are locked.

How do you show legal what changed?

Show the clause, the document that changed it and the chain it sits in, in one place. A lawyer confirming a term in force needs to see the base value, each amendment that touched it with its effective date, and the passage itself; asking her to rebuild that from a folder is how review becomes reconstruction. The confirmation should carry a reason code, an attestation that the clause and the chain were read, and a comment for the record.

Legal confirms the terms in force, with a reason and an attestation

Confirm terms in force dialog for the dealer threshold, Brennmoor × Strategic Income Fund: zero under Amendment No. 3 paragraph 2(a), 21 March 2023; the Load agent will prepare two loads, for the collateral system and the risk engine, each approved by its owner. Reason codes: amendment executed and in force, signed by both parties with the effective date passed (selected); original agreement, never loaded; new agreement executed; protocol adherence applies. A comment field goes to the audit trail, and a tick box reads I have read the cited clause and the amendment chain. Behind it, the election grid shows the term's history from USD 10,000,000 in the 1994 annex in 2014 to zero in 2023, and both systems still holding USD 10,000,000.
ISDA and CSA Data Extraction on the fictional sample estate. Confirming changes nothing in a system; it releases two loads that their owners must still approve.

What does the risk system receive, and when?

Only confirmed terms, only after its owner approves, and only once each night's comparison has run. When legal confirms a term in force, the solution prepares one load per system that holds it, typically one for the collateral system and one for the risk engine, and each goes to that system's owner. On approval the Load agent pushes the value, keeps the earlier record as a restorable version and reads the value back from the system to prove it landed. The comparison runs again after the risk engine's end-of-day run, so a load that did not hold shows up the next morning.

Timing is part of the settings. In the sample firm, any difference over $1 million that has waited more than four hours for an owner is escalated to the Head of Investment Operations, who can watch but cannot approve in the owner's place; differences between $250,000 and $1 million go to legal within one business day, and smaller ones within two. Owners see what approving a load will do to the next margin cycle before they approve it.

What happens to agreements the agent cannot read?

They go to a person, and the tool says so instead of guessing. In ISDA and CSA Data Extraction the Intake agent never reads a page it cannot read: a scan below the legibility setting (92% in the sample firm) is treated as unreadable and routed to a person, a document it cannot match to an agreement with at least 90% confidence is matched by a person, and it never creates a counterparty or an agreement on its own. A missing signature page, a side letter signed by one party or two documents that disagree stop the chain for legal.

The count of such agreements is itself a measure worth reporting. A back catalogue with many unreadable scans or unmatched amendments has a document problem the tool has made visible, not one it has created.

How do you measure trust? Override and disagreement rates

By how often people overrule what the tool says, measured per field and watched over time. Trust that rests on a vendor's accuracy claim lasts until the first wrong call; trust that rests on your own override rate is evidence you can show an auditor. The table below lists the measures we would track from the first week.

Five measures of whether an ISDA data extraction tool can be trusted, and what each tells you

MeasureHow it is countedWhat a rising number means
Override rateTerms both readers agreed on that the analyst or legal changed, per 1,000 terms, by fieldThe tool is confidently wrong on that field; stop loads for it and investigate
Disagreement rateTerms the two readers read differently, per 1,000 terms, by document typeHarder documents or a weaker reader; not an error in itself
Legal reversal rateTerms an analyst verified that legal then changedAmbiguous clauses or a verification step that is being rushed
Stopped chainsAgreements where documents conflict or the order is unclear, per batchFiling problems in the document estate
Read-back failuresLoads whose value read back from the system did not matchA mapping or connector problem; no further loads to that system until explained
Our list of measures. The levels at which each one pauses a stage are the firm's to set and to record before the rollout starts, so the checkpoint cannot be moved to fit the result.

The override rate is the checkpoint between shadow and first loads. Set its level before the shadow stage starts, per field: a firm might accept a higher override rate on notification times than on thresholds. If the rate for a field does not fall below the level, that field stays read-only while others go live.

Who does what during the rollout of ISDA and CSA Data Extraction

Documentation analyst

Today

Shadow stage: compares every term the tool reads with her own abstract and records who was wrong.

With agents

Live: verifies agreed terms in one step and reads each disagreement with both passages beside her.

Still theirs: verification of every term

Derivatives lawyer

Today

Shadow stage: settles each mismatch against the clause and marks genuinely ambiguous ones.

With agents

Live: confirms the terms in force with a reason code and an attestation.

Still theirs: which document governs

Collateral system owner

Today

Shadow stage: reads the nightly differences and checks the largest against her own records.

With agents

Live: approves each load to her system and sees it read back.

Still theirs: every write to her system

Risk engine owner

Today

Shadow stage: checks how differences are sized from his engine’s exposure.

With agents

Live: approves risk engine loads separately from collateral operations.

Still theirs: every write to his engine

Head of investment operations

Today

Sets the checkpoints: override levels, materiality and escalation times, approved with a second person.

With agents

Approves new agent versions after the golden-set test, and receives escalations.

Still theirs: changes to rules and agents

The fictional sample firm's roles. Administrators configure and never approve; agents never verify, confirm or approve.

What will examiners and auditors ask for?

Evidence that the terms in your systems match the agreements, and that people, not software, decided they do. The banking agencies' counterparty guidance asks banks to take "corrective action if documents are not current, active and enforceable", and to automate the tracking of legal documentation where volumes are large (Interagency counterparty credit risk guidance, SR 11-10). A US swap dealer's documentation policies are audited periodically by an independent internal or external auditor (17 CFR 23.504(c)), and the largest financial companies must produce agreement and collateral records within 24 hours, current to the previous end of day (31 CFR part 148). Your dealers carry those obligations, and their questions to you follow from them.

On the AI side, the SEC's examination division says it will prioritise policies for monitoring what it calls "super intelligence", including its use in back-office operations (SEC, Fiscal Year 2027 Examination Priorities), and FINRA lists among effective practices the logging of prompts and outputs and tracking which model version was used (FINRA, 2026 oversight report). The Bank of England and FCA's 2024 survey adds a supplier question: the top three third-party providers accounted for 73% of reported cloud providers, 44% of model providers and 33% of data providers among responding UK firms (Bank of England and FCA, November 2024). Ask where your tool's models run and what they keep.

An append-only trail an auditor can read: who, what, why, evidence, version

Audit trail for an auditor with read-only access, 18,420 entries since go-live, with buttons to verify the chain and export. The latest entries show the Load agent version 1.2 pushing a corrected threshold and reading back zero; the collateral system owner approving the load with the reason matches the agreement in force, confirmed by legal; the legal reviewer confirming the threshold under Amendment No. 3 with the reason amendment executed and in force; her questions to the agents; and a Drift agent version 2.3 escalation of a $2.4M load waiting more than four hours. Each entry has a fingerprint chained to the one before.
ISDA and CSA Data Extraction on the fictional sample estate. Every agent step, question, confirmation, approval and load is on the trail with the evidence it rested on, and the trail can be exported for auditors and the fund board.

Questions to ask an ISDA data extraction tool vendor before go-live

Before go-live the questions move from accuracy to control. These are the ones we would ask of any vendor, ours included.

Go-live questions for any ISDA data extraction tool

  1. Are the connectors read-only until we say otherwise?

    The comparison stage should need read access only; write access should arrive later, per system, with an owner.

  2. Can every write be undone?

    Earlier values kept as versions and restorable, and every write read back from the system.

  3. Who sets the checkpoints, and can they be moved?

    Override levels, materiality and escalation times as settings, changed only with a second approver, every change recorded.

  4. How is a new model or agent version tested?

    On a fixed set of hand-checked agreements before anyone approves it; in the sample firm, 400 agreements, approved by the Head of Investment Operations.

  5. Where do our documents and data live?

    In the region of the entity that owns them, with model providers keeping nothing.

  6. Can we take the data with us?

    Terms in force as CSV, as Common Domain Model JSON, and as a term sheet with every source passage.

  7. What happens to what the tool cannot read?

    Unreadable pages, unmatched documents and conflicting documents routed to people and counted.

  8. What can an auditor see?

    An append-only trail with who, what, why, the evidence and the version, exportable and verifiable.

Our checklist. The sample answers come from ISDA and CSA Data Extraction's settings and design in the fictional sample firm.

The Common Domain Model question is worth a sentence of its own. Exporting terms in the industry's data model makes them portable, and research has started to map derivative contracts into it directly; a 2025 preprint describes a template-driven method for doing so with language models and retrieval (Mridul et al., arXiv 2506.01063, not peer reviewed). A portable format does not make a term right. The confirmation chain does, and it should travel with the data.

Questions people ask

What are the requirements for ISDA data extraction?

Every value cited to its page, the amendment chain applied by effective date, two independent readings with disagreements shown to a person, legal confirmation of the terms in force, and no write to a system without its owner's approval. Without those, extracted data is a draft.

How long does it take to go live with an ISDA data extraction tool?

As long as its checkpoints take. Setup connects systems read-only, loads the estate, maps elections to system fields and runs a back-test on agreements your team abstracted by hand; loads begin only once override rates for a field are below the level you set.

What is an override rate?

The share of terms the tool's two readers agreed on that a person then changed, counted per field. It is the best single measure of whether a tool is confidently wrong, and it should fall during the shadow stage before the tool writes anything.

Can an ISDA extraction tool update our collateral system automatically?

It should not without two approvals: legal's confirmation of the term and the system owner's approval of the load. In ISDA and CSA Data Extraction that rule is fixed, and every write is versioned and read back.

Does an ISDA data extraction tool replace the documentation team?

No. It moves the team's time from finding, ordering and keying to verifying and deciding. The analyst still verifies every term, and the lawyer still confirms every term in force; the design target in the sample firm is 12 minutes of a person's time per agreement.

ISDA and CSA Data Extraction.
See it on your own files.

An agentic solution from Botminds. Reads ISDA, CSA, GMRA and GMSLA agreements into structured terms for collateral and risk teams Book a demo, or explore how it works.