An ISDA data extraction tool reads executed ISDA Master Agreements, Schedules, credit support annexes and their amendments, and returns the terms in force as data a collateral system and risk engine can use. Buying one is the easy part; getting a derivatives lawyer to confirm its output and a collateral operations lead to let it change her system is the hard part. Trust comes from three things a team can see: every value cited to its clause, every disagreement between readers shown to a person, and a record of how often people overrule the tool, measured before it writes anything. The rollout that builds that trust starts read-only beside the abstracts you already have, and moves to loading systems only when the override rate is below a level the firm chose in advance. Botminds' ISDA and CSA Data Extraction goes live in exactly those stages, with an analyst, a lawyer and each system's owner signing every change.
Four stages to go live, each ending at a checkpoint the firm sets in advance
- Set up and back-testSetup
- Read-only beside current abstractsShadow
- Loads for one fund type or dealerFirst loads
- Whole estate, nightly comparisonLive
- Back-test errors explainedCheck 1
- Override rate below the set levelCheck 2
- Every load read back cleanCheck 3
- Version tested on the golden setOngoing
The examples come from a fictional sample firm, Quillfield Capital, with 2,400 executed ISDA, CSA, GMRA and GMSLA agreements, as it runs through the solution. Regulators' expectations are linked where they appear.
What should an ISDA data extraction tool do before anyone trusts it?
Four things, before any accuracy claim. It should cite every value to its document, paragraph and page, in the original language for local-law annexes, so a reviewer checks the source rather than the tool. It should read every term twice, independently, and show disagreements rather than resolve them. It should apply amendments and protocol letters in effective-date order and stop when documents conflict. And it should not write to any system without a lawyer's confirmation and the system owner's approval. A tool that cannot do these is asking to be trusted; one that can is asking to be checked, which is a much easier request.
These are also the requirements most worth putting in writing. The design behind them, two readers and three people, is set out in a maker-checker design for AI on legal agreements, and how to test a tool before buying it in replaying it on CSAs you abstracted.
What should go live first?
The comparison, not the writing. The first thing to switch on is a nightly report of where the agreements in force differ from what the collateral system, the risk engine and the counterparty database hold, sized in dollars, with nothing written anywhere. It costs the operations team nothing to read, it surfaces the drifts that matter most on day one, and every item on it is a test case for the tool: either the agreement says what the tool claims, or the reviewer finds out it does not.
In the sample estate one nightly sweep compared 62,410 terms in force with three systems and found 37 differences worth $41.2 million at stake, nine of them over $1 million. A team that starts there has a reason to engage with the tool that is about its own systems, not about the tool.
Run it read-only beside your current abstracts
For the shadow stage, the tool reads everything and changes nothing, and every difference from your existing abstracts goes to a person. Three outcomes are possible for each difference: the tool is wrong, the abstract is wrong, or the clause is genuinely ambiguous. Record which. The tool's error rate is one result; the abstracts' error rate is another, and in the sample firm's back-test the cited pages showed more errors in the hand abstraction, 31, than agent errors, 14, all of which the two readers' disagreement had flagged.
The shadow stage is also when the review rules are set. In ISDA and CSA Data Extraction they are settings changed only with a second approver: terms both readers agree on at 95% or above are shown as agreed and still verified, an analyst re-reads the page below 85%, intake matches a document to its agreement automatically only at 90% or above, and both "both readers must agree" and "nothing is ever auto-confirmed" are locked.
How do you show legal what changed?
Show the clause, the document that changed it and the chain it sits in, in one place. A lawyer confirming a term in force needs to see the base value, each amendment that touched it with its effective date, and the passage itself; asking her to rebuild that from a folder is how review becomes reconstruction. The confirmation should carry a reason code, an attestation that the clause and the chain were read, and a comment for the record.
Legal confirms the terms in force, with a reason and an attestation

What does the risk system receive, and when?
Only confirmed terms, only after its owner approves, and only once each night's comparison has run. When legal confirms a term in force, the solution prepares one load per system that holds it, typically one for the collateral system and one for the risk engine, and each goes to that system's owner. On approval the Load agent pushes the value, keeps the earlier record as a restorable version and reads the value back from the system to prove it landed. The comparison runs again after the risk engine's end-of-day run, so a load that did not hold shows up the next morning.
Timing is part of the settings. In the sample firm, any difference over $1 million that has waited more than four hours for an owner is escalated to the Head of Investment Operations, who can watch but cannot approve in the owner's place; differences between $250,000 and $1 million go to legal within one business day, and smaller ones within two. Owners see what approving a load will do to the next margin cycle before they approve it.
What happens to agreements the agent cannot read?
They go to a person, and the tool says so instead of guessing. In ISDA and CSA Data Extraction the Intake agent never reads a page it cannot read: a scan below the legibility setting (92% in the sample firm) is treated as unreadable and routed to a person, a document it cannot match to an agreement with at least 90% confidence is matched by a person, and it never creates a counterparty or an agreement on its own. A missing signature page, a side letter signed by one party or two documents that disagree stop the chain for legal.
The count of such agreements is itself a measure worth reporting. A back catalogue with many unreadable scans or unmatched amendments has a document problem the tool has made visible, not one it has created.
How do you measure trust? Override and disagreement rates
By how often people overrule what the tool says, measured per field and watched over time. Trust that rests on a vendor's accuracy claim lasts until the first wrong call; trust that rests on your own override rate is evidence you can show an auditor. The table below lists the measures we would track from the first week.
Five measures of whether an ISDA data extraction tool can be trusted, and what each tells you
| Measure | How it is counted | What a rising number means |
|---|---|---|
| Override rate | Terms both readers agreed on that the analyst or legal changed, per 1,000 terms, by field | The tool is confidently wrong on that field; stop loads for it and investigate |
| Disagreement rate | Terms the two readers read differently, per 1,000 terms, by document type | Harder documents or a weaker reader; not an error in itself |
| Legal reversal rate | Terms an analyst verified that legal then changed | Ambiguous clauses or a verification step that is being rushed |
| Stopped chains | Agreements where documents conflict or the order is unclear, per batch | Filing problems in the document estate |
| Read-back failures | Loads whose value read back from the system did not match | A mapping or connector problem; no further loads to that system until explained |
The override rate is the checkpoint between shadow and first loads. Set its level before the shadow stage starts, per field: a firm might accept a higher override rate on notification times than on thresholds. If the rate for a field does not fall below the level, that field stays read-only while others go live.
Who does what during the rollout of ISDA and CSA Data Extraction
Today
Shadow stage: compares every term the tool reads with her own abstract and records who was wrong.
With agents
Live: verifies agreed terms in one step and reads each disagreement with both passages beside her.
Still theirs: verification of every term
Today
Shadow stage: settles each mismatch against the clause and marks genuinely ambiguous ones.
With agents
Live: confirms the terms in force with a reason code and an attestation.
Still theirs: which document governs
Today
Shadow stage: reads the nightly differences and checks the largest against her own records.
With agents
Live: approves each load to her system and sees it read back.
Still theirs: every write to her system
Today
Shadow stage: checks how differences are sized from his engine’s exposure.
With agents
Live: approves risk engine loads separately from collateral operations.
Still theirs: every write to his engine
Today
Sets the checkpoints: override levels, materiality and escalation times, approved with a second person.
With agents
Approves new agent versions after the golden-set test, and receives escalations.
Still theirs: changes to rules and agents
What will examiners and auditors ask for?
Evidence that the terms in your systems match the agreements, and that people, not software, decided they do. The banking agencies' counterparty guidance asks banks to take "corrective action if documents are not current, active and enforceable", and to automate the tracking of legal documentation where volumes are large (Interagency counterparty credit risk guidance, SR 11-10). A US swap dealer's documentation policies are audited periodically by an independent internal or external auditor (17 CFR 23.504(c)), and the largest financial companies must produce agreement and collateral records within 24 hours, current to the previous end of day (31 CFR part 148). Your dealers carry those obligations, and their questions to you follow from them.
On the AI side, the SEC's examination division says it will prioritise policies for monitoring what it calls "super intelligence", including its use in back-office operations (SEC, Fiscal Year 2027 Examination Priorities), and FINRA lists among effective practices the logging of prompts and outputs and tracking which model version was used (FINRA, 2026 oversight report). The Bank of England and FCA's 2024 survey adds a supplier question: the top three third-party providers accounted for 73% of reported cloud providers, 44% of model providers and 33% of data providers among responding UK firms (Bank of England and FCA, November 2024). Ask where your tool's models run and what they keep.
An append-only trail an auditor can read: who, what, why, evidence, version

Questions to ask an ISDA data extraction tool vendor before go-live
Before go-live the questions move from accuracy to control. These are the ones we would ask of any vendor, ours included.
Go-live questions for any ISDA data extraction tool
- Are the connectors read-only until we say otherwise?
The comparison stage should need read access only; write access should arrive later, per system, with an owner.
- Can every write be undone?
Earlier values kept as versions and restorable, and every write read back from the system.
- Who sets the checkpoints, and can they be moved?
Override levels, materiality and escalation times as settings, changed only with a second approver, every change recorded.
- How is a new model or agent version tested?
On a fixed set of hand-checked agreements before anyone approves it; in the sample firm, 400 agreements, approved by the Head of Investment Operations.
- Where do our documents and data live?
In the region of the entity that owns them, with model providers keeping nothing.
- Can we take the data with us?
Terms in force as CSV, as Common Domain Model JSON, and as a term sheet with every source passage.
- What happens to what the tool cannot read?
Unreadable pages, unmatched documents and conflicting documents routed to people and counted.
- What can an auditor see?
An append-only trail with who, what, why, the evidence and the version, exportable and verifiable.
The Common Domain Model question is worth a sentence of its own. Exporting terms in the industry's data model makes them portable, and research has started to map derivative contracts into it directly; a 2025 preprint describes a template-driven method for doing so with language models and retrieval (Mridul et al., arXiv 2506.01063, not peer reviewed). A portable format does not make a term right. The confirmation chain does, and it should travel with the data.
Questions people ask
What are the requirements for ISDA data extraction?
Every value cited to its page, the amendment chain applied by effective date, two independent readings with disagreements shown to a person, legal confirmation of the terms in force, and no write to a system without its owner's approval. Without those, extracted data is a draft.
How long does it take to go live with an ISDA data extraction tool?
As long as its checkpoints take. Setup connects systems read-only, loads the estate, maps elections to system fields and runs a back-test on agreements your team abstracted by hand; loads begin only once override rates for a field are below the level you set.
What is an override rate?
The share of terms the tool's two readers agreed on that a person then changed, counted per field. It is the best single measure of whether a tool is confidently wrong, and it should fall during the shadow stage before the tool writes anything.
Can an ISDA extraction tool update our collateral system automatically?
It should not without two approvals: legal's confirmation of the term and the system owner's approval of the load. In ISDA and CSA Data Extraction that rule is fixed, and every write is versioned and read back.
Does an ISDA data extraction tool replace the documentation team?
No. It moves the team's time from finding, ordering and keying to verifying and deciding. The analyst still verifies every term, and the lawyer still confirms every term in force; the design target in the sample firm is 12 minutes of a person's time per agreement.