Contract obligation tracking software finds the promises in signed contracts, such as restrictions, deliverables, fee terms, notice periods and consents, assigns each to an owner and tracks it until it is met. The way to choose one is not a demo on the vendor's sample contract but a seeded test on yours: take a real-shaped agreement bundle, plant fifty obligations whose answers you already know, including the hard cases, run the tool, and score what comes back on recall, precision, citation and owner, type by type. For an asset manager the bundle is an investment management agreement, its investment guidelines and fee schedule, and a side letter that changes some of it. Botminds' Investment Management Agreement Review was built for that bundle: agents turn every clause into an obligation with its quote and page, a second agent re-reads each one blind, and each team's lead accepts its own. This guide gives the test so you can judge it, or any other tool, on your own documents.
The bundle design and scoring sheet below are ours. The examples of how the solution behaves come from a fictional sample firm, Quillfield Capital, and its fictional client Marlow County Retirement System. Research on how well machines read contracts is cited from peer-reviewed papers.
A seeded test in three numbers
50obligations
planted in one IMA bundle, each with a known type, page and owner
10hard cases
among them: overrides, exceptions, cross-references, an unsigned draft
4scores
recall, precision, citation and owner, by obligation type
What should contract obligation tracking software prove before you buy it?
It should prove four things on your documents: that it finds the obligations that are there, that it does not invent ones that are not, that each obligation it returns is tied to the exact words and page it came from, and that it sends each one to the right owner. A tool that finds every clause but cites none of them gives a reviewer nothing to check. One that cites every clause but routes the fee terms to legal and the restrictions to billing has moved the work, not done it.
The regulatory reason is the same as for any compliance control. An SEC-registered adviser must have written policies and procedures, actually in use, that are reasonably designed to prevent breaches of the Act, and must review how well they work at least once a year (Rule 206(4)-7). A tool is part of the procedure the moment the firm relies on it, and "reasonably designed" is a claim the firm should be able to support with a test it ran, not a vendor's figure. Fee terms are where examiners look first: the SEC's June 2026 risk alert describes fees out of line with written agreements, including renegotiated arrangements and fee addenda (SEC risk alert, June 9, 2026).
How do you measure recall on contract obligations?
Recall is the share of the obligations you planted that the tool returned; precision is the share of what the tool returned that were real obligations. To measure either you need an answer key written before the run, by someone who will not see the tool's output first, listing each obligation's type, document, page, quoted text and owner. A tool's hit counts only if it matches the key on the obligation, not just the page.
Measure both by type, not overall. A tool with 95% recall overall can still miss half of the fee terms if fee terms are only a tenth of the bundle, and a missed fee term costs more than a missed reporting deadline. Score citations separately: a returned obligation whose quote is not in the document, or whose page is wrong, is a reviewer's problem even if the obligation itself is right. The research on legal AI is a reason to insist on that check. In a preregistered study of AI legal research tools built on retrieval, the tools still produced hallucinated answers somewhere between a sixth and a third of the time (Magesh et al., Journal of Empirical Legal Studies, 2025).
Building the test bundle: fifty obligations an IMA really carries
Start from a real agreement your firm has signed, with names and numbers changed, and its schedules and one side letter. The distribution of obligations should look like your book, because a test weighted toward the easy types flatters every tool. The split below follows the shape of a fixed income separate account, where guideline restrictions dominate.
Fifty planted obligations, weighted the way an IMA bundle is
Notices are the type most often filed and forgotten. A key person clause, for example, decides who must be told, and how fast, when a portfolio manager leaves, as the guide to the key person event clause sets out; plant at least one with a short deadline.
The four overrides are the most important lines in the key. Each is an obligation that exists in the IMA and is changed by the side letter, so the right answer is the side letter's version, and a tool that returns the IMA's version has returned a wrong obligation with a correct citation. In the sample firm's Marlow bundle, the side letter banned securities lending the IMA allowed, cut the deadline for the monthly holdings statement from 15 business days to 10, and reworded a 15% below-investment-grade limit. A tool tested without overrides is tested on the part of the job that rarely goes wrong.
Ten hard cases to plant, and why each one catches tools out
The hard cases are the clauses whose words do not say what a quick reader thinks. Research on machine reading of contracts names some of them. The ContractNLI study, which annotated 607 contracts, found that features of contract language such as negation by exception help make the task hard, and that models then available did badly at it (Koreeda and Manning, Findings of EMNLP 2021). An earlier study of detecting contractual obligations and prohibitions found that a model reading each sentence in the context of the sentences around it did better than one reading sentences alone, "because the hierarchical model has a broader discourse view" (Chalkidis, Androutsopoulos and Michos, ACL 2018). Obligations depend on context; so should the test.
Ten hard cases for the answer key
- 1. A side-letter override
The side letter changes an IMA term and says it governs. Right answer: the side letter's term, with both documents cited.
- 2. An exception inside a limit
"Cash 5% except in transitions." The exception is part of the obligation; dropping it creates a breach that is not one.
- 3. A cross-reference
"The monthly statement of holdings and transactions required by Section 11.1 of the Agreement", in the side letter. The tool must resolve what 11.1 says.
- 4. A clause that reads two ways
A 15% limit "at the time of purchase" in Schedule A and "shall not hold" in the side letter. Right answer: flagged with both readings, not one picked.
- 5. A defined term that changes the number
"Business Day" defined by the client's jurisdiction. Ten business days is a different date in London and New York.
- 6. A number in words and digits
"Fifteen (15)" in one place, "15" in another, "fifteen" in a third. All one obligation.
- 7. A clause in another language
A restriction signed in German or Japanese. The obligation should keep the original and cite it.
- 8. A table in a schedule
Sector limits laid out as a table, or a fee schedule with tiers. Tools that read prose often skip tables.
- 9. An unsigned draft in the bundle
A precision trap: no obligation from a draft should be returned as governing.
- 10. A promise that depends on other clients
An MFN clause: the obligation is found only if its scope, size test, carve-outs and dates are found with it.
Case 9 tests precision, and it is the one vendors' samples never include. In the sample firm's intake, an unsigned renewal draft arrived with no signature, name or date on its execution page; the Intake agent held it and never treats a draft as governing, and the executed copy arrived four minutes later. A tool that reads everything in a folder will happily extract obligations from the draft.
A scoring sheet for recall, precision, citations and owners
The sheet is filled from the answer key and the tool's output, by a person who did not run the tool. Each planted obligation gets one row; each extra obligation the tool returned gets one row too, marked as a false positive or, after review, as a real obligation the key missed.
What to score for each obligation, and what counts as a pass
| Score | Pass when | Why it matters |
|---|---|---|
| Recall, by type | The planted obligation is returned with the right type | A miss is work nobody does: a restriction never coded, a fee never applied |
| Precision | Every returned obligation is in the key, or is real and was missed by the key | False obligations cost reviewers time and teach them to skim |
| Citation | The quoted text is in the document, on the cited page and section | A reviewer must be able to check the obligation without re-reading the bundle |
| Override handling | For each of the four overrides, the side letter's version is returned and the IMA's marked as replaced | The commonest silent error in a separate-account book |
| Ambiguity handling | The two-reading clause is flagged with both readings | A tool that picks one has made a legal decision |
| Owner | The obligation is routed to the team your routing rules name | Right obligation, wrong desk, is still not done |
| Time | Elapsed time from upload to a reviewable list | The cost of the human review that follows matters more than this, but note both |
Set the pass line by consequence, before the run. Most firms will want every planted restriction, fee term and override either found or flagged for a person, and will accept less on low-stakes deliverables. Whatever the line, write it down first: a threshold chosen after seeing the results is a description, not a test.
Running the test: the order of work
The order matters more than the tooling. Write the answer key first and freeze it, so nobody can adjust it after seeing a tool's output; give the tool the bundle exactly as the firm would receive it, including the unsigned draft and the side letter that arrived late; and have the scoring done by someone who neither wrote the key nor ran the tool. Then review every disagreement between key and tool, because some will be errors in the key: a careful tool often finds an obligation the reader missed, and that is a result worth having.
Run the test twice. The second run, on the next version of the tool or a second bundle, shows whether the first score was stable or lucky. It also tests the vendor's change control: a tool whose results move between versions without notice is a tool whose accuracy the firm cannot rely on between tests. In the sample firm's setup, each agent change is versioned, scored against a fixed set of 600 clauses with known answers and signed off before release, and every agent step on the audit trail carries the version that produced it.
Finally, test the hand-off, not only the reading. Send the tool's output to the real owners: the compliance rule analyst, the billing lead, the reporting lead. Ask whether each obligation is something they can act on as written. An obligation that is accurate but unusable, a guideline limit without its measurement basis or a fee term without its fee base, fails the test that matters, which is whether the promise gets kept.
What published research says about machine-read contracts
The best-known benchmark for contract review, CUAD, was built with dozens of legal experts and has more than 13,000 annotations of clauses a lawyer would flag; its authors found transformer models' performance nascent, and strongly shaped by model design and how much training data they had (Hendrycks et al., NeurIPS 2021). A study of general-purpose language models on legal questions found error rates higher still, and that the models struggled to predict their own hallucinations (Dahl et al., Journal of Legal Analysis, 2024).
Two practical conclusions follow for a buyer. First, published benchmarks were built on other contract types, mostly commercial agreements, so their numbers do not transfer to IMAs and side letters; the only number that transfers is the one you measure. Second, a tool should be designed to be checked: quote and page for every obligation, a second independent reading, and a person deciding where the two disagree. In Investment Management Agreement Review, the Verification agent re-reads each clause without seeing the first answer, on a different model; on the sample Marlow bundle the two readers agreed on 40 of 41 obligations, and the one disagreement went to counsel with both readings.
Two independent readings, a drafted rule and a dry run in Investment Management Agreement Review
What a contract obligations tracker template in Excel cannot score
A spreadsheet tracker is the baseline every tool should beat, and it is worth running the seeded test on it too. A person reads the bundle, types each obligation into a row with a page number, assigns an owner and emails the list. Recall depends on that person's attention on the day; citations are as good as their typing; overrides depend on whether the side letter arrived before or after the spreadsheet was written. What the spreadsheet cannot record at all is acceptance: whether each owner took the obligation on.
A tracker template and obligation software, on the same seeded bundle
Spreadsheet tracker: One reader, one pass; recall varies with the reader and the day.
Obligation software worth buying: Every clause read, with a measured recall by type on your own seeded bundle.
Spreadsheet tracker: A page number typed by hand, if any.
Obligation software worth buying: The quote, document, page and section for every obligation, clickable.
Spreadsheet tracker: A second tab, updated if someone remembers.
Obligation software worth buying: Compared clause by clause with what they change; the precedence clause applied.
Spreadsheet tracker: The reader picks one, usually silently.
Obligation software worth buying: Both written out and sent to counsel.
Spreadsheet tracker: An email sent; no proof it was read.
Obligation software worth buying: Each owner accepts or returns its obligations, on a clock, with escalation.
Spreadsheet tracker: Open every file.
Obligation software worth buying: One question across all agreements, answered with each cited.
Where Investment Management Agreement Review fits, and where it does not
The solution fits asset managers whose obligations live in IMAs, investment guidelines, fee schedules, side letters, amendments and client policy statements, often in more than one language, and whose teams are compliance coding, client reporting, billing, trading and legal. It reads executed documents filed from document management, e-signature or email, routes obligations to those teams, drafts compliance rules for the order management system's engine in a sandbox, and tests most-favored-nation clauses against the billing system. It does not draft or negotiate contracts, does not replace e-signature or the document repository, does not calculate invoices, and is not designed for a company's vendor contracts, leases or employment agreements; a general contract management system serves those better.
It is also designed to be tested the way this guide describes. The solution page offers to run it "on a sample of your own IMAs, guidelines and side letters", which is the seeded test in practice; internally, each agent change is versioned, scored on a 600-clause golden set and signed off by the chief compliance officer before release. For fund documents rather than separate accounts, Fund Accounting and Private Markets have their own test: contract obligation management software for LPAs. For what a reviewer should do with the clause that reads two ways, see legal contract review AI tools, and for the comparison with a contract management system, contract management software for legal departments vs agents.
The audit trail a seeded test should leave behind
Questions to ask any contract obligation tracking vendor
The questions below turn the seeded test into a buying conversation. Any vendor whose product is designed to be checked should welcome all of them.
Twelve questions before you sign
- Will you run on our seeded bundle, before contract?
With our answer key held back until the output is in.
- What is recall and precision by obligation type, on our documents?
Not overall, and not on your sample.
- Does every obligation carry its quote, page and section?
And can a reviewer click through to the passage?
- How do you handle a side letter that overrides the agreement?
Show the four overrides in our test.
- What happens when a clause reads two ways?
Both readings to a person, or one picked?
- Is there an independent second reading?
On a different model, without seeing the first answer?
- Do you read documents in the language they were signed in?
With the original kept beside any translation.
- How are unsigned drafts kept out?
Our test bundle includes one.
- Who accepts an obligation, and how is acceptance shown?
By the owner, on a clock, with escalation; never by the tool.
- Can it answer a question across all our agreements?
With each agreement cited, in every language.
- How are model or prompt changes tested and approved?
Against a fixed test set, with a named approver.
- What stays in our systems?
Document repository, CRM, billing and the compliance engine keep their roles.
Questions people ask
What are contract obligations?
The promises a signed contract makes one party keep: restrictions, deliverables, payments, notices, consents and conditions. In an investment management agreement they include investment guideline limits, reporting deadlines, fee terms, broker rules, and legal terms such as the consent-to-assignment clause the Advisers Act requires.
How many obligations should a seeded test hold?
Enough to score each type separately: around fifty in one bundle gives five or more per type for most types, which is the least that makes a type-level score meaningful. Weight the types the way your book is weighted, and include the hard cases.
What recall is good enough for obligation tracking?
No regulator or independent study sets a figure for contract obligations. Set the line by consequence before the run: most firms will want every restriction, fee term and override either found or flagged for a person, and accept less on low-stakes deliverables.
Is obligation tracking the same as contract management?
No. Contract management systems store, draft, negotiate and sign contracts and track key dates; obligation tracking turns each signed clause into work for an owner and proves it was taken on. Many firms need both, joined at signature.
Can obligation tracking software read side letters and amendments?
It should, and the test should prove it: plant overrides where the side letter changes the agreement and check that the tool returns the side letter's version. In the sample firm's bundle, the side letter changed three obligations the IMA set.
How long does a seeded test take?
Most of the time goes into the answer key: a careful reader needs a few days to key fifty obligations with pages, owners and the hard cases. The run itself is short; the sample firm's agents read, checked and routed a 112-page bundle in 12 minutes.