Evaluate · agents and their limits

AI Agents in Financial Services Operations: a Settlement Desk Test

An agent is only as good as the boundary drawn around it. On a loan settlement desk the useful division is that agents read, attribute and draft while people send, sign and approve — and the test of a real one is not what it writes, but whether every claim arrives with the message, notice or clause it rests on. This guide works that through on the six agents inside Loan Trade Settlement.

Updated · 14 min read

AI agents in financial services operations are pieces of software given a narrow job, a set of sources to read and a written limit on what they may decide — reading a mailbox and attributing each message to a transaction, say, rather than answering questions in general. The interesting question is never what they can produce; it is what they are allowed to conclude, what they must show for it, and which decisions stay with a named person. Botminds' Loan Trade Settlement is a concrete case to test that on: six agents on a loan settlement desk, each with a written limit, and three decisions that never leave the people who own them.

The desk in the examples is a fictional sample one inside the working software, with invented banks, funds, borrowers and agents. The governance expectations quoted are public, from the US Treasury, the National Institute of Standards and Technology, the Financial Stability Oversight Council and the banking agencies' joint third-party guidance.

What are AI agents in financial services operations, in plain terms

Strip away the category language and an agent in an operations setting is three things: a defined job, a defined set of things it may read, and a defined set of things it may not do. The job is narrow on purpose — one agent reads the mailbox, another works out what holds a trade — because a narrow job can be checked, and a broad one cannot.

The US Treasury's December 2024 report on artificial intelligence in financial services describes where firms said this was already working: respondents listed back-office work such as recordkeeping, transcribing audio and advanced document searches, and said these uses streamline processes and improve operational efficiency (Treasury, Artificial Intelligence in Financial Services). Note what that list has in common: every item is reading or producing a draft. None of it is deciding.

Loan settlement is an unusually good test of the idea, because the work is almost entirely reading and almost none of it is standardised. The Securities and Exchange Commission's description of the market — settlement that is not standardised, involves many parties and relies on manual processes — is a description of a job made of correspondence. Correspondence is what this kind of software is actually good at, and the decisions buried in it are exactly what it should not be making.

The worked example: six agents on a loan settlement desk

On the sample desk the work splits into six jobs that run in order, and three approvals that interrupt that order before anything leaves the building. Each agent hands its output to the next; none of them talks to a counterparty.

Six jobs in order, and the three approvals that interrupt them

  1. Agent

    Trade intake reader

    Turns confirmations, allocations and platform status into one entry per open trade, and flags an allocation that came back short.

    Never: edits price, par or counterparty

  2. Agent

    Mail and notice reader

    Reads the settlement mailbox and the agents’ notices, ties each message to its trade and says what it means.

    Never: sends or replies to anything

  3. Agent

    Blocker finder

    Names what holds each trade and which firm holds it, and reads the transfer section for a way through.

    Never: calls silence consent without the clause and the delivery date

  4. Agent

    Delayed comp calculator

    Works out what each late day costs or earns, both ways, and shows the arithmetic.

    Never: books an entry or pays anything

  5. Agent

    Chase writer

    Drafts one chase per blocked trade to the person who can clear it, with the proof attached.

    Never: sends — a person sends

  6. Agent

    Settlement reporter

    Writes the weekly and monthly account of median days, share inside target, causes of late days and delayed compensation both ways.

    Never: reports a figure that cannot be traced to a trade

  7. A person decides

    The analyst reads, edits and sends

    Decides which drafts go, to whom and when, and sets the settlement date.

  8. A person decides

    The desk head approves a borrower message

    Anything addressed to a borrower waits here, however routine it looks.

  9. A person decides

    The manager approves a claim

    Delayed-compensation claims above the desk’s threshold — $50,000 on the sample desk — are approved by the head of operations.

The six agents and the three approvals are the fictional sample desk’s arrangement in Loan Trade Settlement. The numbered jobs run in order; the three approvals sit inside that order — the analyst between drafting and sending, the desk head before anything reaches a borrower, the manager before a claim goes out. Where each approval sits is a setting the desk owns, not a fixed property of the software.

The order matters more than it looks. Attribution comes before drafting, which means no chase is written until something has said which firm is holding the trade and why. A tool that drafts first and attributes afterwards produces fluent messages to the wrong people, which is the failure mode the desk already had.

What each agent is allowed to decide, and the line it may not cross

A limit is only meaningful if it is specific enough to be broken. "Acts responsibly" is not a limit; "never edits price, par or counterparty" is, because anyone can check whether it held.

The limits on each of the six agents in Loan Trade Settlement

Trade intake reader

What it doesReads confirmations, allocations and the settlement platform’s status and keeps one entry per open trade — facility, side, counterparty, fund, par, price, trade date and the documents expected. Flags an allocation that comes back short.

What it never doesNever edits price, par or counterparty. A difference between the confirmation and the allocation is reported to the analyst, not quietly resolved.

What it must showEvery field names the document it came from.

Mail and notice reader

What it doesReads the shared settlement mailbox and the administrative agents’ notices, ties each message to its trade, and says what it means — a document request, a consent, an objection, a memo, an out-of-office, or nothing.

What it never doesNever sends and never replies. The mailbox is read only.

What it must showA message it cannot match to a trade goes to the analyst, unread by anything else.

Blocker finder

What it doesWorks out what holds each trade and which firm holds it, and reads the transfer section of the credit agreement for a way through — deemed consent, minimum amounts, approved funds, disqualified lenders — counting business days from the agent’s notice.

What it never doesNever treats silence as consent without both the clause and the date the request was delivered.

What it must showQuotes the clause, with its section, and the notice, with its sender and time.

Chase writer

What it doesDrafts one short chase per blocked trade, addressed to the person who can clear it, copying the agent where that helps, with the hold, the evidence and an ask carrying a date.

What it never doesNever sends — a person sends. Anything addressed to a borrower needs the desk head’s approval first.

What it must showAttaches the message, notice or clause the draft rests on.

Delayed comp calculator

What it doesFrom the seventh business day, accrues the interest on par at the loan’s rate and the cost of carry on the purchase price, nets them by side, and marks trades where the buyer missed its own requirements.

What it never doesNever books an entry and never pays. Nothing in the accounting moves because of it.

What it must showShows the rate, the days, the carry and the dates the figure runs between.

Settlement reporter

What it doesWrites the weekly and monthly settlement performance account — median days to settle, share inside the desk’s own target, causes of late days, and delayed compensation paid and received.

What it never doesNever reports a figure it cannot trace back to a trade.

What it must showEvery number ties to the trades it was made of.

The limits as the working software states them for the fictional sample desk. Each is written as a prohibition rather than a preference, so a reviewer can test whether it held.

Read down the middle column and a pattern appears: every prohibition is about an irreversible act or a judgement that carries a relationship. Editing a trade's economics is irreversible. Sending is irreversible. Booking or paying is irreversible. Calling a borrower's silence consent is a judgement that, if wrong, gives a counterparty a settlement date the agreement does not support.

Everything on the left of that line — reading, matching, attributing, calculating, drafting — is reversible, checkable and dull, which is exactly the work worth handing over.

What should an AI agent never be allowed to do on an operations desk?

Four things, and they generalise well beyond loan settlement.

It should never send. A draft is a proposal a person can reject; a sent message is a statement by the firm. On this desk nothing reaches a counterparty, an agent or a borrower without somebody pressing send.

It should never change the economics. Price, par, counterparty, rate: these come from executed documents, and a difference between two documents is a question for a person, not a reconciliation to be performed quietly.

It should never move money or make an entry. Working out that the desk owes $53,593 on a trade is analysis. Booking it is accounting, and it belongs to people with the authority to do it.

And it should never resolve an ambiguity by picking the likelier option. A message that cannot be matched to a trade goes to the analyst unmatched. This is the limit most often quietly broken, because a tool that passes ten per cent of its input to a human looks worse in a demonstration than one that guesses — right up until the guess sets a settlement date.

Four decisions on a settlement desk, and who holds each

The decisionWho decidesWhat the agents contributeWhy it sits there
Send a chase to a counterparty or agentThe settlement analystA draft naming the hold, the evidence and the ask with a date, with the proof attachedA sent message is the firm speaking. Drafting it is reading; sending it is representation.
Send anything to a borrowerThe loan trading desk headThe same draft, held until the approval is givenThe borrower is a client of the bank, not a party to the trade. The relationship owner decides what it hears.
Set or confirm a settlement dateThe analyst, or the closerThe clause, the delivery date, the business-day count and the date it producesA date given to a counterparty is a commitment, and the arithmetic behind it has to be owned by somebody.
Make a delayed-compensation claim above $50,000The head of loan settlement operationsThe rate, the days, the carry and the dates — the whole workingA claim is a demand on another firm that it will check line by line.
The threshold and the approvals are the fictional sample desk’s own settings. Every bank draws these lines for itself; what matters is that they are drawn somewhere specific rather than implied.

How do you keep a human in the loop without making them a rubber stamp

Oversight that cannot change the outcome is not oversight. The National Institute of Standards and Technology's AI Risk Management Framework asks for the arrangement to be written down rather than assumed: its MAP 3.5 outcome is that processes for human oversight are defined, assessed and documented under the organisation's own policies (NIST AI 100-1). The same framework, under GOVERN 3.2, asks organisations to be explicit about who does what — policies that define and separate the roles and responsibilities of people and AI systems, and of those who oversee them.

In practice three things separate real review from a rubber stamp. The reviewer must be able to see the evidence without leaving the screen, so that disagreeing costs seconds rather than a fresh investigation. The reviewer must have a genuine third option — not just approve or reject, but edit, which is what most drafts actually need. And the volume must be survivable: eight drafted chases in a morning get read, and eighty do not.

The two seats the division of labour is really about

Nina Castellano, loan settlement analyst

Today

Reads whole threads to find out which firm a trade is stuck on, then writes the chase.

With agents

Reads eight drafts, each with the message or clause that produced it, and decides which go, which get edited and which are wrong.

Still theirs: she presses send, every time

Dana Whitlock, loan trading desk head

Today

Hears about a silent borrower when somebody escalates, usually late.

With agents

Sees the trades a borrower holds, what they are costing, and the date each consent period ends — and approves each message before it goes.

Still theirs: nothing reaches a borrower without her

Fictional characters in the working software’s sample data. The point of the figure is the shape of the two seats, not the people.

How do you stop an agent asserting a fact it cannot prove?

You cannot stop a language model from being capable of it; you can refuse to accept any claim that does not arrive with its source. The Treasury report records the concern in the firms' own words: respondents "noted the concern about hallucinations – a risk unique to Generative AI models - in which a model convincingly produces an incorrect output". The dangerous word there is "convincingly". An incorrect output that looks uncertain gets checked. One that reads like the other forty gets acted on.

The design answer is to make the citation a condition of the claim rather than a decoration on it. On this desk a hold is not displayed unless it names the message, notice or clause behind it; a deemed-consent date is not calculated unless both the clause and the date the agent delivered the request exist; a delayed-compensation figure is not shown without its rate, days and carry. Where the evidence is missing, the output is the absence, handed to a person.

That is also the honest answer to "how accurate is it". Accuracy on a reading task is hard to state and easy to overstate. Verifiability is neither: either the sentence that supports a conclusion is on the screen, or it is not.

The evidence test: every conclusion with its message, notice or clause

The sample desk's clearest case is a $25 million sell that had reached twenty-three days past trade date with a borrower that had never replied to a consent request. The conclusion — that the trade could settle later that week — rested on three quoted pieces: the administrative agent's notice recording that the request had been delivered to a named officer on a stated date; the transfer clause itself, which deems consent ten business days after delivery unless the borrower objects; and that morning's report from the agent showing no objection. A ten-box clock showed the business days gone, with a public holiday not counted.

Each of those three is checkable by the analyst in a few seconds, and each would be visibly missing if it were not there. That is the test to apply to any demonstration: not "is the answer right", which you cannot tell, but "can I see the three things it is made of".

A hold, with the three quoted proofs it rests on

The screen explaining what is holding one trade: three quotations, each with its source — the administrative agent's notice that the consent request was delivered to a named officer on a stated date, the transfer clause of the credit agreement that deems consent ten business days after delivery unless the borrower objects, and that morning's agent report showing no objection recorded. A ten-box clock shows eight business days gone, with a public holiday not counted, beside the delayed compensation accrued so far.
Loan Trade Settlement on the fictional sample desk. The conclusion is ordinary; the three quotations under it are the thing worth buying.

The same discipline is what makes a drafted message safe to send. A chase that quotes the agent's own request back at the buyer is not a nudge — it is a document the recipient can act on without looking anything up, which is where most of the round-trip delay in settlement actually comes from.

What Loan Trade Settlement drafts: a notice carrying the sources it was written from

A drafted notice to the buyer, copying the administrative agent: it names the trade, the date the consent request was delivered, the clause and the date the consent period ends, asks the agent to confirm no objection and schedule the trade for that Friday's close list, and asks the buyer to have funds ready. The agent's notice and the clause are attached, and a line says the draft was written from those two sources and that anything may be edited before sending.
Loan Trade Settlement on the fictional sample desk. The draft names its own sources, so the person sending it can check the claim before putting the firm’s name on it.

What the supervisors and the standards bodies already expect

The public material quoted here does not treat this kind of software as a category apart, and nothing in it exempts the software from the controls the desk already has. Three expectations are worth knowing before a risk review.

First, oversight has to be designed and written down, not assumed — that is the NIST framework's MAP 3.5 and GOVERN 3.2 above, and it is why "the analyst reviews everything" is a weaker answer than "the analyst reviews eight drafts a morning, each with its evidence, and may edit any of them".

Second, buying does not transfer responsibility. The joint guidance from the Federal Reserve, the Federal Deposit Insurance Corporation and the Office of the Comptroller of the Currency is explicit that using a third party leaves the bank's own obligations exactly where they were, and that diligence before selection is part of sound risk management (Interagency Guidance on Third-Party Relationships, June 2023). A settlement control does not stop being the bank's control because a tool performs part of it.

Third, the supervisors are actively looking at this. The Financial Stability Oversight Council's 2025 annual report announced that "a new artificial intelligence (AI) working group will explore opportunities for AI to promote the resilience of the financial system while also monitoring for potential risks to financial stability that might be posed by the adoption of AI" (FSOC 2025 Annual Report). A deployment whose limits are written down and whose conclusions carry their evidence is a much easier thing to explain to a supervisor than one whose answer to "why did it say that" is a model version.

What this does not do, and the questions to ask before you believe a demonstration

Being clear about the edges is the fastest way to a serious conversation. This software does not execute, price or book a trade. It does not replace a settlement platform. It sends nothing. It does not decide what the desk pays or claims. And it does not read anything it has not been given — a trade whose settlement correspondence never reaches a mailbox it may read is invisible to it, which is a real limit on desks that negotiate by telephone.

Eight questions to ask of any agent before it touches an operations queue

  1. What exactly is this agent’s job, in one sentence?

    If the sentence needs an “and also”, it is two agents, and the limits on each are about to get vague.

  2. What may it read, and what is it blind to?

    Ask for the list of sources. Anything not on it is a gap in every conclusion the agent reaches.

  3. Show me a conclusion with its evidence.

    The sender, the time, the words; the clause, with its section. A paraphrase is where a wrong answer hides.

  4. What does it do when the evidence is missing?

    The right answer is that it says so and hands the item to a person. The wrong answer is a best guess with a confidence figure.

  5. What is it forbidden to do, in writing?

    Prohibitions should be specific enough to test: never edits price, never sends, never books. “Operates responsibly” is not testable.

  6. Which decisions still belong to a named person?

    Expect sending, signing, approving a customer-facing message and anything irreversible. If the list is empty, the review you are in is the only control.

  7. How many items a day will a reviewer actually see?

    Oversight that arrives in volumes nobody can read is oversight on paper. Eight drafts a morning is a review; eighty is a rubber stamp.

  8. What is kept about what it did?

    Each read, each draft, each decision by a person, with who and when — the thing you will be asked for when something goes wrong, and the thing nobody adds afterwards.

Our own list, written from what distinguished useful demonstrations on this desk from impressive ones. It is not a standard and nobody certifies against it.

If you want the same argument from the buyer's side rather than the designer's, how the four ways of running a settlement desk compare sets out which layer each approach covers. For what the agents are actually reading about, the eight blockers that hold loan trades is the underlying subject matter.

Questions people ask

What can an AI agent do in a back office?

Reading work, mostly, and drafting from what it has read. The US Treasury's 2024 report lists recordkeeping, transcription and advanced document searches among the back-office functions firms reported automating, and notes that respondents saw these as efficiency gains rather than decisions. On a settlement desk that means reading a mailbox and attributing each message to a trade, reading a credit agreement's transfer section, calculating what a delay costs, and drafting the chase — with a person sending it.

What evidence should an AI agent show for its conclusions?

The primary source, not a summary of it: the message with its sender and time, the agent notice with its date, the clause with its section number, the figure with its rate, days and arithmetic. The practical test is whether a reviewer can disagree without starting again. If checking a conclusion means re-reading the thread yourself, the evidence is decorative.

Are AI agents allowed to send emails to customers?

Whatever a bank’s own rules allow, on this desk it is forbidden. Nothing sends: the mailbox is read only, a person sends every chase and notice, and anything addressed to a borrower waits for the loan trading desk head. The reason is what a sent message is — a statement by the bank to a client, which is a relationship decision rather than an operational one.

How do you govern AI in a bank operations team?

Write down the division of labour and the oversight, then hold the usual controls in place. The NIST framework asks for human-oversight processes to be defined, assessed and documented, and for roles and responsibilities in human-AI arrangements to be differentiated explicitly. The banking agencies' third-party guidance adds that none of the bank's own responsibilities move because a third party performs part of the work. Those two together are most of a defensible position.

What is the difference between an agent and a workflow rule?

A rule fires on a condition somebody anticipated; an agent reads something unstructured and reaches a conclusion nobody spelled out in advance. That is exactly why the limits matter more for the second. A rule that mis-fires does a known wrong thing; an agent that mis-reads produces a confident, novel, wrong thing, which is why the useful ones are built so that every claim has to arrive with the text it came from.

Does this reduce the size of a settlement team?

Not on the evidence here, and claiming otherwise would overstate what the division of labour does. What changes is where the hours go: the reading and attribution that used to fill the morning arrive done, and the same people spend their day on the decisions — which chase goes, what the desk claims, what the bank says to a borrower. The sample desk's own measure of success is the median days to settle and the delayed compensation paid, not its headcount.

Loan Trade Settlement.
See it on your own files.

An agentic solution from Botminds. Every open loan trade on one timeline, with the step that is holding it up and the agent already chasing it. Book a demo, or explore how it works.