Failed payments, on a bank's payments desk, are the outgoing wires and transfers that did not pass validation, were rejected by a network or came back as returns; each one lands in a queue where a person has to work out what went wrong and what the client meant. The fastest honest way to judge a tool for that queue is not a demonstration but a replay of last month's failed payments, because the bank already knows how each one was resolved. Botminds' Payment Repair, an agentic solution in which agents diagnose each failed payment and propose a fix with its evidence for people to approve, is built to be piloted that way: on the bank's own month, offline, with the desk's repairs as the answer key.
The examples on this page are from the solution's sample data on a fictional Corvane National Bank. The rules on keeping records and the BIS's measures are real and linked where they appear; the pilot design and its pass marks are our recommendation.
Why last month is the right test bed
28,904
failed payments in 16 business days on the sample bank: enough for every cause family
5years
that BSA records must be kept, so the month's messages and funds-transfer records are on file
0
live payments touched in a replay: it runs on copies, offline
2weeks
enough for a replay, a review of disagreements and a seeded-error test
Why do payments fail?
Payments fail mostly because of data: a field the network requires is missing, unstructured or wrong, or a reference such as a bank identifier no longer matches the directory. Insufficient funds and limits account for some, and sanctions screening stops others, but the repair queue is dominated by data a person has to fix.
The BIS describes the old pattern in its brief on the ISO 20022 migration: under the MT formats, "insufficient or unstructured data in some fields hindered straight through processing (STP), delaying payments and increasing costs" (CPMI Brief No 11, April 2026). The new formats give each item its own field, but they do not oblige a client to fill it. The Federal Reserve's own guidance for Fedwire's ISO 20022 messages says "Account information is optional, but you are encouraged to include account information to help facilitate straight through processing" (Fedwire ISO 20022 format FAQs). Optional fields that downstream banks need are where many of tomorrow's failures will come from.
On the sample bank's October, the largest family was party name and address format, then invalid account numbers and bank identifiers, then missing purpose or regulatory information; the full split, and how to weight a test by it, is in the seeded-error test for payment validation. A pilot will show a bank its own split, which is worth having even if it buys nothing.
Why replay failed payments instead of piloting live?
A replay answers the question a live pilot cannot answer quickly: was the tool right? On live payments, a bank learns whether a proposed fix was right only when the payment settles, comes back or prompts a call. On last month's payments, it already knows. Each failure has a resolution on file: the repair the desk made, the payment it returned to the client, or the hold the sanctions team decided.
A replay also carries no risk to clients. It runs on copies of messages, client history and directory data, with no connection to the payments hub's release path, so nothing it proposes can be sent. And it can be repeated: the same month can be run again after a configuration change, which makes it the natural baseline for every later version of the tool.
The answer key has one flaw a bank should name at the start: the desk's own repairs are not all correct. Where the tool and the desk disagree, the tool may be wrong, the desk may have been wrong, or both may be defensible. That is why a good pilot reviews every disagreement with an experienced repairer rather than scoring it automatically, and why planted faults, whose right answers are certain, sit beside the replay.
How do you pilot payment automation? The steps and their pass marks
Run it as a measurement, not a demonstration: fix the data and the rules before the run, decide the pass marks before seeing results, and keep the answer key away from whoever configures the tool.
A two-week pilot on last month's failed payments, with pass marks
- Pick the month and freeze it
Every failed outgoing payment, with client history, templates and directory data as they stood that month. A directory that already knows the answer flatters the tool.
- Write the pass marks down first
Wrong changes, catch rate, refusals and time, by repair type, signed off by operations and operational risk before the run.
- Replay with no connection to release
The tool reads copies and writes proposals; nothing can reach the payments hub.
- Score agreement with the desk
Same fix, different fix, tool refused, desk returned. Every disagreement on who is paid or how much is reviewed by a senior repairer.
- Count wrong changes separately
A proposal that would have paid the wrong account or amount is a failure however good the rest is. Pass mark: none, in a sample large enough to mean it.
- Check every proposal cites its evidence
Pass mark: all of them. A fix without a source cannot be checked by a second person.
- Plant known faults in a copy of the month
Transposed digits, merged banks, free-text addresses, two-payee mismatches; count caught, missed and wrongly changed.
- Time a sample by hand and with the tool
Minutes from opening to decision, measured on the same repairs, by the people who do them.
- Report by cause, client and lane
The rate of failures by reason tells the bank where to fix the source, whatever it decides about the tool.
The last step is the one most pilots skip, and it is the one the BIS would recognise. Among the measures it suggests for payment systems after an ISO 20022 migration is the "Reject/exception rate by reason", which "Identifies and categorises the reasons for message rejections or exceptions, helping to pinpoint systemic or participant issues" (CPMI Brief No 11, Table A.1). A bank that leaves a pilot knowing its own rate by reason has a list of clients, channels and reference tables to fix.
What data is needed for a payment repair pilot?
Four sets: the failed messages themselves, the client's payment history and templates, the reference data the desk used, and the desk's resolutions. Most of it already exists in systems a bank keeps for other reasons.
What a replay needs, and where it already lives
| Data | Why the replay needs it | Where a bank usually has it |
|---|---|---|
| Failed outgoing messages for the month | The input: what failed, and why the hub said it failed | The payments hub's logs and message archive |
| The client's earlier payments to each beneficiary | The evidence for most who-or-how-much fixes, and whether those payments were credited | Funds transfer records, which must be retrievable by the originator's name and, for established customers, by account number |
| Client templates and their change logs | Where repeat failures come from | The treasury management portal |
| Bank directory and settlement instructions, as of that month | To confirm banks, successors and routes without hindsight | The directory provider's dated files |
| The desk's resolution of each failure | The answer key: repaired value, return, or hold | The repair tool or case system, and the released messages |
| Network rules in force that month | To re-validate repaired messages as the network would have | The network's published usage guidelines for that release |
Mask what the replay does not need. Account numbers outside the payment being repaired can be shown to the last four digits, as Payment Repair does on its own screens, and the client's history can be limited to payments to the same beneficiary. A pilot that needs a bank's whole customer file is a pilot that will not get approved.
How long should a pilot run?
Two weeks is enough for a month of the sample bank's size, and the time goes mostly on people, not on processing. The replay itself runs in hours; reviewing disagreements and planted faults with experienced repairers takes the days.
Two weeks, most of them spent by people reviewing results
- Extract and freeze2 d
- Replay1 d
- Review disagreements4 d
- Score faults2 d
- Decide1 d
Longer pilots tend to measure the tool's patience rather than its accuracy. If the first two weeks do not settle the wrong-change question, the problem is usually the size of the planted sample, not the length of the run.
What pass rate is acceptable in a pilot?
There is no single pass rate, because the repairs are not equal. For changes to who is paid or how much, the acceptable number of wrong proposals that a checker would have had to catch is as close to zero as the sample can show; for format repairs, a modest catch rate is still a saving, because a missed format fault simply goes to a person as it does today.
- Wrong changes to the beneficiary, account, bank or amount: none observed, in at least a few hundred cases of those types. With 300 cases and no wrong change, the true rate is below 1% at 95% confidence.
- Refusals: every planted case with no provable fix should be refused or returned. A tool that fixes those at high confidence fails, whatever else it does.
- Evidence: every proposal cites what proves it.
- Catch rate by type: compared with the desk's own resolution, type by type, without a single blended score.
- Time: measured on the same repairs by the people who do them, by hand and with the tool.
A pilot that meets the first three and shows a real time saving on the fourth and fifth is a reason to move to the next stage. One that misses the first is not, however impressive the saving.
Can a pilot run without touching live payments?
Yes, and the first stage should. A replay needs read-only copies and no route to the payments hub, so the bank's release controls are untouched. The second stage, if the replay passes, is a shadow run on the live queue: the tool proposes, the desk works as normal, and the two are compared daily for a few weeks. Only then does a bank let proposals into the desk's workflow, starting with the format and route repairs that take one approval, and adding who-or-how-much repairs behind a checker's blind re-key once the shadow results support it.
The order matters for an examiner as much as for the bank. The FFIEC expects a bank to have procedures to verify all processed payment orders, and calls verification and security procedures "extremely important in light of the potential for fraud or errors" (FFIEC IT Examination Handbook, Funds Transfer Operations). A staged go-live that leaves the release controls untouched until the evidence is in is the easiest kind to defend.
What a Payment Repair pilot shows on each failed payment
In a replay of Payment Repair, each failed payment comes back with the diagnosis, the proposal or the candidates, the sources behind them and the agents' re-check of the repaired message. A reviewer comparing the tool's answer with the desk's can open the evidence in one click: for the sample bank's hero case, the client's six earlier payments to the same supplier, all credited to the account the agents proposed.
Every reason opens its source: the evidence a reviewer checks
Two settings make the replay a fair test. The confidence threshold below which the agents offer candidates instead of a proposal, 0.90 on the sample bank, should be set before the run and not tuned to the answer key. And the version of each agent should be recorded with each result, so that a second replay after a change can be compared with the first. Payment Repair's audit trail keeps both, as the sample bank's settings change from 0.92 to 0.90 shows.
The Payment Repair trail a pilot report can be built from
What a pilot will not show is just as useful to know. A replay cannot test the cutoff forecast against a real afternoon, because last month's afternoons have already happened; the shadow stage tests that. And it cannot show how clients respond to drafted return messages and template corrections, which is a question for the relationship team. For how the minutes a pilot measures turn into a cost per repair, see what each payment that is not straight-through costs.
Questions people ask
What is a failed payment in banking?
For a bank's operations team, it is a payment that did not complete as instructed: it failed validation before release, was rejected by the network or the receiving bank, or was returned after settlement. For a consumer, the same words usually mean a card or account payment that was declined, which is a different process.
What happens to a failed outgoing wire?
It is held, not lost. The bank either repairs it from evidence and releases it, returns it to the client with a request for correct details, or, if it was stopped by screening, passes it to the sanctions team. If the bank had the funds and does not execute, Article 4A rules on notice and interest apply.
Should the vendor or the bank run the pilot?
The bank should own it: the data, the pass marks and the answer key. The vendor configures the tool and explains its results, but a pilot whose scoring the vendor controls measures the vendor's confidence, not the tool's accuracy.
Can a pilot include instant payments?
It can replay their rejections and returns, but instant payments leave little or no time for repair before release, so their failures are mostly prevented or returned rather than repaired. A pilot is usually more useful on wires and cross-border payments, where a repair can still save the payment the same day.
What should happen after a successful pilot?
A shadow run on the live queue, then proposals in the desk's workflow for format repairs, then who-or-how-much repairs behind a checker's blind re-key. Each stage keeps the last stage's measures running, so a regression shows up before it costs anything.
To see what a replay of your own month would show, start from Payment Repair.