Remediation validation report: why two population builds must agree

A remediation validation report is the document in which a reviewer independent of the remediation team tests and certifies that a customer remediation was done right: the affected population was complete, each refund was calculated under the approved method, the money reached customers, and every change along the way was explained. The OCC wrote the requirement into its 2018 order against Wells Fargo, which called it a “Remediation Review Report”. The section that fails most often is the first one, the population, because one query against one system can only find the customers that system still holds. Botminds' Customer Remediation answers that structurally: two agents build the population by different methods without seeing each other's result, and nothing is paid until they agree on every account.

The largest remediation the US banking agencies ever ran shows what happens when the population is not settled first. In the Independent Foreclosure Review, the population of borrowers in scope kept growing for more than a year after the work began, and the Federal Reserve's inspector general could not say at the end that it was complete.

What is a remediation validation report?

It is an independent assessment of whether a remediation plan was carried out as designed, written after the work and before the bank tells its regulator the program is complete. The OCC's Wells Fargo order required that “an independent unit, third party, or Internal Audit shall prepare a report validating that the Bank’s implementation of the Remediation Plan is in compliance with the requirements of the Bank’s Remediation Program”, for every remediation the bank ran, and allowed the OCC to insist on an outside party (OCC Consent Order AA-EC-2018-15). The order was terminated on February 13, 2025 (OCC AA-ENF-2025-13), but its list of what a validation must cover remains the most complete one a US regulator has published.

Other orders use other names. The OCC's 2018 order against TCF National Bank required that “an independent review and assessment of compliance with the terms of the Restitution Plan shall be performed” within 60 days of completing restitution (OCC AA-EC-2018-38; terminated December 18, 2019). CFPB orders have instead asked for a compliance report from the bank itself, in one case with internal audit's validation of the redress plan behind it. Whatever it is called, the reader is the same: an examiner deciding whether the bank's “complete” can be believed.

Why the population is where remediations fail

Because every later step inherits it. A wrong rate can be corrected with a top-up check to the same people; a missed customer needs a second program, a second letter and, usually, a second finding. The Independent Foreclosure Review is the public record of the problem. The Fed's inspector general found that “All but one of the large-scale additions related to data reliability issues”, and concluded: “we are unable to confirm that all potential in-scope borrowers have been identified and have received a payment” (Board of Governors and CFPB Office of Inspector General, 2015).

Sampling did not catch it, because a sample tests the files in the population, not the files missing from it. The GAO found that the consultants' plans varied widely, “with 8 of the 14 reviews assuming an error rate of zero” (GAO-13-277). A plan that expects no errors draws the smallest sample the precision target allows, and still never sees the borrower the extract left out.

A sample can tell you whether the people on the list were paid correctly. Only a second, independent build of the list can tell you whether the list is right.

How do you validate a remediation population? Two builds, blind

Build it twice, by methods that fail in different ways, and make the two builders blind to each other's answer. The first method follows the symptom: the fee postings themselves, by fee code and date, net of reversals. It is fast, and it misses anything the code does not label: a fee booked under a different code after a system change, an account whose postings were never migrated. The second method follows the cause: replay every qualifying transaction against the fee schedule in force on its date, in every system that holds history, and flag any fee the schedule did not allow. It is slower and catches what the first misses, but it depends on the schedule tables being right.

When two methods with different blind spots agree on every account, the agreement is evidence. When they disagree, the disagreement is a list of accounts to explain, which is exactly what a validator needs. Blindness matters: a second builder who can see the first answer tends to stop when the count matches.

How do you reconcile two population counts?

Account by account, never count against count. Two builds can report the same total and still be wrong about hundreds of customers: 600 accounts only in A and 600 only in B produce matching counts and 1,200 errors. A reconciliation sorts every account into three groups, in both builds, in A only and in B only, and the second and third groups must be empty or explained before anyone signs.

Each explanation needs a cause and a document. “Closed before the conversion” is a cause; the conversion log entry that says closed accounts were not migrated is the document. An explanation without a source is an opinion, and a validation report built on opinions will not survive the examiner's first question. The decision on each group belongs to a person, with a reason code, because some differences are true exclusions: a fee already refunded, an account outside the product scope.

Who validates a remediation, and how independent must they be?

Someone who did not build, approve or pay the program: internal audit, an independent unit inside the bank, or an outside firm, in that order of frequency. The CFPB judges a bank's own review partly on its independence, asking “Who conducted the review and did they have a vested interest or bias in the outcome? Were scope limitations placed on the review? If so, why and what were they?” (CFPB Bulletin 2020-01).

ASIC's guide sets out when internal independence is not enough and an independent expert should provide assurance: among other cases, “when a number of assumptions are made that may materially affect the scope or amount of compensation” and “when a licensee has nobody sufficiently independent internally” (RG 277). The test translates directly to US practice. If the population rests on judgment calls, such as how to treat accounts with missing history, the validator must be someone whose own work does not depend on those calls.

What goes in a validation report?

At least an assessment of how the population was found, why it changed, how customers were located and paid, and what customers said afterwards. The Wells Fargo order's list of minimum contents is the best template available, and two items in it are routinely skipped: whether the bank actually reached the customers, and complaints from people who believe they were wrongly left out.

What do regulators expect in validation?

Evidence that would let them reach the same answer, and a reviewer who looked beyond the plan's own definition of who counts. The complaints item exists because a plan can be executed perfectly and still be wrong about its scope; customers who write in to say they were missed are the cheapest second population there is. The OCC kept the power to widen the program even after it had been run: “the OCC, prior to providing no supervisory objection for any Remediation Plan, may require the Bank to provide additional remediation to consumers and/or remediate other consumers”.

Examiners also distinguish attempts from results. The TCF order asked the reviewer to assess both “The effectiveness of procedures used to locate and contact Affected Consumers” and “The Bank’s success in contacting Affected Consumers”. A report that describes the address search but not how many customers it found answers only the first.

When is a remediation validated?

Twice, if the bank wants to avoid a second program: once on the population and method before payment, and once on execution after the last payment has a final state. The orders above describe the second, after-the-fact review. The first is cheaper and more useful. Correcting a population after payment means reopening the reserve, a second set of letters and a supplementary payment run; in the Independent Foreclosure Review, the paying agent sent about 96,000 checks for less than borrowers were owed and had to issue supplementary checks worth about $45 million (GAO-14-376).

Pre-payment validation does not replace the final review; it changes what the final review finds. When the population was rebuilt independently and reconciled before money moved, the closing validation can rebuild it once more and confirm the same number, then spend its effort on delivery. How internal audit tests a remediation covers that closing review.

Two population builds in Customer Remediation: the 1,184

Take a fictional sample bank that kept charging a $5.00 savings transfer fee for fifteen months after removing it from its fee schedule. In Botminds' Customer Remediation, population agent A read the fee postings on the new core: 61,230 accounts and 1,699,024 fees. Population agent B replayed every transfer out of savings against the fee schedule in force on its date, in the new core and the legacy archive: 62,414 accounts and 1,702,436 fees. No account was in A only. All 1,184 accounts in B only had closed between March 1 and November 7, 2025.

Two population builds in Customer Remediation, blind to each other, the difference listed account by account

The cause was a single sentence in the conversion records: accounts closed before cutover were not converted, and their history was kept read-only in the legacy archive, which build A never read. Build A's own run log said so. Without the second build, those 1,184 former customers, owed $17,060.00 in fees and $993.68 in interest, would have been the program's next finding.

The difference cited to its source: the core conversion log

The analyst proposed including all 1,184; the program manager approved as checker, with a reason code, a note and a confirmation that she had read both builds, the difference file and the conversion log. That approval, with both builds and the source, is the first section of the validation report, written before a dollar moved. Paying the 1,184 former customers is its own problem.

The population approved by a person, with the reason recorded

Questions people ask

What is the difference between remediation and validation?

Remediation is the program that finds and pays affected customers; validation is the independent check that it did so. The remediation team owns the population, the method and the payments, and the validator owns the opinion on whether they can be relied on. The two must be different people, or the opinion is worth little.

Can the team that ran the remediation validate it?

Not credibly. The OCC's model order named an independent unit, a third party or internal audit, and ASIC expects whoever provides oversight to be removed from the operation of the remediation. The remediation team can and should reconcile its own figures; that is preparation for validation, not validation.

How big a sample does validation of a remediation need?

A sample tests calculation and delivery, not completeness. For the IFR file reviews, regulators asked consultants to size samples for 3 percent precision and 95 percent confidence, and the GAO still found plans that assumed zero errors. The fictional sample program's closing validation re-performs 60 accounts in four strata and rebuilds the whole population separately.

What happens if validation finds customers who were missed?

They are added and paid, with the cause documented and the reserve trued up, and the validator re-tests the change. The earlier that happens the cheaper it is: before payment it is a population decision, after payment it is a supplementary run, new letters and, often, a disclosure to the regulator.

Does the CFPB require a validation report?

Not under one name. CFPB orders typically require a redress plan the Bureau does not object to, a report of what was paid, and sometimes internal audit's validation within a set period. Examiners outside an order apply the same questions through the rating system, so most banks prepare the report whether or not an order requires it.