Model risk management for credit models, explained.

What model risk management means for the default and loss models a bank lends on

The life of a credit model, in five steps

1. Built and documentedA credit analytics team fits a default model on fifteen years of loan history and writes a development document: purpose, data, method, tests, limits.
2. Validated before useA validator checks the design, the data and the results, and rates it fit for use, with or without conditions.
3. Watched on new dataEvery month or quarter, its estimates are compared with what happened. This is where a drifting office segment is caught, or missed until next year.
4. Changed, overlaid or restrictedWhen it drifts, the bank recalibrates it, adds a governed adjustment, or limits where it may be used.
5. Revalidated, and eventually retiredThe cycle repeats on a schedule set by how much the model matters.

Step 3 and the validation around it are the focus of this guide. The bank, people and figures in the example are fictional.

Guidance nobody can cite against you, and no yearly validation rule

Model inventory, validation, materiality and finding, defined

Fourteen terms are defined in the one-page summary.

What counts as a model under SR 26-2, and what moves to other registers

In shortA model is now "a complex quantitative method" that applies statistical, economic or financial theory to turn data into estimates. Spreadsheet arithmetic and fixed rules are out, and so are generative AI and AI that acts on its own. Everything a bank listed under the 2011 definition needs a scope decision with a reason an auditor can test.

The 2026 text defines a model as "a complex quantitative method, system, or approach that applies statistical, economic, or financial theories to process input data into quantitative estimates". It then excludes "simple arithmetic calculations, such as those found within spreadsheets, as well as deterministic rule-based processes and software where there are no statistical, economic, or financial theories underpinning their design or use".1 Compared with 2011, the word "complex" is new and "mathematical" theories are gone, which takes many calculators and rule engines out of scope.6

A footnote then removes a whole class. Generative AI models, and AI models that act on their own, "are novel and rapidly evolving. As such, they are not within the scope of this guidance." The same footnote keeps "traditional statistical and quantitative models" and other machine learning inside it, so a gradient-boosted default model or a machine-learning fraud score is still a model.1

Sorting a lending bank's tools

The hard cases are spreadsheets with forecasting logic: a regression of loan growth pasted into a workbook is arithmetic on the surface and statistical theory underneath. A sound policy sends those to a person with the reasoning written down, because internal audit's job, under the new text, is "to evaluate whether the model risk management practices are rigorous and effective".1 An entry that silently disappears from the inventory is the first thing an auditor asks about.

What a re-scope can look like

Nothing should be deleted. An entry that leaves the model inventory still has an owner and still needs controls; it moves to a register of tools and calculations, attested by its owner, or to a register of AI tools governed by the bank's AI policy. The fictional bank's 86 tools are calculators, fee and budget workbooks, and fixed rules such as wire monitoring scenarios.

The model risk rules a US lender works under, and their status in October 2026

In shortModel risk guidance is supervisory guidance, now explicitly unenforceable on its own. The binding pieces sit elsewhere: Regulation B's revalidation test for statistically sound credit scoring systems, the allowance policy statement, fair lending law, and the safety and soundness standard that a new 2026 rule defines. Several of these changed between 2025 and 2027.

The OCC's own pages refuse automated reading; its bulletin's list of rescissions is taken from Orrick's summary, a law firm's secondary reading of it.6

SR 26-2 against SR 11-7: what changed, which banks it aims at, and what stayed

In shortThe 2026 guidance is shorter, explicitly non-binding, aimed at banks above $30 billion, and built on materiality. It drops the detailed duties for boards and internal audit, judges a validation by how rigorous it is, wherever the validators report, and gives vendor models their own section. The core work, validating conceptual soundness, outcomes and ongoing performance, is unchanged.

Who the $30 billion line reaches

The guidance speaks of "banking organizations", so a holding company's total matters, but the bank-level count shows the shape: about 66 banks carry most of the industry's balance sheet and most of its model risk.18

Smaller banks are told their models "typically are subject to internal risk management and governance practices appropriate for the size and risk profile", with an exception for banks whose models are complex or whose activities lie "outside the scope of traditional community banking".1 A $12 billion bank that buys a machine-learning credit card score or runs a fintech partnership program should read itself into that exception.

What did not change

Advisers read the change the same way: the Sia Partners comparison lists objective review, accountability and responsibility for third-party models as unchanged, and timing "may now better reflect materiality rather than default annual cycles".17

Building a model inventory and tiering credit models by exposure, purpose and inherent risk

In shortThe inventory should hold enough about each model to understand its risk alone and alongside others. Tiers come from materiality, which the guidance defines as exposure plus purpose, raised where the model's inherent risk is high. A re-tier usually lowers most models and raises a few, and the plan of validations follows.

The guidance asks for an inventory with "sufficient information to understand model risks, so as to support effective model risk management at the individual and aggregate levels".1 In practice that means, for each model: an ID, the owner and developer, purpose and uses, the portfolios and dollars it touches, inputs and upstream models, method, vendor if any, tier with its reason, validation history and next due date, open findings, monitoring measures and their limits, and every change since the last validation.

The aggregate view is the easiest to skip. Several credit models can lean on the same input, a house price index, an unemployment forecast or a bureau score, so one bad feed can move them all at once. The guidance calls this out: aggregate risk "reflects interactions and dependencies among models; reliance on common assumptions, data, or methodologies".1

Materiality, the two-part test

The text: "Model purpose, together with model exposure, determines model materiality." Models used "to help meet regulatory requirements or manage a banking organization's financial risk exposures are generally considered to be of greater risk". Inherent risk comes from "the assumptions made in developing the model, the model's complexity, the quality of inputs for the model, and data constraints".1

What a re-tier does to the plan

Tier rules have to be written as rules. The example bank's policy puts a model in Tier 1 at $1 billion of exposure or more, or when it is used for financial reporting, capital or regulatory reporting; Tier 2 from $100 million to that line or for risk management; Tier 3 below; and lets high inherent risk raise a tier. With those rules every tier decision states its exposure and purpose, which is what lets the next validator, auditor or examiner check it in minutes.

How validation cadence looks across banks

No public source counts the models at US banks. Deloitte's 2025 survey of 87 banks in Europe, the Middle East and South Africa found averages of 58 models at small banks, 132 at medium and 365 at large ones, with six large banks above 1,000 left out as outliers.19 Even the Federal Reserve's own inventories fell short: its inspector general found in 2022 that two of the Board's model inventories did not align with SR 11-7.20

How to validate a credit model: conceptual soundness, outcomes analysis and a worked back-test

In shortValidation asks whether the design makes sense for the use, whether the estimates match what happened, and whether the model still fits today's loans. For a default model, outcomes analysis means comparing predicted with observed defaults segment by segment, measuring how well the model ranks good loans above bad (AUC), and checking whether today's borrowers look like the ones it was built on (PSI).

The three components, in the guidance's words

A worked back-test: four quarters to June 30, 2026

Take the example's default model. For each segment, multiply loans by the predicted default rate to get the defaults you should have seen, then ask how surprising the actual count is if the model were right. The binomial distribution answers that exactly.

A probability of 0.8% says the model is very likely wrong for office. Two cautions apply. First, the p-value is one-sided: it asks only whether defaults ran too high, which is the direction that hurts a lender, and a test at 1% rejects. A two-sided 99% range, which puts 0.5% in each tail, runs from 2 to 16 defaults, so office would sit exactly on its edge. A validation report should state which test it used.

Second, across many segments and many models, a few will look surprising by chance, so a single bad p-value needs a reason: here the reason is plain, since office occupancy and values fell after the development data ended.

Ranking power and population shift

The AUC (area under the ROC curve) is the chance that the model gives a loan that later defaulted a higher risk score than one that did not: 0.5 is a coin toss, 1.0 is perfect. The example model's AUC fell from 0.83 at development to 0.79 overall, and from 0.80 to 0.68 for office. The population stability index (PSI) compares today's spread of an input across bands with the development sample.

Treat the PSI cut-offs with care. The widely used reading, below 0.10 little shift and above 0.25 significant shift, is a rule of thumb; a 2018 study of its statistical properties found these "benchmarks are being used without reference to Type I or Type II error rates" and set out to give the index's statistical properties and recommendations for those rules of thumb.21 Regulation B's commentary names the same two checks for keeping a statistically sound credit scoring system current: delinquency by score band and "population stability over time".5

The finding, and what it costs

A good validation report ends with a rating and conditions. In the example: fit for use with conditions, meaning a governed office overlay, no construction loans rated with the model until that use is validated, and the office segment recalibrated by a set date. The guidance's section on use applies to the last point: "Using a model beyond its intended purpose introduces additional uncertainty and risk", and extending it calls for "additional analysis of the new usage and its limitations".1

Catching model drift between validations with control charts on production data

In shortA control chart plots a model's monthly measure against a center line and limits set from a stable period, with rules that flag a breach or a run heading toward one. It turns monitoring from a quarterly report someone reads into an event someone acts on. The limits must come from the measure's real noise, and the rules have a false-alarm price you can calculate.

Read the chart the way a quality engineer would. September 2026's point, 0.788, is beyond the lower limit, the classic signal. But the run below center began in September 2025, and from December 2025 every month was lower than the one before. Two standard run rules, eight points in a row on one side of center and six points in a row moving one way, had fired months earlier.22

The timing matters. A bank that acts on those rules opens an investigation in spring; one that waits for the limit acts in October; one that waits for the annual validation acts in November.

Where the limits come from matters

Limits are usually the center plus or minus three standard deviations (σ) of the measure in a stable period. For a rolling twelve-month AUC, consecutive points share eleven of their twelve months, so their scatter badly understates the true noise: the example's first twelve points have a standard deviation of 0.0035, which would put the lower limit at 0.810 and flag the model in January 2026.

Estimate σ from the measure's own sampling error instead, for example by re-sampling the loans (a bootstrap), and write the method into the monitoring plan. The NIST handbook adds a second warning: change limits only for "a valid, compelling reason", or the chart stops comparing the present with the past.22

The price of sensitivity: false alarms

Twenty-four false signals a year across a large monitor is manageable if each comes with its evidence and a person triages it in minutes. It is unmanageable if each needs a meeting. That is why the run rules are usually set to raise a watch status, and only a limit breach on a material model opens a revalidation task.

Why credit models are drifting now

Securitized loans are a different book from a bank's balance sheet, but the direction is the warning: a default model fitted on 2009–2024 office loans learned an office market that no longer exists. Trepp's overall rate was 8.02% in September 2026, its highest since 2020.23 Segment-level monitoring is the only way to see that in time, and it needs the data at the segment level every month.

Vendor models, model changes and overlays: the three places credit models go wrong quietly

In shortA bought model is still the bank's model, and most vendors do not hand over what a validator needs. Changes to data, inputs and assumptions need the same control as changes to code. Overlays are legitimate when they are governed: sized from evidence, approved, monitored and removed when the model is fixed.

Vendor models

The guidance gives vendor products their own section. Banks "may not receive from the vendor the underlying code, data, or methodology", yet "the principles of model risk management remain applicable": the bank should understand the vendor model's "conceptual soundness, design, development data, and performance" and run "ongoing monitoring and outcome analysis to assess whether vendor models are accurate, remain fit for purpose, and continue to be reliable". Customizations should be documented, justified and evaluated as part of validation.1

What a validator can do without the code: benchmark the vendor score against the bank's own outcomes, test its stability on the bank's population, run sensitivity tests on the inputs the bank controls, and read the vendor's development summary critically. Regulation B's commentary sets the same bar for a statistically sound scoring system built on pooled data: it qualifies only if the creditor validates and revalidates it "based on the creditor's own data when it becomes available".5

A change in a vendor score is a model change for every model that uses it. A mortgage lender that switches scores, or receives both, must check cut-offs, pricing grids, scorecards with a bureau score as an input, and the allowance model, against its own outcomes before relying on them.

Model changes

The quiet changes are the dangerous ones: a data feed replaced when a contract ends, a field redefined upstream, an assumption updated by the owner. Every assumption change on a material model deserves a validator's eye and a line on the inventory record.

The Federal Reserve's review of Silicon Valley Bank describes an extreme case. In April 2022 the bank made "a poorly supported change in assumption to increase the duration of its deposits", which "gave the appearance of reduced IRR; however, no risk had been taken off the balance sheet". Internal audit had findings on "inadequate governance of IRR models" going back to December 2020, and supervisors issued a matter requiring attention on interest rate risk modeling in November 2022, months before the bank failed.25

Overlays

An overlay is an adjustment applied to a model's output, or a qualitative adjustment applied beside it, because the model is known to be off. The allowance policy statement allows a qualitative factor "to capture the variance between modeled credit loss expectations and actual historical losses when the model is still considered predictive and fit for use", and says the bank "should monitor this variance".14 The 2026 guidance lists overlays among the responses to a model "that no longer performs as expected".1

How SR 26-2 treats generative AI and AI that acts on its own, and how a bank governs what it leaves out

In shortGenerative AI, and AI that takes actions by itself, sit outside the 2026 guidance by its footnote 3, while machine-learning credit models stay inside. The bank's own risk management must decide the controls. A practical answer is a separate register: what each tool does, whether it drafts, informs or acts, who checks its output, what it may touch, and when it is reviewed.

The agencies kept the carve-out short: generative models, and models that act on their own, "are novel and rapidly evolving. As such, they are not within the scope of this guidance. Nonetheless, a banking organization's risk management and governance practices should guide the determination of appropriate governance and controls for any tools, processes, or systems not covered in this document."1 They said they would follow with a request for information on model risk and AI.6 As of October 30, 2026 none had been published.

The Treasury Department's 2024 report on AI in financial services put the main failure mode plainly: generative AI "can be susceptible to 'AI hallucinations,' producing confidently stated but incorrect output".26 A tool that takes actions, such as opening a file or writing to a system, adds a second question: what can it change without a person approving?

What an AI tool register records

State law reaches some of these tools directly. Colorado's SB26-189, signed May 14, 2026, covers automated decision-making in "financial or lending services": notice at the point of interaction, a plain-language description of the technology's role within 30 days of an adverse outcome, and "meaningful human review and reconsideration" on request, with most duties from January 1, 2027.11

Fair lending testing of credit models after the 2026 Regulation B rule

In shortSince July 21, 2026, the CFPB reads the Equal Credit Opportunity Act as banning intentional discrimination only, and federal bank examiners have stopped testing for disparate impact. Disparate treatment, including proxies, is still illegal; the Fair Housing Act still recognizes disparate impact for housing credit; and several states have moved the other way. A credit model still needs fairness testing, scoped to the laws that reach it.

Fair lending law separates two kinds of discrimination. Disparate treatment is treating an applicant differently because of race, sex, age or another protected trait. Disparate impact is a neutral-looking practice whose results fall harder on a protected group. The 2026 changes concern the second.

The CFPB's final rule, published April 22, 2026 and effective July 21, 2026, amends Regulation B so that ECOA does not support disparate-impact claims. It keeps the ban on disparate treatment and warns that "facially neutral policies may still violate the law if they are proxies or pretexts for discrimination on a prohibited basis". Its own cost analysis expects lenders to save on "creating, testing, validating, and auditing models for potential disparate-impact risks", benefits "bounded by the ongoing need to comply with other State and Federal fair lending laws".5

The CFPB's rule is in force and contested. National Fair Housing Alliance v. CFPB, filed May 27, 2026 by two advocacy groups and two fair lending analytics firms, asks the court to set it aside; no preliminary injunction was sought, and briefing runs to January 12, 2027.27

Bank examiners had moved first. The FDIC removed disparate impact from its compliance examination manual on August 29, 2025, and now evaluates discrimination "only through evidence of disparate treatment"; the OCC made a similar change in July 2025.9,28

The Supreme Court held in 2015 that disparate-impact claims are cognizable under the Fair Housing Act, a point the CFPB's rule itself notes, so mortgage and home equity models still face that test.5 The Department of Housing and Urban Development (HUD) has proposed removing its own disparate-impact regulation and leaving the standard to the courts (January 2026, with a supplemental proposal on August 10, 2026); as of October 2026 it was not final.30

The states are moving the other way. Illinois amended its Human Rights Act to reach lending criteria with discriminatory effects that are not necessary to a substantial, legitimate, nondiscriminatory interest, or whose interest "another practice having a less discriminatory effect" could serve, effective January 1, 2027.12 New Jersey's Division on Civil Rights adopted disparate-impact rules under its Law Against Discrimination, effective December 15, 2025, covering lending among other fields; commentators discuss what they mean for credit scores, underwriting models and AI tools.29,31

Massachusetts settled with a student lender for $2.5 million in July 2025 over an AI underwriting model, alleging that a "Cohort Default Rate" variable produced a disparate impact.32,33

A fairness testing routine that holds up in 2026

1. Variable review: every input checked for a prohibited basis or a close proxy for one, with the reason it is predictive. This is disparate-treatment work and every lender needs it.
2. Outcome comparison: approval rates, prices and score distributions by group, using proxies where the bank does not collect the data, for housing credit and in states that require it.
3. Alternative search: when a gap appears, look for a variant with similar accuracy and a smaller gap, and record what was tried.
4. Adverse action reasons: Regulation B still requires specific reasons for a denial; a complex model must be able to produce them.
5. Ownership: fair lending sits with compliance, accuracy with model risk; the validation report should say which tests were run and who signed them.

What examiners and enforcers have cited when models went wrong

In shortPublic enforcement rarely names "model risk" alone. It shows up inside larger failures: weak validation oversight at JPMorgan in 2013, model assumption changes that hid interest-rate risk at Silicon Valley Bank, a fair lending case built on one model variable in 2025. From November 2026, OCC and FDIC matters requiring attention must rest on material financial risk or a violation of law.

The 2026 rule on matters requiring attention

An MRA is the formal finding an examiner writes when a bank must fix something. The OCC and FDIC's final rule, effective November 2, 2026, defines an unsafe or unsound practice by its likely material harm to the bank's financial condition or the deposit insurance fund, and says examiners should "prioritize material financial risks over concerns related to policies, process, documentation, and other nonfinancial risks".10

For model risk this cuts both ways. A missing signature on a Tier 3 model's documentation is less likely to become an MRA. A Tier 1 allowance model drifting with nobody acting is exactly the kind of practice the rule still reaches, and SR 26-2's footnote keeps "unsafe or unsound practices stemming from insufficient management of model risk" open to supervisory action.1

How model risk teams run validation today: who does what, and where the weeks go

In shortModel owners build and monitor, validators challenge and write, the head of model risk decides, and a committee chaired by the chief risk officer approves what only it can. Most of a validator's time goes to gathering evidence and writing the report, and most of the delay sits between a problem appearing and someone reading the report that shows it.

Model owner, validator, head of model risk, committee and audit

One credit model validation, from evidence to committee approval

Validator days on one annual validation, by step

Capacity is the binding constraint. In the ProSight 2026 survey, cost (69%), talent (55%) and resources (55%) were the top barriers to expanding validation; an earlier survey of 53 institutions found a median of two model developers for every validator.7,36 The example bank shows the arithmetic: a plan written on 2011-era cycles called for 214 validations and 34,600 validator hours in a year, against 22,400 hours from 14 validators.

In-house validators, consultants or model risk software

From a breached control limit to a signed validation in one morning

In shortThis is the example from the top of the page, inside the product. September's defaults push the commercial real estate default model below its lower control limit, and a revalidation task opens with its evidence. Agents read the 61 documents behind the model and draft the annual validation. Elena Marsh, head of model risk management, signs as approver; Owen Adeyemi, the lead validator, co-signs.

1

The model crosses its lower limit, and revalidation opens itself

The 6:00 a.m. run had scored 183 models on September production data. CRE-PD-v4 was waiting for its defaults file, which landed at 8:52 a.m. Elena scores it. The new point, an AUC of 0.788, falls below the lower control limit of 0.795, and the loan-to-value stability measure reaches 0.27 against a 0.25 limit.

At 9:16 a.m. revalidation task RV-2026-031 opens with its trigger, its scope (office recalibration and construction-loan use), Owen as validator and four pieces of evidence. The model owner, Rachel Tolland, is told. The monitor now reads one model out of control, one trending, 182 in control.

2

The drift point opens its evidence

Elena opens September's point. By segment, office has fallen to an AUC of 0.68 from 0.80 at development, while multifamily, retail, industrial and hospitality hold within two hundredths of their development figures. Office loans defaulted at twice the predicted rate: 16 against 8.1 expected.

The office back-test, the third-quarter monitoring report, the challenger results and change log CL-44 are one click away, each the source of a sentence on the panel.

3

The whole inventory, re-scoped to SR 26-2

The default model is one item on the committee's agenda. The other is the inventory built under the 2011 guidance. Elena starts the re-scope, and four agents test all 412 entries against the 2026 definition in 4 minutes 10 seconds: 298 stay models, 86 move to the tools and calculations register, 28 to the AI tool register.

Fourteen borderline spreadsheets come to her to decide. Re-tiered by materiality, the twelve-month plan falls from 214 validations to 131, and from 34,600 validator hours to 22,320, inside the team's 22,400.

4

The annual validation, drafted from 61 documents

Owen's validation starts from the evidence. In 2 minutes 59 seconds the agents read the 146-page development document, four monitoring reports, the change log, the 2025 validation and the user guide; ran 23 tests; and drafted six report sections with 31 citations and four findings: one high, two medium, one low.

The proposed rating is "fit for use with conditions". The severity of each finding is the validator's to change, and each finding names its owner and due date.

5

Two signatures, with the meaning recorded

Elena approves the 2026 validation as fit for use with conditions: a governed office overlay of +181 basis points, no construction loans rated with the model until that use is validated, and the office segment recalibrated by January 18, 2027. Her signature records its meaning, "Approved as Head of Model Risk Management".

Owen co-signs as lead validator, and the report joins the pack for the model risk committee on Tuesday, October 20, at 2:00 p.m.

The solution's design target is drift caught in the month it starts, every model charted from production data, and every finding cited to the document page or test run that proves it. These are targets, to be measured on your own models.

What the rest of the model risk team sees: the monitor wall, the 412-entry inventory, findings and the AI tool register

In shortOne model is the story. These screens show the rest of the work: every watched model on its own chart, the inventory with its scope decisions, the tier migration and validation plan, a model's tier reasons and uses, the report sections, the back-test, the findings, the AI tool register, the committee and the policy.

Select any screen to open it larger. The bank, people, models, vendors and figures are fictional sample data.

Which validation steps the agents take, and the scope, severity and sign-off decisions that stay with people

In shortEach agent has one job and limits it does not cross. None signs a validation, closes a finding, sets a threshold or approves an AI tool. The head of model risk approves scope, tiers and validations; a validator co-signs and closes findings; the committee approves conditional use and extensions.

Seven agents, one job each

Who decides scope, severity and sign-off

Nothing leaves the inventory without a reason internal audit can open. A signed report cannot be edited; a new finding is raised instead. The solution's own agents sit in the AI tool register like any other agentic tool, with the head of model risk as owner.

The model risk team in the screens

Connecting monthly default data, model documents and your inventory to control charts and signed validations

In shortIt reads what a model risk team already keeps: the inventory, monthly scores and outcomes, development and vendor documents, prior validations and findings, and AI use cases. It leaves behind a chart for every model, revalidation tasks with evidence, a tiered inventory with reasons, drafted and signed validations, a findings tracker and an AI tool register. Your systems of record stay where they are.

Systems that feed model monitoring and validation

For each, check whether it has an API, a file feed or only screens, and who owns it.

Fits the inventory, data and report template you already keep

What every signed validation carries

The price of a drift found a year late: a misstated reserve, overdue findings and a validation plan nobody can staff

In shortThe costs that matter are your own: months between a model going wrong and someone acting, validator hours your plan needs and your team does not have, findings past due, and reserve or pricing errors from a model off in one segment. Published figures frame them but do not replace them.

Validator hours and overdue findings: counting yours in a week

Hidden failing segments, MRAs and departed validators

Validator hours released when each validation starts from a cited draft, and the volume where it pays

In shortPut in your own numbers. The calculator values validator time released by drafting and evidence gathering done for the validator, against what the tool and your testing cost. Earlier detection and fewer overdue findings are real but harder to price, so they are listed as levers.

Where validator hours and earlier detection come from

Price the validator hours on your own volumes

A bank with more than a hundred validations a year clears the middle line comfortably. A bank with twenty models that validates a handful a year will not recover the cost through drafting alone, and should weigh monitoring and the AI register on their own.

A readiness score for automating model risk work, from the re-scoped inventory to the AI tool register

In shortPick the answer closest to your team today. Eight questions score 0 to 2; the last is for reflection. The meter adds up your score as you go, and each question says why it matters and what a strong answer sounds like.

What each readiness band means for a model risk team

Testing a model risk software vendor on scope decisions, control limits, cited findings and who signs

In shortAsk every vendor the same questions, us included. The answers show how the tool decides what a model is, how it watches models, what a drafted validation rests on, who signs, and how you would measure it on your own models.

We found no published independent test of any model risk software's drafting accuracy, as of October 2026. Redrafting a validation you already signed is the fairest comparison, because you know the answer.

Redraft a validation you already signed: a twelve-week pilot on five to ten of your credit models

In shortPick a few material credit models, chart them monthly, redraft a validation you already signed, and decide with four numbers. None of this commits you to a vendor, and the first two steps are worth doing anyway.

1. Measure your baseline: validator hours by step for your last ten validations, and the months from first signal to action for your last five model issues.
2. Choose five to ten models, Tier 1 and Tier 2 credit models with monthly outcome data by segment.
3. Set limits and rules per model from a stable year, with σ from sampling error, and agree what a breach opens.
4. Redraft one past validation blind from its evidence, and compare the findings with the ones your validator signed.
5. Decide with four numbers: findings matched and missed, false alarms per model-month, validator hours, and months to detection.

The numbers the pilot is judged on

SR 26-2, model validation, PSI thresholds and vendor models: the questions model risk teams search

In shortThe questions model risk teams, credit officers and buyers search for most, each answered on its own.