Credit risk model validation is the independent work of establishing that a risk measurement is conceptually sound, that its outputs bear out against what actually happened, and that someone is watching it for drift. Applied to a commercial risk rating system the exercise runs into a definitional surprise: under the model risk guidance the agencies issued in April 2026, a judgemental rating scorecard usually is not a model at all. That changes who owns the review and what it is called; it changes almost nothing about what has to be shown. Botminds' Risk Rating Validation is built to produce that showing continuously rather than annually — every rated commercial loan re-scored nightly on the bank's own approved scorecard, each borrower compared with look-alikes graded by other lenders, and the disagreements collected where a validator, an internal auditor or an examiner can read them.
Examples come from a working demonstration on sample data: Kestrel Valley Bank, a fictional commercial lender with 2,305 rated loans. Regulatory material is quoted and linked where it appears; none of it is legal advice, and a bank's own model risk policy may draw its lines more conservatively.
The April 2026 guidance, in four figures
SR 26-2
issued 17 April 2026 by the Federal Reserve, OCC and FDIC — it supersedes and replaces SR 11-7
$30B
in total assets: the size above which the guidance is expected to be most relevant
0
risk rating scorecards it names — the word "model" is defined by method, not by use case
v4.2
the approved scorecard version in the sample bank's demonstration, with a v4.3 draft going to validation before use
What credit risk model validation means when the output is a grade
It means three activities, whatever the thing being validated is called. Assess whether the design is sound — do the criteria measure what they claim to measure, and are the judgements inside them defensible. Compare the outputs against real outcomes. And watch, between validations, for the day the thing stops working. The 2026 guidance names the first two in its own words: validating conceptual soundness "involves assessing and documenting model design (including key modeling choices, assumptions, qualitative judgments, and data selection), construction, and developmental testing", while "outcomes analysis compares model outputs to corresponding real-world outcomes to assess model performance relative to model objectives and business use".
Translate that to a nine-grade commercial scale and nothing exotic appears. Conceptual soundness asks whether coverage, leverage, management quality and industry outlook are the right factors, whether the weights reflect how the bank actually loses money, and whether the grade guides are specific enough that two raters reach the same answer. Outcomes analysis asks whether worse grades defaulted more. Ongoing monitoring asks whether a region has started grading differently from the rest of the bank. The difficulty is not conceptual — it is that two of those three are hard to evidence on a book with twenty defaults a year, and the third is usually not instrumented at all.
The April 2026 definition, and what it leaves out
The definition is two sentences, and the second does more work than the first.
For the purposes of this guidance, the term "model" refers to a complex quantitative method, system, or approach that applies statistical, economic, or financial theories to process input data into quantitative estimates. The term "model" in this guidance excludes simple arithmetic calculations, such as those found within spreadsheets, as well as deterministic rule-based processes and software where there are no statistical, economic, or financial theories underpinning their design or use.
Supervisory Guidance on Model Risk Management (SR 26-2), 17 April 2026
The exclusion names two things a rating system is often made of. A weighted scorecard that multiplies six factor scores by six fixed weights and reads the total off a grade guide is arithmetic followed by a deterministic rule. The weights were chosen by a credit committee, not estimated from data; no statistical theory underpins them. A spreadsheet that does the same thing is excluded by name. The covering letter confirms the guidance "supersedes and replaces SR letter 11-7", so a bank whose model inventory was built against the 2011 definition is working from a superseded document, and the inventory is worth re-reading rather than re-filing (SR 26-2 covering letter).
Two other scoping facts matter before anyone argues about a particular scorecard. The guidance "is expected to be most relevant to banking organizations with over $30 billion in total assets", with smaller organisations generally excluded as a matter of tailoring — though it adds that it may still be relevant to a smaller bank with significant model risk exposure. And generative and agentic AI are outside its scope entirely, which is a separate question taken up in the guide on agents near a grade. For the definition applied component by component to a different piece of credit software, the sibling series on commercial financial spreading works through whether spreading software is a model in detail.
Is your risk rating scorecard a model?
Probably not, and the answer depends on how the grade is produced rather than on what it is used for. Three designs are common, and they land in different places.
Three ways a bank produces a grade, and what each one owes
| How the grade is produced | Inside the 2026 definition? | Who reviews it | What the review has to show |
|---|---|---|---|
| A statistical probability-of-default model estimated from default data, mapped to grades | Yes — it applies statistical theory to produce a quantitative estimate. | Model risk management, with effective challenge by objective experts independent of development. | Conceptual soundness, outcomes analysis against realised defaults, ongoing monitoring, and documentation a third party can follow. |
| A weighted scorecard with committee-set weights and a grade guide, plus an override | Usually not — arithmetic and a deterministic rule, with no statistical theory underpinning the weights. | Credit risk review, internal audit, and whoever the bank's own policy names. | The same three things, under the credit rules rather than the model rules: that the criteria are sound, that the grades rank-order outcomes, and that nothing has drifted. |
| Expert judgement against written definitions, with no scorecard at all | No. | Credit risk review and internal audit. | That the definitions are detailed enough for an independent party to replicate a rating, and that ratings are in fact consistent across raters. |
| Any of the above with a generative or agentic tool preparing the analysis | Outside the guidance's scope by its own footnote — so the bank's own governance decides. | Whoever the bank's risk management and governance practices nominate. | The controls around the human decision the tool feeds: who may act on it, what is kept, how disagreements are handled. |
A bank is free to call its scorecard a model anyway, and some should. If the weights were fitted to internal loss data, the statistical theory is there and the first row applies. If the scorecard feeds an allowance estimate that is itself a model, the classification of the upstream component is worth settling in writing rather than by assumption. What is not defensible is the inverse move — classifying a scorecard out of scope and treating that as the end of the obligation.
Why "not a model" does not mean "not validated"
Because the credit rules asked for independent validation of the rating process long before any model risk guidance existed, and they still do. The Comptroller's Handbook lists independent validation of the rating process among the attributes of a sound rating system, separately from and in addition to regulatory examinations. The Basel framework requires that "internal audit or an equally independent function must review at least annually the bank's rating system and its operations, including the operations of the credit function and the estimation of PDs, LGDs and EADs" (Basel Framework, CRE36). Neither obligation is conditional on the scorecard being a model.
What changes, and what does not, when a scorecard falls outside the model definition
If it is a model: Model risk management, under the bank's model risk policy, with the model on the inventory.
If it is not: Credit risk review and internal audit, under the credit policy. It belongs on some inventory even so, or nobody owns it.
If it is a model: Validation, on a cycle set by model materiality.
If it is not: Independent validation of the rating process — the handbook's own phrase, and annual review of the rating system under the Basel framework.
If it is a model: Assessed and documented: design, assumptions, qualitative judgements, data selection.
If it is not: The same work. Are these the right factors, are the weights defensible, can two raters reach one answer from the grade guide.
If it is a model: Required, and compared against performance thresholds the bank has set.
If it is not: Required as back-testing: worse grades must show higher defaults and losses.
If it is a model: Effective challenge by objective experts with standing to force a change.
If it is not: A reviewer independent of the lending function, with access to the board.
If it is a model: The vocabulary, the inventory, the cycle and the committee that hears the result.
If it is not: The vocabulary, the inventory, the cycle and the committee that hears the result.
The Basel Committee's validation study set out principles that survive the classification question entirely, and they read like a brief for whoever ends up owning the work: the bank has primary responsibility for validation; validation "is fundamentally about assessing the predictive ability of a bank's risk estimates and the use of ratings in credit processes"; it is an iterative process; there is no single validation method; it "should encompass both quantitative and qualitative elements"; and validation processes and outcomes "should be subject to independent review" (Basel Committee, Working Paper No. 14).
The second principle is the one most often missed in practice. Validating a rating system is not only about whether the grades predict — it is about "the use of ratings in credit processes". A scale that discriminates beautifully and is overridden by lenders in a third of cases has failed as a rating system, and no amount of statistical work on the scorecard will show it.
Conceptual soundness: testing the criteria, not the arithmetic
The arithmetic of a scorecard is trivially checkable and almost never wrong. The interesting failures are in the criteria and in the places where judgement enters, and the 2026 guidance has a sentence aimed squarely at them: "when a model's design relies substantially on expert judgment, quantitative outcomes analysis helps to evaluate the quality of that judgment." That is the method. You cannot inspect a credit committee's weighting choice and declare it sound; you can look at what the grades it produces did afterwards.
A conceptual-soundness review of a rating scorecard has five practical questions behind it, and each has an evidence trail a bank can produce without new systems. Are the factors the ones that actually distinguish the bank's losses, or the ones that were in the previous scorecard? Do the grade guides describe conditions precisely enough that two raters agree — the Basel framework's test is that written rating definitions be detailed enough for an independent party to replicate a rating assignment? Is anything material missing from the factor set? Is the override path defined, with limits on who may use it and by how much? And is the version in use the approved one?
The sample bank's demonstration contains a worked example of the fourth and fifth questions at once. Its approved scorecard is v4.2, approved in January 2026; a v4.3 draft adds spot-rate exposure for truckload carriers and is going to validation before use. The missing factor is not hypothetical — the quarterly finding in the demonstration names it as one of three causes of the drift it found, alongside loans under $20M being graded by the lender alone and nineteen of twenty-two overrides carrying no reason. A conceptual-soundness review that had asked "what is missing from the factor set for this industry?" would have found the same gap a year earlier.
Versioned thresholds and tolerances: what a validator reads first in Risk Rating Validation
The approved scorecard version in use is a setting, not a constant: the re-scorer uses the approved version only and never a draft.
Outcomes analysis, back-testing and the low-default problem
Outcomes analysis for a rating system is a default back-test, and it runs into the oldest problem in credit validation: a bank that lends well does not generate enough failures to measure itself with. The Basel Committee's study is candid that statistical back-testing of rating systems is severely limited by data constraints, and that default correlation inside a portfolio makes naive tests reject sound systems in bad years.
What a validator can reasonably ask for, and a bank can reasonably produce, is therefore a hierarchy rather than a single test. Rank-ordering over a pooled multi-year window, reported by band rather than by grade where the counts are thin. The migration matrix and the share of double downgrades. The ratio of upgrades to downgrades by region and by lender. The disagreement rate of independent reviewers. And the same tests run inside segments, because a scale can rank-order across a book and fail inside one industry in one region. How to build the back-test, and how many years it needs, is the mechanics of this paragraph in full.
The 2026 guidance adds a procedural expectation that matters more than the statistics. Outcomes analysis and the rest of validation "may identify material errors or persistent deviations outside of the banking organization's established performance thresholds" — and in that case "model adjustment, recalibration, or redevelopment may be warranted". The phrase to notice is "established performance thresholds". A bank that has never written down how far its grades may deviate before someone acts has no way of identifying a persistent deviation, because it has no line. Writing the thresholds down first is the cheap half of the work.
Benchmarking when the data runs out
Benchmarking, in validation, means comparing your estimates with somebody else's for the same borrowers. The Basel study treats it as the ordinary companion to a weak statistical test, and gives the warning that goes with it: "a bank's internal rating should reflect its internal risk management practices, and should not be a mere replication of an external benchmark model". A scale calibrated until it matches an external reference has stopped being the bank's own view of its own borrowers.
Three benchmarks are available for a commercial rating system, in descending order of how much they prove and ascending order of how easy they are to get. An external rating or vendor model on the subset of borrowers that carries one is the strongest and the scarcest. The bank's own independent reviewers are the most useful in practice, because their disagreements arrive quarterly and are already documented. And internal cross-sectional comparison — this borrower against like borrowers graded by other lenders in other regions — needs no external data at all, which is what makes it runnable on the whole book rather than on a sample.
That third benchmark is the one most often skipped, and the one a validator should ask for by name. It tests the property the rating definitions exist to deliver: that the same borrower gets the same grade whoever grades it. In the sample bank's demonstration it is the nightly job — each borrower matched to three to five like-for-like borrowers in the same three-digit industry, within 25% on revenue, 0.10x on coverage and 0.5x on leverage, graded by a different lender, with the median of their grades reported beside the grade given.
Ongoing monitoring: what drift looks like in a rating system
Model drift in a statistical system is a population moving away from the data the model was fitted to. Drift in a judgemental rating system is people — a region, a line of business or one lender grading differently from the rest of the bank, usually gradually and usually in one direction. It does not show up in the scorecard, because the scorecard has not changed. It shows up in the gap between the scorecard and the grades actually assigned.
The three validation activities, the evidence each produces, and the way each one fails
| Activity | Evidence it should produce | How it fails in practice |
|---|---|---|
| Conceptual soundness | The factor set with a rationale for each factor, the weights and who approved them, the grade guides, the override rules, and a note of what is deliberately excluded. | Becomes a description of the scorecard rather than a test of it. The tell is a report with no finding and no rejected alternative. |
| Outcomes analysis | Rank-ordering by band over a pooled window, the migration matrix, double downgrades, upgrades per downgrade, and the same figures by segment. | Run bank-wide only, where a single drifting cell is a rounding difference; or run on too few defaults and reported as a point estimate. |
| Ongoing monitoring | A measurement that repeats on a schedule: grades against the scorecard, grades against matched peers, overrides with and without a reason, write-up scores, reviewer disagreement by region. | Not instrumented at all. Most banks discover drift from a sample or an examination, which is monitoring at an interval of a year or more. |
The 2026 guidance describes ongoing monitoring as an evaluation of the extent to which something is "performing as expected given potential changes in products, exposures, activities, clients, data relevance, or market conditions". For a rating system, add one more source of change the list does not name: the people doing the rating. Lenders join, leave, take on new industries and absorb the habits of whoever trained them. A region that was calibrated three years ago is a region that was calibrated on a different staff.
Who has to be independent, and of what?
Of the thing being judged, and specifically of benefiting from it. The requirement in the rating process is that assignments and periodic reviews be completed or approved by a party that does not directly stand to benefit from the credit, and the Basel framework adds that supervisors will look closely at whichever practices a bank uses to achieve that independence, which must be documented in its procedures and policies. For the validation itself, the expectation is "effective challenge" — in the 2026 guidance's words, "the critical analysis conducted by objective experts who evaluate model risk and effect appropriate changes", performed by people with the expertise, "sufficient independence to maintain objectivity, as well as the organizational standing and influence to effect any change".
Organizational standing is the part that fails quietly. A reviewer who finds that one region's grades lean a notch, and whose finding is answered by the regional executive and closed, has been independent and ineffective. The handbook's framing of the credit review function is the same idea from the other direction: the function should be "sufficiently staffed (both in numbers and in expertise) and appropriately empowered to independently validate and communicate the effectiveness of the risk rating system to the board and senior management" — and a smaller bank with no separate department can meet the requirement with staff who had no part in approving or managing the credits they rate.
Two signatures, in order: how Risk Rating Validation keeps the chain visible
The validator's questions, in the order a validation report answers them
- Is it a model under the bank's own policy, and is that written down?
The classification is a decision with a date and an owner, not an omission. Re-read it against the April 2026 definition rather than the 2011 one.
- Are these the right factors, and what is deliberately missing?
A factor set with no stated exclusions has not been reviewed. The sample bank's missing factor for one industry was named in a finding, not in a review.
- Can two raters reach the same grade from the written definitions?
The framework's test is replication by an independent party. Test it the obvious way: give two raters the same file.
- Do worse grades default more, inside each segment as well as across the book?
Pooled by band where the counts are thin, over at least three years, with the years named.
- Are the overrides defined, counted and tracked?
How many, by whom, in which direction, with what reason, and how the overridden loans performed afterwards.
- What has been measured since the last validation?
If the answer is "nothing until now", the validation is an event rather than a control, and the gap between events is where drift lives.
- Who can force a change, and have they?
Independence without standing produces findings that are answered and closed. Ask what the last three findings changed.
The honest summary of this page is that the classification argument is worth half an hour and no more. Whether the scorecard is a model decides the vocabulary, the inventory and the committee. What decides whether the rating system is trustworthy is the same three questions either way, and the third — is anyone watching between reviews — is the one a bank can change this quarter. To see continuous monitoring on a rated book, with the tolerances versioned and the disagreements kept, start from Risk Rating Validation; to weigh it against the other ways of keeping grades right, read the four approaches compared.
Questions people ask
Is a credit risk rating system a model?
Usually not, under the definition the agencies issued in April 2026. That guidance defines a model as a complex quantitative method applying statistical, economic or financial theories to produce quantitative estimates, and expressly excludes simple arithmetic such as spreadsheet calculations and deterministic rule-based processes with no such theory behind them. A weighted scorecard with committee-set weights and a grade guide is normally both of those. A probability-of-default model estimated from default data is not, and is squarely inside.
Does SR 26-2 apply to a judgemental rating scorecard?
Generally no, on two counts: the scorecard usually falls outside the definition of a model, and the guidance is expected to be most relevant to banking organisations with over $30 billion in total assets. Neither point removes any obligation — the Comptroller's Handbook expects the rating process to be independently validated in addition to examinations, and the Basel framework expects internal audit or an equally independent function to review the rating system at least annually.
What is outcomes analysis in model validation?
Comparing what the thing predicted with what actually happened. The April 2026 guidance describes it as comparing outputs to corresponding real-world outcomes to assess performance against objectives and business use, and notes that it can take the form of testing during development, ongoing monitoring reports, or standalone work such as back-testing or outlier analysis. For a rating system it is the default back-test, plus the migration matrix and the segment-level versions of both.
How do you validate a rating system with few defaults?
By pooling, by lowering the claim, and by adding benchmarks. Pool grades into three or four bands and years into a window of three or more, so a band has enough defaults to carry a rate. Claim rank-ordering rather than calibration, which is the claim the OCC actually asks of a scale not tied to statistical probabilities. Then add comparisons that need no defaults at all: independent reviewers' disagreement rates, and each borrower against like borrowers graded by other lenders.
Who has to be independent of the rating system, and of what?
Of benefiting from the credit, and of the work being judged. Rating assignments and periodic reviews must be completed or approved by a party that does not directly stand to benefit from the extension of credit, and the review of the system itself belongs to internal audit or an equally independent function at least annually. The 2026 guidance adds the quality that makes independence real: effective challenge needs objective experts with the standing and influence to actually force a change.
Do community banks need model validation for risk ratings?
They need independent validation of the rating process; whether it is called model validation depends on what produces the grade and on the bank's own policy. The model risk guidance is tailored away from organisations under $30 billion in assets, but the credit expectations are not size-tailored in the same way. The handbook explicitly accommodates a bank with no separate credit review department: the requirement can be satisfied using staff not directly involved in approving or managing the rated credits.