"Rating Credit Risk" is the Comptroller's Handbook booklet that sets out what the OCC expects of a bank's commercial risk rating system, and almost everything in it reduces to two demands and one published number. The demands are that ratings be accurate and timely, and that the rating process be validated by somebody independent. The number is the point at which a bad sample stops being a list of disagreements and becomes an investigation of the rating system itself: more than 5 percent of the credits reviewed, or 3 percent of the dollars. Botminds' Risk Rating Validation runs the bank's own version of that test every night over every rated loan, rather than on the sample a quarterly review can reach, so a credit team sees the number before an examiner does.
The booklet was issued in April 2001 and last updated on 26 June 2017; it has applied to federal savings associations as well as national banks since May 2012. Quotations below are from the booklet unless another source is named. The bank in the worked examples is fictional sample data from the solution: Kestrel Valley Bank, 2,305 rated commercial loans.
The numbers the booklet publishes, and the one it does not
5%
of the number of credits reviewed — past this, examiners investigate the root cause
3%
of the dollar amount reviewed, the same line measured the other way
1yr
the longest the booklet expects between formal reviews of a credit — smaller performing credits may be reviewed by exception
None
published target for how many grades a bank should have, or what they should be called
What the handbook asks of a risk rating system
The booklet opens by refusing to prescribe one. "No single credit risk rating system is ideal for every bank," it says, and then lists the attributes that "should be present in all systems". That list is the most useful page in the document, because it is short, it is testable, and most rating-system findings are a failure of one of its lines rather than of anything exotic.
The eleven attributes the OCC expects of a national bank's rating system
01
Integrated
The system "should be integrated into the bank’s overall portfolio risk management" and form the foundation for measuring, monitoring and reporting credit risk.
02
Owned by the board
The board "should approve the credit risk rating system and assign clear responsibility and accountability for the risk rating process", and should get enough information to oversee it.
03
Complete
All credit exposures should be rated; where individual grades are not assigned, the portfolio of such exposures gets a composite grade.
04
Granular enough
Enough grades to differentiate risk, which "most rating systems" achieve with several pass grades.
05
Accurate and timely
One sentence, with “must” where the lines around it say “should”.
06
Defined
Criteria "clear and precisely defined" using objective and subjective factors.
07
About borrower and structure
Ratings "should reflect the risks posed by both the borrower’s expected performance and the transaction’s structure".
08
Dynamic
The system "should be dynamic — ratings should change when risk changes".
09
Independently validated
The rating process "should be independently validated (in addition to regulatory examinations)".
10
Back-tested
The bank should determine through back-testing whether the assumptions implicit in its definitions hold, and change the definitions if they do not.
11
Documented
Each rating "well supported and documented in the credit file".
Two of those lines do most of the work in an examination. "Accurate and timely" is the one a sample of credits tests directly. "Independently validated" is the one that decides whether the bank found its own problems first. The rest are conditions that make the first two possible: a system nobody defined, nobody owns and nobody updates will not be accurate for long.
What the OCC means by accurate and timely — the two words the booklet turns on
The booklet states it as one short sentence in the imperative: "Risk ratings must be accurate and timely." Not "should", as in most of the list around it. The two words mean different things and fail differently.
Accurate means the grade matches the credit's condition under the bank's own written criteria. It is tested by re-grading: somebody independent reads the file, applies the definitions, and compares. Timely means the grade matches the credit's current condition. A grade that was right at approval and has not moved while the borrower's earnings fell 18 percent is inaccurate now, and the booklet is specific about how often that has to be checked: "All credits should receive a formal review at least annually to ensure that risk ratings are accurate and up-to-date." Larger, newer, higher-risk and complex credits should be looked at more often, and ratings should be updated "whenever relevant new information is received" rather than only on the review cycle.
The 2020 interagency guidance on credit risk review systems makes the same pair the foundation of everything built on top of it: "The foundation for any effective credit risk review system is accurate and timely risk ratings to assess credit quality and identify or confirm problem loans." Every other control — the watch list, the allowance, the board reporting — is downstream of two words.
What is the 5 percent risk rating rule, and what happens past it?
It is the threshold at which a set of rating disagreements becomes a question about the system. The sentence is worth reading exactly as written, because three things in it are commonly misquoted.
When examiners discover significant risk rating inaccuracies (generally, greater than 5 percent of the number of credits reviewed, or 3 percent of the dollar amount), they must investigate to determine the root causes and decide whether to expand their loan review sample.
OCC, Comptroller's Handbook, ‘Rating Credit Risk’
First, "generally". The figures describe what significant usually means, not a line that converts automatically into a finding. Second, there are two measures and they are joined by "or" — a book can pass on count and fail on dollars, which is what happens when the wrong grades are the large ones. Third, what follows is not a downgrade. It is an investigation of root cause, and a decision about whether to widen the sample. The consequence of a bad sample is a bigger sample.
That matters for how a bank should think about the number. Five percent of a quarterly sample at a mid-sized bank can be a handful of loans. The exercise that follows is not about those loans; it is about whether the bank's rating process produces them systematically, which is why the booklet's next move is to list what examiners look at.
How examiners test a sample, and what they look at when it fails
The test itself is simple: examiners grade a sample of credits under the bank's criteria and compare with the bank's grades, documenting the analysis and the bank's agreement or disagreement. What makes the exercise more than a tally is the second step. Reviewing individual grades, the booklet says, "discloses much about how well the overall process is functioning", and it sets out what examiners should determine about the process itself.
What examiners determine when they review the rating process, not the loans
- Whether there are enough grades
"Whether there is a sufficient number of ratings to distinguish between the various types of credit risk the bank assumes." A scale too coarse to separate risks cannot be accurate, however carefully it is applied.
- Whether the controls around the process work
The effectiveness of the controls on rating assignment, change and review — who may set a grade, who confirms it, what happens to an override.
- Whether the people who use it understand and support it
"Whether line lenders, management, and key administrative and control staff understand and effectively use and support the risk rating process." A system the lenders work around is not a system.
- Whether it holds up its end of credit risk management
The effectiveness of the rating system as part of the bank’s overall credit risk management process — the allowance, pricing, approval authority and portfolio reporting that all rest on the grade.
- Whether anything in the bank discourages honest grading
Examiners are told to be alert for "impediments or disincentives that may prevent the system from functioning properly" — compensation and relationship structures among them.
A bank can run the same arithmetic on itself, and the useful version is not one number for the institution. Measured bank-wide, a rate below the line reassures; measured by region, by industry and by lender, it either stays below the line everywhere or it does not. That is the difference between a figure for the board and a figure somebody can act on: the second one names a cell. It is also the measurement the booklet's own reporting list is built to support: the last of its six items is rating changes by line of business, loan officer and location.
One practical caution about counting. The two measures move apart when the inaccurate grades are the large ones, so a bank that tracks only the count will miss exactly the cases that matter to the allowance. The booklet's instruction about reporting applies here too — show both dollar volume and item count, "because some reports can be skewed by changes in one large account". A single $40 million credit graded two notches high can take a region past the dollar line on its own while the count barely moves.
Severity, pattern and location: what decides how far the sample widens
Having found significant inaccuracies, examiners decide whether to expand. The booklet names four determining factors, and they are worth knowing because they explain why two banks with the same error rate get very different examinations.
The four factors that decide whether a loan review sample is widened
| The factor | What the booklet says | What makes it worse |
|---|---|---|
| Nature and pattern | Inaccuracies "within pass categories, problem credits that are passed, missed ratings with a few large credits or several smaller credits, and inaccuracies in a specific portfolio or location". | A pattern. Scattered errors look like noise; errors concentrated in one industry, one region or one lender look like a cause. |
| Severity | "The severity of inaccuracy, i.e., how many grades away the rating is from what it should be." | Two notches rather than one. A single-notch disagreement is a judgement call; a two-notch gap is a different reading of the definitions. |
| The allowance and capital | The adequacy of the allowance and of capital. | Inaccuracies large enough to mean the allowance was set from the wrong grades. |
| Distortion of the picture | Whether inaccurate ratings "distort overall portfolio risk and the bank’s financial statements". | Optimism that runs one way, because errors in one direction do not cancel out. |
Pattern and location are where a bank can act before anyone else looks. At the fictional sample bank, loan review changed 5 of the 82 Mountain-region grades it sampled last quarter — 6.1 percent of the loans and 4.5 percent of the dollars, against 3.7 percent and 2.4 percent for the bank as a whole. All five changes were downgrades and three were trucking loans. That is not five loans with a problem; on the booklet's own factors it is a pattern in a specific portfolio and location.
One region against the lines examiners watch, and against its own bank
Independent validation: who counts as independent?
The booklet's requirement is one line: "The risk rating process should be independently validated (in addition to regulatory examinations)." The parenthesis is the important half. An examination is not the bank's validation, and a bank whose only check on its grades is the one the examiners perform has not met the expectation — it has outsourced it to the people who will write the finding.
Who qualifies is answered more fully in the 2020 interagency guidance. The principle is stated as an outcome rather than a job title: "A risk rating review that is independent of the lending function and approval process can provide a more objective assessment of credit quality." And independence is defined by what the reviewer is free of, not by where someone sits — the reviewers must not control the loan or be part of or influenced by the people who approved it, and where a smaller bank uses its own staff, their pay must not be influenced by the ratings assigned.
That definition is deliberately practical, because many banks cannot staff a separate department. The guidance allows the review to be done by qualified and independent peers, managers or loan committees, by part-time or full-time employees, by an internal credit review department, or by external consultants. A small bank can satisfy the expectation with staff who were not involved in approving the credits they are checking. What it cannot do is have the lender confirm the lender's own grade — which is the subject of who stands behind a grade.
The reports the handbook expects a board to see
The booklet treats reporting as a control rather than as paperwork: these reports "provide feedback about the risk rating system". It then lists, unusually specifically, what the reporting should generate or let the user calculate — and the list is a good self-assessment, because most banks produce the first two and few produce all six.
Six measures the booklet expects a rating system's reporting to produce
| The measure | What it tells you | Where it usually goes wrong |
|---|---|---|
| Credits whose ratings changed more than one grade | Double downgrades: where the system missed deterioration and then caught up all at once. | Counted, but not traced back to who set the original grade. |
| Seasoning of ratings | How long credits sit in one grade. | A grade that has not moved in four years is either a stable credit or an unexamined one, and the report alone cannot tell you which. |
| The velocity of rating changes | How quickly grades are moving. | Looks healthy in a good year and says nothing about direction. |
| Default and loss history by rating category | Whether the grades predict anything. | Needs several years of history per grade, which is why many banks never start. |
| The ratio of upgrades to downgrades | The direction of travel, net of everything else. | Rarely cut by region or lender, which is where the asymmetry lives. |
| Rating changes by line of business, loan officer and location | Pattern and location — the first two of the four factors that widen a sample. | The report most often missing, and the one that would find drift first. |
Risk Rating Validation: loan review disagreement by region, against the published lines
Why the handbook is more worried about optimism than about error
Rating errors are not symmetric, and the booklet says so twice. The first time is about consequence: "Examiners should be most concerned when rating inaccuracies understate risk; however, any significant inaccuracy should be criticized because it will distort the picture of portfolio risk and diminish the effectiveness of interdependent portfolio management processes." Both directions are criticized; one is worse.
The second time is about cause, and it is more pointed: "Whatever the cause, it can be relatively easy for loan officers and line managers to rate credits a step, or more, above what they deserve." A step, or more. That is the mechanism behind almost every drift finding — not fraud, not incompetence, but a long series of defensible single-notch decisions that all lean the same way.
The booklet then names the structural reason, and it is the sentence a credit committee should read before setting next year's targets: examiners are told to watch for "compensation programs that encourage lenders and management to understate risk in order to boost risk-adjusted returns or to generate incremental business by lowering risk-based pricing." A bank that prices off the grade and pays off risk-adjusted return has built an incentive to grade generously, and no amount of definition-writing will cancel it.
This asymmetry is why a one-sided error rate is more alarming than a two-sided one. At the sample bank, the Mountain region upgraded 1.8 loans for every downgrade while the bank ran at 0.9, none of its 38 trucking loans was graded stricter than the scorecard, and its grade 4 loans defaulted at 1.9 percent over three years against 0.8 percent expected. Each of those is a weak signal on its own. Together they point one way.
Pattern and location: nine industries against five regions
Where credit risk review guidance picks up
The booklet is about the rating system. The 2020 interagency guidance on credit risk review systems — issued by the Federal Reserve, the FDIC, the OCC and the NCUA — is about the function that checks it, and the two are designed to be read together. It describes the attributes of an effective rating framework, sets out who may perform the review and what independence means, asks that risk rating differences between lenders and reviewers be resolved "according to a pre-arranged process", and expects a bank whose own scale differs from the regulatory categories to keep documentation translating between them.
One procedural detail from the FDIC's examination manual is worth knowing, because it sets the clock a credit team is actually working against. A full-scope, onsite examination is required "at least once during each 12-month period", extendable to 18 months for banks with less than $6 billion of total assets that are well capitalized, rated 1 or 2 on both management and composite, not under a formal enforcement proceeding, and not recently acquired. Twelve to eighteen months is how long a drift can run before an examination lands on it — and the booklet's own expectation of an annual formal review of every credit is the inner loop meant to catch it sooner.
What the booklet does not do is tell a bank how to find its own problems between those dates. Sampling is the historical answer: the sample bank's loan review reaches 401 of 2,305 rated loans in a quarter, 17 percent of the book, chosen for size and risk. Re-scoring every rated loan nightly on the bank's own scorecard and comparing each borrower with look-alikes graded elsewhere answers the booklet's question — are our grades accurate, by pattern and location? — over the other 83 percent too. The grades themselves do not move: the comparison proposes, a regional credit officer confirms and the chief credit officer signs, which is the governance the override line depends on.
Questions people ask
What does the OCC require of a risk rating system?
Eleven attributes, listed in the "Rating Credit Risk" booklet: the system is integrated into portfolio risk management, approved by the board with clear accountability, applied to all exposures, granular enough to differentiate risk, accurate and timely, precisely defined on objective and subjective factors, reflective of both borrower and structure, dynamic, independently validated, back-tested, and documented in the credit file. The OCC "does not advocate any particular rating system" beyond those attributes.
Does a bank have to validate its risk rating system?
Yes, and separately from the examination. The booklet says the rating process "should be independently validated (in addition to regulatory examinations)". The 2020 interagency guidance sets the test for independence: the reviewers must not control the loan or be part of or influenced by the approval process, and where a smaller bank uses its own staff, their pay must not be influenced by the ratings assigned.
When do examiners expand a loan review sample?
When they find significant rating inaccuracies — generally more than 5 percent of the credits reviewed or 3 percent of the dollars — they must investigate the root causes and decide whether to widen. Four factors drive the decision: the nature and pattern of the inaccuracies, their severity in notches, the adequacy of the allowance and capital, and whether the errors distort the picture of portfolio risk.
How often must risk ratings be updated?
Whenever relevant new information arrives, and formally at least once a year. The booklet asks that whoever is responsible for the grade "review and update risk ratings whenever relevant new information is received", and that all credits receive a formal review at least annually; large, new, higher-risk and complex credits should be reviewed more often. Smaller performing credits may be handled as exceptions.
What makes a risk rating inaccurate?
A grade that does not match the credit's current condition under the bank's own criteria — either because it was wrong when assigned or because it has not moved since. Severity is measured in notches: "how many grades away the rating is from what it should be". Inaccuracies that understate risk concern examiners most, but any significant inaccuracy is criticized because it distorts the portfolio picture.
Can the rating system itself be an exam finding?
Yes, and that is the real exposure. Past the 5 percent and 3 percent lines, examiners investigate root causes rather than simply regrading the loans, and look at whether there are enough grades, whether the controls work, whether lenders understand and support the process, and whether compensation or relationship structures discourage honest grading. The outcome can be a criticism of the system, not of a handful of credits.