VaR backtesting is the check that a value-at-risk model forecasts losses as often as it says it will. Each business day the fund's actual gain or loss is set against the one-day VaR the model produced for that day; a loss larger than the forecast is an exception, and the number of exceptions over a window, usually 250 business days, is compared with the number the confidence level implies. At 99%, a correct model is beaten on about one day in a hundred, so about 2.5 times in 250 days. Rule 18f-4 makes every full-program fund run this check at least weekly and report the count to the SEC, confidentially, on Form N-PORT. Counting is the easy half. The harder half is deciding what an unusual count means and explaining each exception, which is the half Botminds' Derivatives Risk Management Program is built to carry for the derivatives risk manager.
The probabilities on this page are our arithmetic from the binomial distribution. Fund examples are fictional sample data.
A correct 99% model: how many exceptions to expect in 250 days
What is VaR backtesting?
VaR backtesting is a comparison of forecasts with outcomes. The Basel Committee put it in one sentence in January 1996, when it set the framework banks still use: "The essence of the technique is to compare actual trading results with model-generated risk measures" (BCBS). A VaR model at 99% claims that losses beyond its number happen one day in a hundred. Backtesting checks the claim.
It tests only one thing: the frequency of losses beyond VaR. It says nothing directly about how large those losses were, which is the tail VaR ignores and stress testing exists to examine. A model can pass its backtest and still badly understate what happens on the worst days, and a model can fail it because the market changed while the model's three-year history did not.
The research record on models in use is thin. Berkowitz and O'Brien's 2002 study in the Journal of Finance described itself as "the first to provide a detailed analysis of the performance of models actually in use", for a sample of large US bank holding companies (Journal of Finance). No regulator publishes backtest results for US funds, because the SEC keeps them confidential.
How Rule 18f-4 defines a backtesting exception
Rule 18f-4 defines the test precisely. A fund must backtest "no less frequently than weekly" by "comparing the fund's gain or loss that occurred on each business day during the backtesting period with the corresponding VaR calculation for that day, estimated over a one-trading day time horizon, and identifying as an exception any instance in which the fund experiences a loss exceeding the corresponding VaR calculation's estimated loss" (17 CFR 270.18f-4(c)(1)(iv)).
Three details matter in practice. First, the horizon: the VaR test itself uses 20 trading days, but the backtest uses a one-day VaR from the same model, so the engine has to produce both every day. Second, the comparison is with "the fund's gain or loss that occurred", the actual result, not a hypothetical one. Third, every business day in the period is tested even though the test runs weekly; a weekly run covers five days, not one.
The rule does not set a window or a threshold. It does not say how many exceptions are too many, or over what period to count them; that is left to the program, which must include the backtesting results in the derivatives risk manager's annual review of the VaR model. The SEC chose not to require board reporting of each exception, "Because a fund is expected to experience a given number of backtesting exceedances" (adopting release).
How many backtesting exceptions are too many?
More than about four in 250 days begins to look suspicious, and ten or more is very unlikely from a correct 99% model. The first figure shows why. A correct model produces 0 to 4 exceptions about 89% of the time, 5 to 9 about 11% of the time, and 10 or more in about one window in 4,000. The reasonable reading of a count of six is "look at the model", not "the model is wrong".
Bank regulators turned that arithmetic into the traffic light the Basel Committee introduced in 1996. Its yellow zone starts at five exceptions because "five or fewer exceptions will be obtained 95.88% of the time" from a correct model, and ten or more "should generally lead to an automatic presumption that a problem exists with a bank's model" (BCBS supervisory framework). US bank capital rules still count exceptions each quarter over the most recent 250 business days and raise the multiplier on a bank's VaR-based capital as the count rises (12 CFR 217.204). Funds pay no capital charge, but the table is the most widely used scale for reading a count.
How US bank rules read an exception count: the multiplier on VaR capital
| Exceptions in 250 business days | Multiplication factor | Chance from a correct 99% model | Reading |
|---|---|---|---|
| 4 or fewer | 3.00 | 89.2% | Green: consistent with the model |
| 5 | 3.40 | 6.7% | Yellow: review |
| 6 | 3.50 | 2.7% | Yellow: review |
| 7 | 3.65 | 1.0% | Yellow: review |
| 8 | 3.75 | 0.3% | Yellow: review |
| 9 | 3.85 | 0.08% | Yellow: review |
| 10 or more | 4.00 | under 0.03% | Red: a problem with the model is presumed |
Across a fund complex the arithmetic has a second consequence. With 45 funds whose models are all correct, and exceptions that fell independently, about five would be expected to show five or more in a given 250-day window by chance alone; funds that share markets and a model will cluster even more. A firm that treats every fund in the yellow zone as a model failure will spend its reviews on noise; one that never looks will miss the fund whose model is actually wrong.
VaR backtesting methods: the Kupiec test, Christoffersen and the traffic light
Three methods dominate. The traffic light counts exceptions against fixed bands. Kupiec's proportion-of-failures test, published in the Journal of Derivatives in 1995, asks whether the observed rate of exceptions is statistically different from the model's 1%, using a likelihood ratio compared with a chi-squared distribution with one degree of freedom (Kupiec, 1995). Christoffersen's 1998 work in the International Economic Review added a test of independence: whether exceptions cluster, which a correct model's should not (Christoffersen, 1998).
The Kupiec test on 250 days at 99%: two-sided, and stricter than the traffic light at seven
0
5.03
Rejected: too few. The model overstates risk.
1
1.18
Consistent with 99%
2–4
0.09–0.77
Consistent with 99%
5
1.96
Consistent with 99%
6
3.56
Consistent, only just
7
5.50
Rejected: too many
10
12.96
Rejected: too many
The Kupiec test reads both tails. Zero exceptions in 250 days rejects a 99% model as firmly as seven does, because a model that is never beaten is probably overstating risk. Under Rule 18f-4 an overstated fund VaR is conservative for the fund's own VaR, but a model that overstates the reference portfolio's VaR flatters the ratio, so a zero deserves a look too.
The independence test catches a different failure. Six exceptions spread across a year are one thing; six in three weeks during a sell-off mean the model reacted too slowly to a change in volatility, and the next stressed month will likely bring more. A program that reports only the count throws that information away.
Clean or dirty: which gain or loss to backtest
Rule 18f-4's text points to actual results: "the fund's gain or loss that occurred on each business day". The European guidance for UCITS allows both and prefers the other. CESR's 2010 guidelines describe backtesting on "the effective changes ('dirty back testing') or the hypothetical changes ('clean back testing')" and add that it "is ideally performed on the hypothetical changes in the portfolio's value", that is, on end-of-day positions held unchanged to the next day (CESR/10-788).
The difference matters most for funds that trade actively. Actual results include the profit or loss from trades made during the day, fees and income, none of which the previous evening's VaR could have known about. A fund that cuts a losing position at noon may show a smaller loss than the hypothetical one; a fund that adds to it may show a larger one. Running both, and explaining the gap when they disagree about an exception, is good practice even where only one is required.
Backtesting under three regimes: funds in the US and EU, banks in the US
Rule 18f-4 (US funds): No less frequently than weekly, covering each business day.
UCITS (CESR/10-788) and US banks: UCITS: "at least on a monthly basis", each business day tested. Banks: exceptions counted each quarter.
Rule 18f-4 (US funds): Actual gain or loss against that day’s one-day VaR.
UCITS (CESR/10-788) and US banks: UCITS: actual or hypothetical, hypothetical preferred. Banks: trading losses, excluding fees, commissions and intraday trading.
Rule 18f-4 (US funds): None set; the program decides.
UCITS (CESR/10-788) and US banks: UCITS: more than 4 in 250 days reported to senior management quarterly. Banks: the 250-day multiplier table.
Rule 18f-4 (US funds): The SEC on Form N-PORT, confidentially; the board through regular reports at a frequency it sets.
UCITS (CESR/10-788) and US banks: UCITS: the competent authority semi-annually when the count exceeds 4. Banks: the count sets the capital multiplier.
How often must a fund backtest its VaR model, and why weekly?
At least weekly. The SEC proposed daily backtesting and moved to weekly after commenters asked for weekly or monthly, writing that "the limited additional costs for weekly backtesting relative to monthly testing are justified by the benefits of providing more-recent information regarding the effectiveness of a fund's VaR model", and lined it up with the weekly stress test. The derivatives risk manager may run it more often, never less.
Europe is slower but has a hard reporting line. CESR expects backtesting "at least on a monthly basis", states that "the accurate number of 'overshootings' for each UCITS is 2.5 for the most recent 250 business days", and asks UCITS to tell senior management at least quarterly, and the regulator semi-annually, when the count "exceeds 4". A US manager running both kinds of fund can satisfy both with one weekly process and two thresholds.
Does the SEC see a fund's backtest results?
Yes, but nobody else does. Form N-PORT asks each full-program fund for the "Number of exceptions that the Fund identified as a result of its backtesting of its VaR calculation model", alongside its median daily VaR and median VaR ratio. The SEC made all three confidential: "Information about a fund's designated index will be made publicly available, but not a fund's median daily VaR, median daily VaR ratio, and backtesting information" (85 FR 83162).
The consequence is that no public benchmark exists. A derivatives risk manager cannot compare a fund's exception count with its peers', and a board cannot either. The useful comparison is internal: this fund against the binomial expectation, against its own history and against the other funds in the complex that use the same model.
One sample quarter of backtesting in Derivatives Risk Management Program, across 45 funds
23
backtest exceptions across 45 funds in the third quarter of 2026
~29
expected at 99%: 45 funds × 64 business days × 1%
2
funds in the firm’s 5–9 review zone over 250 days
0
funds at 10 or more, where the firm’s policy requires a model review
Explaining an exception, not just counting it
A count tells the derivatives risk manager whether to worry; an explanation tells him what to change. Each exception has a cause that is usually findable: a position the model treats too simply, a risk factor it lacks, a gap between the market the model prices and the one the fund trades in, or an event no three-year history contains. A program that records the cause beside the count turns the annual model review from a statistics exercise into a list of fixes, which is the "basis for any material changes" to the VaR model the rule requires a fund to keep.
In the sample firm's data, the Backtest agent in Derivatives Risk Management Program counted 6 exceptions in 250 days for the Emerging Markets Debt Fund. The Explainer agent traced each to its source; the largest, on September 24, 2026, was a 1.88% loss against a one-day VaR of 1.21%, on the day a central bank suspended onshore currency settlement for non-resident investors. The model had no factor for the gap between onshore and offshore currency prices, and four of the six exceptions shared that cause, so the analyst prepared a model review rather than an explanation.
The weekly backtest in Derivatives Risk Management Program: exceptions by fund, and the reasons behind six
The source behind the exception, kept in its own language
The same discipline applies to the other test the program runs every day. The relative VaR test and its reference portfolio depends on two VaR figures, and only the fund's is backtested by the rule. How one day's figures travel from the model to the board is traced in one VaR ratio, followed from model to board report. The weekly backtest, the explanations and the model reviews all sit in Derivatives Risk Management Program.
Questions people ask
What is an overshoot in a value at risk model?
An overshoot, overshooting or exception is a day on which the actual loss was larger than the VaR the model forecast for that day. CESR's guidelines define it as "a one-day change in the portfolio's value that exceeds the related one-day value-at-risk measure calculated by the model". Rule 18f-4 calls the same event an exception.
What is VaR backtesting under BCBS rules?
The Basel Committee's 1996 framework compares each of the last 250 trading days' results with the one-day 99% VaR and sorts the count of exceptions into green (0–4), yellow (5–9) and red (10 or more) zones. US bank rules apply it through the multiplier table in 12 CFR 217.204, from 3.00 at four or fewer exceptions to 4.00 at ten or more.
Does Rule 18f-4 say how many exceptions are too many?
No. The rule defines an exception and requires weekly backtesting, but sets no window or threshold. The derivatives risk manager must review the VaR model, including its backtesting, at least annually, and the fund must record the basis for any material change to the model.
Can expected shortfall be backtested?
Not as simply as VaR, which is why it was long thought hard to evaluate. Fissler and Ziegel showed in the Annals of Statistics in 2016 that "the pair (Value at Risk, Expected Shortfall) is jointly elicitable under mild conditions", which gives a statistical basis for comparing forecasts of the two together (Fissler and Ziegel, 2016). Rule 18f-4 requires VaR and its backtest, not expected shortfall.
Should backtesting use hypothetical or actual results?
Rule 18f-4's wording refers to the fund's actual gain or loss on each business day. CESR's UCITS guidelines prefer hypothetical changes, with positions held fixed, because they isolate the model from intraday trading. Running both shows whether an exception came from the model or from what the fund did that day.
Is a fund's backtest count public?
No. The number of backtesting exceptions is reported to the SEC on Form N-PORT but kept confidential, as are the fund's median daily VaR and median VaR ratio. Only the name of the fund's designated reference portfolio is published.