A bank estimates VaR daily; each day the actual loss either does or does not exceed the estimate. Backtesting converts this stream of violation indicators into a formal statistical test of whether the VaR model is correctly calibrated. At 99% VaR, violations should occur on about 1% of days — roughly two to three times a year. Too many violations signals a model that underestimates risk; too few suggests it is too conservative, tying up needless capital.
The statistical challenge is that violation indicators have two natural properties to test: (1) correct coverage — the unconditional violation probability is ; and (2) independence — violations are not clustered in time. A model that gives exactly 1% violations but always clusters them into stressed periods has failed to capture volatility dynamics, even though it passes the count test. The Christoffersen conditional coverage test catches both.
Expected shortfall (ES), now the preferred measure under FRTB, presents a harder backtesting problem: ES averages the size of exceedances, not just their frequency. The violation residual test provides a formal framework, but ES's non-elicitability means there is no natural scoring function for comparing ES estimators — unlike VaR, for which the scoring function from Proposition 9.8 gives a direct model comparison criterion.
Learning objectives. By the end of this note you should be able to:
- state Lemma 9.5 and explain why VaR violations form an i.i.d. Bernoulli process;
- compute the Kupiec POF test statistic and interpret its conclusion;
- describe the Christoffersen geometric-spacing test for violation independence;
- define the ES violation residual and explain how to test ES calibration;
- define elicitability, state which risk measures are and are not elicitable, and explain why this matters for backtesting;
- describe the Basel traffic-light system for market risk model monitoring.
In this lesson
-
- The violation indicator process
-
- The Kupiec POF test (unconditional coverage)
-
- The Christoffersen test (conditional coverage)
-
- Backtesting ES: violation residuals
-
- Elicitability and the limits of ES comparison
-
- Basel traffic-light system
Before you start
- VaR Estimation in Practice: Variance-Covariance, Historical Simulation, Monte Carlo
- Scaling, Multi-Period Risk, and Regulatory Capital
The full lesson, with worked examples and exercise solutions, is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or browse the Quantitative Risk Management syllabus.