Skip to content
Portfolio Theory and Quantitative Investment / Robust Portfolio Construction

Estimation Error and Shrinkage Covariance

A mean–variance optimiser does exactly what it is told, and that is the problem. Feed it a covariance matrix estimated from five years of monthly data on fifty stocks and it will find the combinations of stocks whose risk looks lowest in that sample. Those are the combinations where estimation error happens to be most favourable, so the optimiser concentrates its biggest bets on the least reliable numbers. The realised risk of the "optimal" portfolio comes out far above what the optimiser promised, and the manager's track record understates their skill. This note makes that failure precise, then fixes the risk side of the problem with the Ledoit–Wolf shrinkage estimator: a weighted average of the sample covariance matrix and a highly structured target, with the weight chosen by formula. The expected-return side of the same problem is handled in Black–Litterman.
Learning objectives. By the end of this note you should be able to:
  • explain why estimation error makes mean–variance optimisers "error maximisers", and why the sample covariance matrix is singular when there are more stocks than observations;
  • state the shrinkage estimator Σ^=δF+(1−δ)S\hat\Sigma = \delta F + (1-\delta)S and describe the constant-correlation target;
  • state the Ledoit–Wolf formula for the optimal shrinkage intensity and read each of its three ingredients;
  • implement the estimator and measure what it does to the realised risk of a minimum-variance portfolio;
  • place shrinkage alongside the other defences against estimation error: factor models, constraints and 1/N1/N.

In this lesson

  1. The problem: optimisers maximise estimation error
  2. Two extremes and the compromise
  3. The shrinkage target: constant correlation
  4. The optimal shrinkage intensity
  5. What shrinkage buys: the evidence
  6. Shrinkage among the other defences
  7. Code: the estimator and its effect on a minimum-variance portfolio

Before you start

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Portfolio Theory and Quantitative Investment syllabus.