A fund manager who generates alpha is exploiting an informational advantage — but that advantage often originates in the security analysts who supply earnings forecasts and buy/sell recommendations. Evaluating whether analysts have genuine forecasting skill is therefore a precondition for evaluating the portfolio-level alpha they are supposed to generate. An analyst who is systematically optimistic, who forecasts no better than a naive extrapolation, or whose errors follow predictable patterns is a liability rather than an edge.
This note develops the quantitative tools for auditing analyst forecasts: Theil's inequality coefficient as an overall accuracy metric, the Prediction Realization Diagram for diagnosing types of error, and the decomposition of forecast error into bias, inefficiency, and irreducible noise. The link to the performance-persistence evidence and the market anomalies literature is drawn at the close.
In this lesson
- Why evaluate forecasts formally?
- Overall forecast accuracy: mean squared forecast error
- Theil's inequality coefficient
- Diagnosing forecast errors: the Prediction Realization Diagram
- Numerical error decomposition
- Comparing individual analysts against the consensus
- Link to the market anomalies evidence
Before you start
The full lesson, with worked examples and exercise solutions, is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or browse the Portfolio Theory and Quantitative Investment syllabus.