A quant fitting a factor beta, a hedge ratio, or a yield-curve slope is doing the same thing every time: finding the straight line through a scatter of points that a downstream trading decision will lean on. This note builds that line from first principles. It states the simple linear regression model, derives the ordinary least squares (OLS) estimators by minimising the residual sum of squares, and — the payoff for having already worked through Least Squares as Orthogonal Projection — shows that this calculus exercise is identical to projecting the data vector onto a two-dimensional subspace. Every property used in the rest of this topic (unbiasedness, the residual identities that drive ANOVA, the exact sampling distributions of the next lesson) traces back to this one geometric fact, so it is worth seeing clearly before the algebra starts.
Learning objectives. By the end of this note you should be able to:
- state the simple linear regression model and the assumptions it rests on;
- derive the OLS normal equations by minimising the residual sum of squares, and solve them in closed form;
- prove the key residual identities (residuals sum to zero; residuals are uncorrelated with ) directly from the normal equations;
- recognise OLS as an orthogonal projection onto , and read the fitted-value/residual decomposition off that picture;
- fit a line to a small dataset by hand and interpret the slope as a quant would.
In this lesson
- The model
- Ordinary least squares
- Residual identities
- The projection geometry
Before you start
- Sample Covariance, Correlation, and Bivariate Dispersion
- Orthogonal Projections and Minimization Problems
- Least Squares as Orthogonal Projection
The full lesson, with worked examples and exercise solutions, is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or browse the Probability, Statistics and Financial Data syllabus.