Skip to content
Probability, Statistics and Financial Data / Linear Regression and Analysis of Variance

Multiple Linear Regression

No real factor model stops at one regressor. A multi-factor risk model regresses returns on a market factor, a size factor, a value factor, and more, all at once — and every result of this topic so far (the closed-form estimator, its sampling distribution, the FF-test, R2R^2) needs to generalise from a line in the plane to a hyperplane in Rp\mathbb R^p. This note makes that jump using machinery already built: Least Squares as Orthogonal Projection already solved the general overdetermined least-squares problem, so most of this note is specialising that result to the regression setting rather than re-deriving it. What is genuinely new is what happens when several correlated predictors compete for the same explanatory job — multicollinearity — and the partial FF-test, which asks whether a specific subset of predictors earns its place once the others are already in the model.
Learning objectives. By the end of this note you should be able to:
  • write the multiple regression model in matrix form and state β^=(X⊤X)−1X⊤y\hat{\boldsymbol\beta}=(X^\top X)^{-1}X^\top\mathbf y as a direct application of the least-squares projection theorem;
  • derive Var(β^)=σ2(X⊤X)−1\text{Var}(\hat{\boldsymbol\beta})=\sigma^2(X^\top X)^{-1} and identify σ^2\hat\sigma^2's degrees of freedom;
  • construct the global FF-test for "does the model explain anything?" and the partial FF-test for "does this subset of predictors add anything, given the rest?";
  • define the variance inflation factor and diagnose multicollinearity from it;
  • read a real multi-predictor fit (the catheter-length example) and explain why individually insignificant coefficients can coexist with a highly significant model.

In this lesson

  1. The matrix formulation
  2. Testing the whole model: the global FF-test
  3. Testing a subset: the partial FF-test
  4. Multicollinearity and the variance inflation factor

Before you start

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Probability, Statistics and Financial Data syllabus.