Skip to content
Linear Algebra, Optimisation and Numerical Methods / QR Factorisation and Least Squares

Least Squares as Orthogonal Projection

Every OLS regression a quant runs — a factor-model beta, a hedge ratio, a yield-curve fit — is an instance of the same geometric problem Theorem 1 of the projections note already solved: find the closest point in a subspace to a target vector. This note makes the connection explicit for the case that matters most in practice — an overdetermined system Ax=bA\mathbf{x}=\mathbf{b} with more rows than columns, generically unsolvable exactly — and derives the two standard computational routes, the normal equations and the QRQR-based solve.
Learning objectives. By the end of this note you should be able to:
  • set up a least-squares problem as min⁡x∥Ax−b∥\min_\mathbf{x}\lVert A\mathbf{x}-\mathbf{b}\rVert;
  • explain why the best-fit residual is orthogonal to C(A)C(A);
  • solve a least-squares problem via the normal equations and via QRQR;
  • state why the QRQR route is numerically preferred.

In this lesson

  1. The least-squares problem
  2. Two computational routes

Before you start

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Linear Algebra, Optimisation and Numerical Methods syllabus.