Skip to content
Linear Algebra, Optimisation and Numerical Methods / Linear systems and least squares

Least Squares and Regression Geometry

Factor loadings, hedge ratios, and yield-curve fits are almost never exact solutions of Ax=bA\mathbf{x}=\mathbf{b}: there are more observations than free parameters, and the target sits outside the column space. The desk question becomes geometric — find the closest attainable vector in C(A)C(A) — and that is least squares as orthogonal projection. This note derives the normal equations from the residual-orthogonality condition, reads OLS as regression geometry, and lands on one finance regression.
Learning objectives. By the end of this note you should be able to:
  • state the least-squares problem min⁡x∥Ax−b∥\min_{\mathbf{x}}\lVert A\mathbf{x}-\mathbf{b}\rVert and its projection characterisation;
  • derive the normal equations from residual orthogonality to C(A)C(A);
  • verify that fitted residuals are orthogonal to every regressor column;
  • set up and interpret a one-factor (or two-factor) return regression.

In this lesson

  1. When Ax=bA\mathbf{x}=\mathbf{b} has no exact solution
  2. Computational routes (statement level)
  3. One finance regression reading

Before you start

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Linear Algebra, Optimisation and Numerical Methods syllabus.