Skip to content
Linear Algebra, Optimisation and Numerical Methods / Low-Rank Approximation, PCA, and Applications

The Moore-Penrose Pseudoinverse

Exercise 2 of the LA-II normal-equations derivation note flagged that the normal equations break down when AA's columns are dependent — A⊤AA^\top A becomes singular, and the least-squares coefficients (though not the fitted values) become non-unique. This note resolves that gap completely: the Moore-Penrose pseudoinverse A+A^+, built directly from the SVD, generalises A−1A^{-1} to every matrix — square or not, full-rank or not — and, among the infinitely many valid least-squares coefficient vectors when AA is rank-deficient, picks out the one with minimum norm.
Learning objectives. By the end of this note you should be able to:
  • construct the pseudoinverse from the SVD;
  • verify the pseudoinverse reduces to A−1A^{-1} when AA is square and invertible;
  • use the pseudoinverse to solve a rank-deficient least-squares problem;
  • explain the minimum-norm property.

In this lesson

  1. Construction
  2. Solving least squares via the pseudoinverse

Before you start

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Linear Algebra, Optimisation and Numerical Methods syllabus.