Skip to content
Linear Algebra, Optimisation and Numerical Methods / Introduction to the Singular Value Decomposition

Singular Value Decomposition: A = UΣVᵀ

Diagonalisation asked for an eigenvector basis of AA itself — impossible when AA isn't square, and unreliable even when it is (a defective matrix has no such basis). This note previews the fix: instead of one basis serving both the domain and codomain of AA, use two — an orthonormal basis VV for the domain and a different orthonormal basis UU for the codomain — chosen so that AA maps each domain basis vector to a scalar multiple of the matching codomain basis vector. This is the singular value decomposition, and it exists for every matrix, with no exceptions.
Learning objectives. By the end of this note you should be able to:
  • state what the SVD is and what UU, Σ\Sigma, VV each represent;
  • explain the geometric picture: orthogonal input directions mapping to orthogonal output directions;
  • connect the singular values to the eigenvalues of A⊤AA^\top A;
  • recognise why the SVD removes every restriction diagonalisation required.

In this lesson

  1. Motivation: why diagonalisation isn't enough
  2. The decomposition
  3. Connection to the eigenvalues of A⊤AA^\top A

Before you start

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Linear Algebra, Optimisation and Numerical Methods syllabus.