Skip to content
Linear Algebra, Optimisation and Numerical Methods / Low-Rank Approximation, PCA, and Applications

Derivation — Proof of the Eckart-Young Theorem

The previous note stated the Eckart-Young theorem without proof. This sub-note carries the spectral-norm case in full — the Frobenius-norm case follows a similar argument, referenced but not repeated in full detail — using a clean two-part argument: an upper bound showing AkA_k achieves a specific error, and a matching lower bound showing no rank-kk matrix can do better.
Learning objectives. By the end of this note you should be able to:
  • prove the upper bound ∥A−Ak∥2=σk+1\lVert A-A_k\rVert_2 = \sigma_{k+1} directly;
  • prove the matching lower bound using a dimension-counting/pigeonhole argument;
  • explain why the two bounds together give the full theorem;
  • identify where the argument would break for a non-orthonormal approximation scheme.

In this lesson

  1. The upper bound
  2. The lower bound
  3. Assembling the theorem

Before you start

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Linear Algebra, Optimisation and Numerical Methods syllabus.