Lagrange Duality and the SVM Dual Problem

The previous lesson reduced margin maximisation to a solvable convex quadratic program and could, in principle, stop there. This note takes a deliberate detour through Lagrange duality — reformulating the problem in terms of dual variables rather than solving the primal directly — because the resulting dual problem exposes two facts the primal hides entirely: the solution depends on the training data only through pairwise dot products, and only a small subset of training points (the support vectors) matter at all. Both facts are exploited immediately, the second here and the first in the kernel-trick lesson.