Derivation 2 of the LA-II normal-equations note flagged, without proof, that the Hessian being positive semi-definite was why the least-squares stationary point was genuinely a minimum. This note supplies the general theory: for any twice-differentiable multivariable function, the Hessian matrix's classification (via Theorem 1 of the quadratic-forms note) is exactly the second-order test that distinguishes a genuine local minimum from a maximum or a saddle point.
Learning objectives. By the end of this note you should be able to:
- state the second-order Taylor expansion of a multivariable function near a critical point;
- state the Hessian-based optimality test and connect it to quadratic-form classification;
- apply the test to a small worked optimisation problem;
- explain the inconclusive case (semi-definite Hessian).
In this lesson
- The Hessian and the second-order test
- The inconclusive case
Before you start
The full lesson, with worked examples and exercise solutions, is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or browse the Linear Algebra, Optimisation and Numerical Methods syllabus.