The Normal Equations: Closed-Form Least Squares

Gradient descent from the previous note finds the minimiser of the LMS cost by iterating; for linear regression specifically, the minimiser also has a closed form — a direct formula requiring no learning rate, no convergence check, and no iteration at all. This note derives that formula, the normal equations, states when to prefer it over iterative descent, and hands the heavy matrix-calculus algebra to a sibling derivation note so this note can stay focused on the result and its use.