Derivation — The Normal Equations via Matrix Calculus

The parent note stated $\nabla_\theta J(\theta) = X^\top X\theta - X^\top y$ without proof. This note supplies the matrix-derivative machinery — the trace trick — needed to differentiate a matrix expression like $\|X\theta - y\|^2$ with respect to a vector $\theta$, the same technique reused later for ridge regression's closed form and the PCA maximum-variance derivation.