Ridge normal equations

For centred data, minimise the squared residual norm plus lambda times the squared coefficient norm. Derive the ridge equations and state when the solution is unique.

Differentiating gives twice X-transpose times (X beta minus y), plus twice lambda beta. Setting this to zero gives (X-transpose X + lambda I) beta = X-transpose y. For positive lambda the matrix is positive definite, even when X is rank deficient. Solve this system rather than explicitly computing its inverse. Centre data or fit an unpenalised intercept separately.