A mean-squared-error gradient

Question

For loss equal to mean squared residual error, implement one gradient descent update for a finite X, coefficient vector beta and target y. State the factor of two.

Short answer

The gradient is 2 Xᵀ(X beta - y) / n. Omitting the factor changes the effective learning rate. The function below assumes a separately encoded intercept if needed.

Original question and answer written by SelfMadeQuant.com.

See the hint and the full worked solution in the question bank (free account).

Open the worked solution

Lessons behind this question

The Supervised Learning Problem and the ML Pipeline
Matrix Algebra: Operations, Rules, and the Transpose
Kernel, Image, and the Rank-Nullity Theorem

Start with a free lesson: Basic Statistics and Regression is part of a free topic. Create a free account, no card needed, to read it in full with worked solutions.

Read it free