Linear Regression via LMS and Gradient Descent

The empirical-risk-minimisation recipe from the previous note needs an optimiser to actually find $\theta$. This note builds the first one — gradient descent on the least-mean-squares (LMS) cost — and its stochastic variant, the workhorse optimiser reused, in one form or another, by every parametric model in this unit through backpropagation in Topic 6. It also introduces the running example — fitting a single-factor return model — carried through the rest of this topic.