Overview

Every model built later in this unit — a random forest scoring credit defaults, a neural network reading limit-order-book state, a Gaussian process fitting an implied-volatility smile — is an instance of the same four-part recipe: pick a hypothesis class, define a loss, run an optimiser, and check that the fitted model generalises to data it has not seen. This topic introduces that recipe by building the simplest non-trivial instance of it end to end: linear regression, fit by gradient descent and by the closed-form normal equations, with a probabilistic interpretation that explains why squared error is the natural loss for Gaussian-noise data. Every later topic in this unit — logistic regression, regularised regression, kernel methods, even backpropagation — is a variation on the machinery built here. The topic closes with locally weighted regression, which gives the unit's first informal look at the bias-variance trade-off that Topic 8 treats formally.