Locally Weighted Regression and a First Look at Bias and Variance

Every lesson so far in this topic has fit one global linear model to the entire dataset. Locally weighted regression (LWR) breaks that assumption: instead of one $\theta$ for all $x$, it fits a different weighted least-squares problem for every query point, giving the hypothesis class the flexibility to follow curvature a single hyperplane cannot. That flexibility is a double-edged sword — it can fit the training data essentially perfectly while generalising worse than the simpler global model — which gives this topic its first, informal encounter with the bias-variance trade-off that Topic 8 treats formally.