Kernel Ridge Regression and a Preview of Gaussian Processes

The kernel trick lesson stated the recipe generically: any algorithm expressible purely in inner products can be kernelized. This note cashes that promise in for regression, showing that ridge regression — a seemingly unrelated algorithm from a much earlier topic — has exactly this dot-product structure once its solution is rewritten in dual form, yielding kernel ridge regression. The note then previews Gaussian processes, a fully Bayesian, distribution-over-functions perspective that recovers the same predictive mean as kernel ridge regression while additionally quantifying predictive uncertainty — a genuine upgrade this topic can only sketch, deferring the full machinery to Topic 9.