Gaussian Processes for Regression and Classification

The margin-methods topic's closing lesson previewed Gaussian processes (GPs) as "the Bayesian non-parametric sibling" of kernel ridge regression, promising full derivation here. This note delivers it: a GP places a prior directly over functions rather than over a finite parameter vector, and the predictive equations that result — derived entirely from the properties of the multivariate Gaussian distribution — give both a point prediction and a calibrated uncertainty estimate at every input, something a plain kernel ridge regression fit does not natively provide.