Regularization as MAP Estimation

The previous lesson introduced ridge and lasso as penalised least-squares problems — a purely optimisation-flavoured story. This note supplies the probabilistic story underneath, using the same MLE machinery from the probabilistic-interpretation lesson: both penalties turn out to be exactly what maximum a posteriori (MAP) estimation produces under a specific choice of prior on $\theta$. This reframing is not decorative — it explains why the $\ell_2$ penalty shrinks smoothly while the $\ell_1$ penalty produces exact zeros, in terms of the shape of a probability density rather than the shape of a constraint region.