The Probabilistic Interpretation of Least Squares

Squared-error loss was introduced in the LMS note without justifying why squared rather than absolute, quartic, or any other error measure. This note supplies the missing justification: under one natural noise assumption, minimising squared error is exactly maximum likelihood estimation (MLE). This is the template Topic 2 reuses for logistic regression's cross-entropy loss and Topic 3 for regularisation-as-MAP — every loss function in this unit turns out to be MLE (or MAP) under some probabilistic model of the data, and this note is the first instance.