Bias, Variance, and Mean Squared Error

The Cramér-Rao lower bound set a variance floor — but only for unbiased estimators, and the MLE lesson already flagged a loose end: $\hat\sigma^2_{\text{MLE}}$ divides by $n$, not $n-1$, and is therefore biased. Is that a defect to be corrected, or could a biased estimator ever be genuinely better than an unbiased one? This note answers with a precise criterion — mean squared error (MSE) — that scores bias and variance on the same footing, shows the $\hat\sigma^2_{\text{MLE}}$-versus-$S^2$ comparison is a real instance where biased wins, and then goes further: it introduces the James-Stein phenomenon, one of the most startling results in classical statistics, where an entire class of textbook-optimal unbiased estimators turns out to be dominated the moment three or more parameters are estimated jointly. This is the direct mathematical ancestor of every shrinkage estimator a quant desk uses to tame an overfit sample covariance matrix or a noisy vector of expected returns.

Learning objectives. By the end of this note you should be able to: