The Rao-Blackwell Theorem

Topic 2 of this unit built sufficient statistics on a qualitative promise: a sufficient statistic $T$ captures everything the sample says about $\theta$, so nothing is lost by reducing the data to $T$. The Rao-Blackwell theorem turns that promise into an exploitable, quantitative fact about estimator quality. Given any unbiased estimator — even a wasteful one that uses only a fraction of the data, like a single observation out of a sample of a thousand — conditioning it on a sufficient statistic produces a new estimator that is still unbiased and has variance (hence, by the bias-variance decomposition) mean squared error no larger, and generically strictly smaller. This is a free lunch: no new data, no new modelling assumptions, just a conditioning step that recovers the information a naive estimator left on the table.

This note states and proves the theorem via the law of total variance, works a fully explicit example — Rao-Blackwellising a naive single-observation estimator of a Uniform$(0,\theta)$ upper bound into exactly the estimator that met the Cramér-Rao bound analysis two lessons ago — and closes with the reason this theorem alone does not yet finish the search for the best unbiased estimator, motivating the Lehmann-Scheffé theorem next.

Learning objectives. By the end of this note you should be able to: