Fisher Information and the Cramér-Rao Lower Bound

Two lessons ago, the Method of Moments and Maximum Likelihood Estimation lessons both produced consistent estimators of the same parameters — the Poisson $\lambda$, the Bernoulli $p$ — and left an open question hanging: given two consistent, unbiased estimators, is one provably better than the other, or just different? The Fisher Information and the Likelihood Principle lesson supplied the tool that finally answers this — Fisher information $I(\theta)$, the expected curvature of the log-likelihood — and this note turns it into a hard number: the Cramér-Rao lower bound (CRLB), a variance floor that every unbiased estimator must respect. No unbiased estimator can beat it, and an estimator that attains it is, in the precise sense this note defines, unimprovable.

This note states the general Cramér-Rao inequality (for possibly biased estimators of a function of $\theta$), specialises it to the unbiased i.i.d. case using the additivity fact from the Fisher-information lesson, defines efficiency, works the Poisson and muon-decay examples through to a concrete efficiency comparison between MOM and MLE, states the multiparameter extension, and closes with the classical counterexample that shows exactly what breaks when the bound's regularity conditions fail. The next lesson supplies the full Cauchy-Schwarz proof this note only sketches.

Learning objectives. By the end of this note you should be able to: