The previous lesson stated the Cramér-Rao inequality and sketched its proof as "a single application of Cauchy-Schwarz to the estimator and the score." This note supplies every step of that application in full: the covariance identity that links an estimator's bias-derivative to its covariance with the score, the information-equality substitution that turns a raw variance into , the exact point where the regularity condition (interchange of differentiation and integration) is used — twice — and the extension from a single observation to the i.i.d. sample corollary the parent lesson actually used throughout. Everything here is reproducible algebra; nothing is asserted without a line showing why.
Learning objectives. By the end of this note you should be able to:
- state the Cauchy-Schwarz inequality for two random variables and identify which two random variables the Cramér-Rao proof applies it to;
- derive from the regularity condition, showing exactly where differentiation is passed inside an integral;
- combine this with the information equality to complete the single-observation bound;
- extend the argument to the i.i.d. corollary by showing the cross-covariance terms between different observations' scores vanish.
In this lesson
- The two ingredients
- The single-observation proof
- Extending to the i.i.d. sample
Before you start
The full lesson, with worked examples and exercise solutions, is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or browse the Probability, Statistics and Financial Data syllabus.