Topic 2 built the machinery — the likelihood, sufficiency, and Fisher information — without ever using it to produce an actual number for . This topic closes that gap. It is the richest topic in the unit: it builds two competing families of estimator (method of moments and maximum likelihood), proves a hard variance floor every unbiased estimator must respect (the Cramér-Rao bound), and then shows how to reach that floor exactly — first by improving any unbiased estimator via conditioning on a sufficient statistic (Rao-Blackwell), then by proving the improved estimator is not just better but optimal whenever the sufficient statistic is also complete (Lehmann-Scheffé). It closes by showing that even when no estimator attains the bound exactly in finite samples, the maximum likelihood estimator gets there asymptotically, as the sample size grows.
Every quantitative model calibration a desk runs — fitting a return distribution, a volatility process, a default intensity, a factor loading — is an instance of the estimation theory built here. This topic is also where a first real complication enters: not every calibration problem has a closed-form likelihood, and the EM algorithm supplies the standard iterative fix for models with a hidden or latent structure, such as a regime-switching return model.
By the end of this topic you should be able to:
- construct method-of-moments and maximum likelihood estimators for standard parametric families, and know when each is preferable;
- run the EM algorithm on an incomplete-data model and verify its monotone likelihood-ascent property;
- state and apply the Cramér-Rao lower bound, and reproduce its Cauchy-Schwarz proof;
- decompose mean squared error into bias and variance, and explain when a biased estimator can beat every unbiased one;
- apply the Rao-Blackwell theorem to improve an estimator by conditioning on a sufficient statistic, and the Lehmann-Scheffé theorem to certify a UMVUE;
- state the three large-sample properties of the MLE — consistency, asymptotic normality, and asymptotic efficiency — and apply the delta method to a transformed estimator.
In this lesson
- Contents
- The big picture
Before you start
- Parametric Statistical Models and the Likelihood
- Sufficient Statistics and the Neyman-Fisher Factorization Theorem
- Fisher Information and the Likelihood Principle
The full lesson is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or browse the Probability, Statistics and Financial Data syllabus.