Skip to content
Probability, Statistics and Financial Data / Statistical Models and Data Reduction

Overview — Statistical Models, Exponential Families, and Sufficient Statistics

Topic 1 of this unit built the sampling distributions of Xˉ\bar X and S2S^2 under the assumption that the population distribution was fully known except for its parameters. This topic asks the question that makes those parameters estimable in the first place: given a family of candidate distributions {Pθ:θ∈Θ}\{P_\theta : \theta\in\Theta\} and observed data, how does the data speak about θ\theta? The answer is the likelihood function — the single object every estimation and testing procedure in the rest of this unit is built from — and this topic's central discovery is that, for an enormous class of models, the likelihood's entire dependence on the data collapses onto a low-dimensional summary statistic. That collapse is sufficiency, and it is the precise sense in which a risk system can discard a raw return history once it has retained a running mean and sum of squares, or a trading desk can discard individual trade outcomes once it has retained a win count.
This topic builds the conceptual chapter of the unit: it introduces parametric models and the likelihood, specialises to the algebraically rich class of exponential families that covers nearly every named distribution used in quantitative finance, proves the Neyman-Fisher factorization theorem that turns "is this statistic sufficient?" into pure algebra, sharpens sufficiency to minimal sufficiency and completeness (the ingredients behind Basu's theorem and, later, the Lehmann-Scheffé optimality theorem), and closes with Fisher information, which quantifies exactly how much a sample knows about θ\theta and sets the stage for the Cramér-Rao bound.

By the end of this topic you should be able to:

  • write down the likelihood and log-likelihood function for a parametric model, and distinguish the likelihood from a probability density in θ\theta;
  • recognise the canonical exponential-family form h(x)exp⁡(η(θ)T(x)−A(θ))h(x)\exp(\eta(\theta)T(x)-A(\theta)) and compute moments of the natural sufficient statistic from the log-partition function AA;
  • state and apply the Neyman-Fisher factorization theorem to find sufficient statistics for standard families, and reproduce its proof in the discrete case;
  • verify minimal sufficiency via the likelihood-ratio criterion, verify completeness directly for simple families, and apply Basu's theorem to prove independence results without distributional computation;
  • define Fisher information via the score function, compute it for exponential families, and state the likelihood principle.

In this lesson

  1. Contents
  2. The big picture

Before you start

The full lesson is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Probability, Statistics and Financial Data syllabus.