The Poisson-Gamma calculation in Prior, Likelihood, and Posterior had a conspicuously clean outcome: a Gamma prior, multiplied by a Poisson likelihood, produced another Gamma density — updating meant nothing more than adding two numbers to the prior's parameters. That was not luck. It is a systematic algebraic fact about pairing a prior with a likelihood from the same exponential family, and it is the single reason closed-form Bayesian updating is even possible without numerical integration on every trading desk that runs one. A conjugate prior family is a class of priors closed under Bayesian updating: start with a prior in the family, observe data from the matching likelihood, and the posterior lands back in the same family, with only the parameters changed.
This note derives conjugacy for the three workhorse pairs — Beta-Binomial, Gamma-Poisson, and Normal-Normal — in full, then proves the general exponential-family conjugacy result that explains why these three (and every other exponential-family likelihood) all behave the same way.
Learning objectives. By the end of this note you should be able to:
- state the definition of a conjugate prior family and recognise the update-the-parameters pattern in a specific calculation;
- derive the Beta-Binomial posterior update and read the posterior mean as a weighted average of the prior mean and the sample proportion;
- derive the Gamma-Poisson posterior update (completing the Poisson-Gamma calculation from the previous lesson for a general sample);
- derive the Normal-Normal posterior update for an unknown mean with known variance, and state the update in precision form;
- state and prove the general theorem that every one-parameter exponential family has a conjugate prior built from its own pair.
In this lesson
- Definition and the general pattern
- The Beta-Binomial pair
- The Gamma-Poisson pair
- The Normal-Normal pair
- The general exponential-family conjugate prior
Before you start
The full lesson, with worked examples and exercise solutions, is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or browse the Probability, Statistics and Financial Data syllabus.