The conjugate-prior calculations of the previous lesson all quietly reported "the posterior mean" as if it were the obvious single number to hand a desk head — but the posterior is an entire distribution, and a distribution has a mean, a mode, a median, and a whole family of intervals, not one canonical summary. Which one to report is not a matter of taste: it is the answer to a precisely posed decision problem. This note makes that problem explicit through loss functions and Bayes risk, proves that the posterior mean, median, and mode are each optimal under a different loss, and then builds the Bayesian analogue of a confidence interval — the credible interval, and its tightest form, the highest posterior density (HPD) region — closing with the sharp philosophical distinction between what a credible interval and a frequentist confidence interval each actually claim.
Learning objectives. By the end of this note you should be able to:
- define a loss function, the risk function, and the Bayes risk, and state the general result that a Bayes estimator minimises posterior expected loss pointwise;
- prove that the posterior mean minimises Bayes risk under squared error loss, and that the posterior median and mode do the same under absolute error loss and - loss respectively;
- construct an equal-tailed credible interval from a posterior CDF and compute one for a Beta or Normal posterior;
- define a highest posterior density (HPD) region and construct one for a symmetric unimodal posterior;
- state precisely how the Bayesian interpretation of a credible interval differs from the frequentist interpretation of a confidence interval, and identify the numerical case in which the two nearly coincide.
In this lesson
- Loss functions and Bayes risk
- The posterior mean minimises squared error loss
- The posterior median and mode
- Credible intervals
- Bayesian credible intervals vs frequentist confidence intervals
Before you start
The full lesson, with worked examples and exercise solutions, is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or browse the Probability, Statistics and Financial Data syllabus.