Skip to content
Probability, Statistics and Financial Data / Bayesian Estimation and Posterior Inference

Bayesian Shrinkage and Empirical Bayes

Every conjugate posterior mean computed in this topic — Beta-Binomial, Gamma-Poisson, Normal-Normal — turned out to be a weighted average of the prior mean and the MLE. That was flagged each time as a "reading" of the algebra; this note promotes it to the organising idea of the topic. The posterior mean is a shrinkage estimator: it pulls the MLE toward the prior mean, and the amount of pull is set entirely by the ratio of prior precision to data precision. This connects directly back to Bias, Variance, and Mean Squared Error's observation that a biased estimator can beat an unbiased one on MSE — shrinkage is why. The note then removes the one remaining piece of subjectivity in the whole machinery: where do the prior's own hyperparameters come from? Empirical Bayes answers by estimating them from the data itself, via a "type-II" marginal likelihood maximisation. It closes with the worked example every portfolio-construction desk eventually meets: the Black-Litterman model, which is nothing more than the Normal-Normal update of this topic applied to a vector of asset views.
Learning objectives. By the end of this note you should be able to:
  • write the posterior mean of any conjugate pair as w⋅θ^MLE+(1−w)⋅θ0w\cdot\hat\theta_{\text{MLE}} + (1-w)\cdot\theta_0 and identify ww as a function of prior and data precision;
  • connect Bayesian shrinkage to the bias-variance trade-off and the James-Stein phenomenon;
  • state the empirical Bayes idea — estimating hyperparameters by maximising the marginal likelihood m(x)m(x) — and carry it out for the Normal-Normal model;
  • derive the Black-Litterman posterior as a direct application of Normal-Normal conjugacy to a vector of views;
  • explain the practical trade-off between a fully subjective prior and an empirical Bayes prior.

In this lesson

  1. The posterior mean as a shrinkage estimator
  2. Empirical Bayes: estimating the prior from the data
  3. Black-Litterman as a Normal-Normal posterior

Before you start

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Probability, Statistics and Financial Data syllabus.