The Bernstein-von Mises Theorem

Posterior Summaries drew a sharp philosophical line between a Bayesian credible interval and a frequentist confidence interval — one is a genuine probability statement about $\theta$, the other is a coverage-frequency guarantee about the interval — and warned that the two can diverge sharply for informative priors or small samples. Yet every worked example with a diffuse prior in this topic — Prior, Likelihood, and Posterior's flat-prior Normal posterior, Bayesian Shrinkage's shrinkage weight $w_n\to1$ — landed numerically on exactly the MLE's own sampling distribution. The Bernstein-von Mises (BvM) theorem proves this is not a string of coincidences: for a broad class of models, as $n\to\infty$, the posterior distribution of $\sqrt n(\theta-\hat\theta_{\text{MLE}})$ converges to $\mathcal N(0, I(\theta_0)^{-1})$ — the same limit Asymptotic Properties of the MLE proved for the MLE's own sampling distribution — for almost any proper prior, not just a flat one. This note states the theorem, proves it via a Taylor expansion of the log-posterior that directly parallels the MLE asymptotic-normality proof, draws out the corollary that Bayesian credible intervals become asymptotically valid frequentist confidence intervals, and closes by naming precisely where the theorem breaks down.

Learning objectives. By the end of this note you should be able to: