Variational Inference and a Frontier Pointer to Deep Generative Models

EM's ELBO derivation solved a specific problem: fit point-estimate parameters $\theta$ under a latent variable $z$ with a tractable exact posterior $p(z\mid x;\theta)$. Variational inference removes both restrictions at once — it handles fully Bayesian models where every parameter is itself a random variable, and it works even when the true posterior is not tractable to compute exactly. The tool that makes this possible, the mean-field factorisation, closes out this topic and points forward to the machinery underneath variational autoencoders and the field of deep generative models generally.