Skip to content
Financial Machine Learning / Unsupervised Learning and Latent Structure

Derivation: Deriving EM via Jensen's Inequality and the ELBO

The previous lesson presented EM's E-step and M-step for the Gaussian mixture specifically, justified only by the intuitive "guess zz, then do easy MLE" story. This note derives the general EM algorithm — applicable to any latent-variable model p(x,z;θ)p(x,z;\theta), not just Gaussian mixtures — via Jensen's inequality, and proves the algorithm's central theoretical guarantee: the log-likelihood increases monotonically at every iteration.

In this lesson

  1. Learning objectives
  2. Jensen's inequality
  3. The evidence lower bound (ELBO)
  4. The E-step: choosing QiQ_i to make the bound tight
  5. The general EM algorithm
  6. Proving monotone convergence
  7. Worked numerical check

Before you start

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Financial Machine Learning syllabus.