Mixture of Gaussians and the EM Algorithm

k-means makes a hard decision at every step: each point belongs entirely to one cluster or another. The Gaussian mixture model (GMM) softens this into a genuinely probabilistic story — every point has some probability of having come from each of $k$ underlying Gaussian components — and the EM algorithm is exactly how such a model's parameters are fitted when the component membership is never directly observed. This note derives EM specifically for the Gaussian mixture case; the derivation sub-note generalises the same idea to any latent-variable model via Jensen's inequality.