Principal Component Analysis and the SVD View

Mixture models and EM discover discrete latent structure — which of $k$ groups a point belongs to. Principal component analysis (PCA) discovers continuous latent structure: the low-dimensional subspace along which high-dimensional data actually varies. Two nearly-collinear features — a car's top speed in mph and the same top speed in kph, or an RC-helicopter pilot's measured "skill" and "enjoyment" scores, which tend to move together — are really telling you about one underlying quantity, not two independent ones. PCA finds that one (or few) underlying quantity directly, via nothing more exotic than an eigenvector calculation — no EM iteration required.