Variational Autoencoders and the Reparameterization Idea

A standard autoencoder maps each input to one code — useful for compression and anomaly screening, but awkward for sampling new plausible observations. A variational autoencoder (VAE) treats the code as a distribution: the encoder outputs parameters of $q_\phi(z\mid x)$, the decoder models $p_\theta(x\mid z)$, and training maximises a variational lower bound combining reconstruction with a penalty that shapes the latent space. The reparameterization trick makes sampling from that latent distribution differentiable, enabling end-to-end training with stochastic gradient methods.