Generative Classifiers: Gaussian Discriminant Analysis and Naive Bayes

Every classifier so far — logistic regression, the perceptron, softmax — has been discriminative: it models $p(y\mid x)$ directly, drawing a decision boundary and ignoring how $x$ itself is distributed within each class. This note introduces the opposite philosophy: generative classifiers model $p(x\mid y)$ and $p(y)$ separately, then invoke Bayes' rule to get $p(y\mid x)$. Understanding when to reach for which philosophy — a decision that recurs for naive-Bayes-style credit models and text-based sentiment classifiers alike — is the payoff of this note.