The Perceptron and Multi-Class Classification

Logistic regression's sigmoid output is a smooth probability; the perceptron replaces it with a hard $\{0,1\}$ decision and a strikingly simple mistake-driven update rule — historically the first learning algorithm to be analysed rigorously, and the direct ancestor of the margin-based classifiers in Topic 5. This note also extends binary classification to $k > 2$ classes via softmax regression, needed the moment a quant classification problem has more than two outcomes (e.g. "upgrade / hold / downgrade" for a credit rating).