Output Layers, Activation Functions, and Loss Functions

The previous lesson established that hidden layers need non-linear activations to make depth useful, but left open a more practical question: how should the output layer — the one layer whose job is to produce an actual prediction, not an intermediate representation — be designed for a given task? This note answers it as a small, systematic introductory recipe: the output layer's width, its activation function, and the training loss function are all determined jointly by whether the task is regression, binary classification, multilabel classification, or (exclusive) multiclass classification.