Overview

Topic 2 fit every model by unconstrained maximum likelihood — more features and more flexibility could only ever improve the training-set fit, which is exactly the overfitting trap flagged back in the bias–variance lesson. This topic supplies the missing discipline: penalise complexity directly (ridge, lasso, elastic net), interpret the penalty as a prior (MAP estimation, connecting cleanly to the GLM/exponential-family machinery), choose the penalty's strength honestly (cross-validation, AIC/BIC), and relax linearity in a controlled way (splines, local regression, GAMs) rather than jumping straight to an uninterpretable non-linear model.