The Double-Descent Phenomenon

The bias-variance decomposition predicts a single U-shaped test-error curve: increasing flexibility past some point should always hurt, as variance growth overwhelms further bias reduction. In certain specific settings — most strikingly, modern overparameterised neural networks — this is not the whole story: test error can rise sharply at the point where the model first achieves zero training error, then descend again as flexibility increases further still. This note works through the phenomenon in a fully worked, reproducible one-dimensional example, and — the crucial caveat — explains precisely why it does not overturn anything derived in the previous two lessons.