k-Nearest Neighbours and the Curse of Dimensionality

Every model so far — even the flexible splines and local regression of Topic 3 — has fitted a set of parameters and thrown the training data away once fitting is done. $k$-nearest neighbours (KNN) does the opposite: it keeps the entire training set and, at prediction time, simply asks "what happened near here before?" This note develops that idea and its central limitation — the curse of dimensionality — which explains why a technique so intuitive in low dimensions can fail badly the moment a feature set grows, a genuine risk for any quant model with dozens of candidate factors.