Decision Trees: Splitting Criteria and Pruning

$k$NN predicts from raw distance with no interpretable structure at all. A decision tree takes the opposite non-parametric strategy: it partitions feature space with a sequence of simple, human-readable if/then rules — "if years in the league $<4.5$, predict a low salary; otherwise, split further on hits" — producing a model a domain expert can read and sanity-check directly, at the cost of the high variance this note's final section confronts head-on.