Impurity importance favours flexible splits
Question
Why might a tree give high impurity-based importance to a continuous noise feature compared with a binary noise feature?
Short answer
A continuous feature often offers many candidate thresholds, increasing opportunities to find a chance impurity decrease in training. A binary feature has fewer alternatives. Prefer held-out evaluation, permutation or refit-based comparisons, while accounting for correlated predictors and temporal dependence. High training impurity importance does not by itself establish predictive information.
Original question and answer written by SelfMadeQuant.com.
See the hint and the full worked solution in the question bank (free account).
Open the worked solution