Permutation importance with correlated factors
Question
Two factors contain almost the same information. Why can single-feature permutation importance underestimate their joint usefulness?
Short answer
Permuting one factor leaves its correlated substitute intact, so model performance may change little even if the pair is important. Independent permutation can also create unrealistic combinations. Consider grouped permutation, conditional methods with explicit assumptions, and refit-based ablations. Measure importance on held-out periods with dependence-aware uncertainty. Importance describes a fitted model and evaluation distribution, not an invariant causal contribution.
Original question and answer written by SelfMadeQuant.com.
See the hint and the full worked solution in the question bank (free account).
Open the worked solutionLessons behind this question
- Decision Trees: Splitting Criteria and Pruning
- Boosting: AdaBoost and Gradient Boosting
- Counting and Combinatorics
Start with a free lesson: Probability Fundamentals is part of a free topic. Create a free account, no card needed, to read it in full with worked solutions.
Read it free