Skip to content

Permutation importance with correlated factors

Question

Two factors contain almost the same information. Why can single-feature permutation importance underestimate their joint usefulness?

Short answer

Permuting one factor leaves its correlated substitute intact, so model performance may change little even if the pair is important. Independent permutation can also create unrealistic combinations. Consider grouped permutation, conditional methods with explicit assumptions, and refit-based ablations. Measure importance on held-out periods with dependence-aware uncertainty. Importance describes a fitted model and evaluation distribution, not an invariant causal contribution.

Original question and answer written by SelfMadeQuant.com.

See the hint and the full worked solution in the question bank (free account).

Open the worked solution

Lessons behind this question

Start with a free lesson: Probability Fundamentals is part of a free topic. Create a free account, no card needed, to read it in full with worked solutions.

Read it free