Skip to content

Random forests and correlated trees

Question

Why does reducing correlation among trees help a random forest, even if individual trees become slightly weaker?

Short answer

The variance of an average depends on both individual variances and pairwise covariances. Averaging highly correlated errors yields limited variance reduction. Feature subsampling and bootstrap variation can reduce correlation, potentially improving the ensemble despite weaker components. The benefit is empirical and does not remove bias or distribution shift. Tune feature sampling and leaf controls using the intended evaluation procedure.

Original question and answer written by SelfMadeQuant.com.

See the hint and the full worked solution in the question bank (free account).

Open the worked solution

Lessons behind this question

Start with a free lesson: Expectation, Variance and Covariance is part of a free topic. Create a free account, no card needed, to read it in full with worked solutions.

Read it free