Skip to content

Boosting learning rate and tree count

Question

Why should learning rate and number of boosting rounds be tuned together?

Short answer

Each boosting update is multiplied by the learning rate, so smaller steps usually need more rounds to reach comparable fit. Their interaction affects both optimisation and regularisation. Tree depth and leaf constraints also change each update's capacity. Choose combinations or use early stopping inside development folds, then assess on outer periods. A smaller learning rate with enough rounds can still overfit.

Original question and answer written by SelfMadeQuant.com.

See the hint and the full worked solution in the question bank (free account).

Open the worked solution

Lessons behind this question

Start with a free lesson: Probability Fundamentals is part of a free topic. Create a free account, no card needed, to read it in full with worked solutions.

Read it free