Boosting learning rate and tree count
Question
Why should learning rate and number of boosting rounds be tuned together?
Short answer
Each boosting update is multiplied by the learning rate, so smaller steps usually need more rounds to reach comparable fit. Their interaction affects both optimisation and regularisation. Tree depth and leaf constraints also change each update's capacity. Choose combinations or use early stopping inside development folds, then assess on outer periods. A smaller learning rate with enough rounds can still overfit.
Original question and answer written by SelfMadeQuant.com.
See the hint and the full worked solution in the question bank (free account).
Open the worked solutionLessons behind this question
- Decision Trees: Splitting Criteria and Pruning
- Boosting: AdaBoost and Gradient Boosting
- Counting and Combinatorics
Start with a free lesson: Probability Fundamentals is part of a free topic. Create a free account, no card needed, to read it in full with worked solutions.
Read it free