Nested chronological model selection
Question
How would you compare ridge and boosting while tuning both without contaminating the reported performance?
Short answer
Use outer chronological evaluation windows to assess the complete selection procedure. Inside each outer training period, use earlier time-respecting folds to fit preprocessing and tune each model. Choose the model using only inner results, refit on permitted outer training observations, and evaluate on the outer test block. Respect label overlap and data availability at both levels. Report aggregate performance and variation across outer periods.
Original question and answer written by SelfMadeQuant.com.
See the hint and the full worked solution in the question bank (free account).
Open the worked solution