Averaging errors across folds
Question
Your time-series folds contain very different numbers of observations. When do equal-weighted and observation-weighted mean losses answer different questions?
Short answer
Equal fold weights estimate an average across the chosen periods, giving small and large periods equal influence. Observation weights estimate average loss across pooled observations. Neither necessarily matches a portfolio-level objective when dates have different asset counts or capital allocations. State the unit of evaluation, aggregate accordingly, and inspect period-level results. Do not pool nonlinear metrics by averaging their component scores without checking the intended definition.
Original question and answer written by SelfMadeQuant.com.
See the hint and the full worked solution in the question bank (free account).
Open the worked solution