Skip to content

Two models on the same dates

Question

How would you assess whether one forecasting model improves on another when both predict the same return observations?

Short answer

Compare paired loss differences on common evaluation observations rather than treating model errors as independent samples. Examine their mean, economic magnitude and serial dependence; use an appropriate dependence-aware uncertainty estimate. Account for overlapping horizons and whether either model was selected using the same sample. Also inspect performance by period and asset group so a single mean does not hide concentrated failures.

Original question and answer written by SelfMadeQuant.com.

See the hint and the full worked solution in the question bank (free account).

Open the worked solution

Lessons behind this question

Start with a free lesson: Probability Fundamentals is part of a free topic. Create a free account, no card needed, to read it in full with worked solutions.

Read it free