A research Sharpe of for Model M is meaningless until it is compared with a simpler rule on the same universe, dates and cost model. This note decides what M must beat before it is allowed to replace a simpler rule, and records a challenger comparison — linking backtest evaluation and multiple testing without reteaching performance statistics from scratch.
Learning objectives. By the end of this note you should be able to:
- state a challenger that is nested in M's opportunity set;
- compare M and the challenger under identical costs and chronology;
- refuse an in-sample win as out-of-sample evidence;
- document the comparison as a validation input, not a marketing slide.
In this lesson
- The decision: beat what, on which terms?
- Limitation: in-sample challenger wins are not OOS evidence
- What "must beat" means for a trial gate
- Worked decision: replace C or not?
- Desk checkpoint: challenger table required
- Pitfall: moving the goalposts
- Decision card: is the gap real enough to trial?
Before you start
The full lesson, with worked examples and exercise solutions, is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or browse the Professional Quant Practice syllabus.