Hundreds of attempted signals
Question
You tested 300 signal variants and report the strongest out-of-sample Sharpe ratio. What information is missing from that claim?
Short answer
Selection among many variants inflates the apparent quality of the winner, especially when the same evaluation period guided selection. Record the full search process, correlations among trials and all evaluation decisions. Use an independent final assessment of the selection procedure and suitable multiple-testing or selection-aware analysis where assumptions permit. The raw number of trials alone is not an exact effective trial count.
Original question and answer written by SelfMadeQuant.com.
See the hint and the full worked solution in the question bank (free account).
Open the worked solutionLessons behind this question
Start with a free lesson: Expectation, Variance and Covariance is part of a free topic. Create a free account, no card needed, to read it in full with worked solutions.
Read it free