Skip to content

Why Random Splits Break Financial Time-Series Validation

Original question

Can you explain why random train-test splits are generally inappropriate for financial time series prediction problems?
A random split lets the training set see dates that occur after the test dates. That changes the task from forecasting the future to reconstructing a date with knowledge of later markets. If the targets use overlapping return windows, randomly assigned rows can also share realised returns across training and test. Use chronological splits and enforce the time at which each training label becomes known.

Count the outcomes that neighbouring labels share

For five-day log return labels yt=∑j=15rt+jy_t=\sum_{j=1}^5 r_{t+j}, adjacent outcomes share four daily returns. Under independent daily returns with variance σ2\sigma^2,
Cov⁡(yt,yt+1)=4σ2,Corr⁡(yt,yt+1)=45.\operatorname{Cov}(y_t,y_{t+1})=4\sigma^2, \qquad \operatorname{Corr}(y_t,y_{t+1})=\frac45.

The labels are strongly related even when daily returns themselves are independent. A random split frequently places such labels on opposite sides of a fold.

The covariance calculation isolates overlap even in a deliberately simple market with independent daily shocks. The shared days enter both label sums with coefficient one. All cross-products involving different days have expectation zero after centring, so only those four shared daily variances remain. A random split can put outcomes that share the same price moves on either side of the boundary. It can also fit on future regimes when labels do not overlap at all. Forward validation addresses ordering, and the cutoff on known labels addresses overlap.

With 252 sessions, use successive 21-session test blocks. At each boundary, fit on earlier observations whose five-day labels are known. If labels correspond to the next five closes and the first test decision is after close 106, a fit frozen by close 105 may include row 100, whose label ends at close 105. It may not include row 101, whose label ends at close 106. In a real calendar, the comparison should use timestamps and the intended cutoff, not an estimate of the number of rows.

It helps to write the dates next to the actual label sums. If the test target contains Thursday's and Friday's returns, a training row whose outcome also uses those returns is sharing observations with the test set even when the feature dates differ. A forward split with an appropriate gap excludes training labels that have not fully matured at the first test decision. A gap defined in row counts can be misleading for an irregular panel with multiple assets or missing dates. Check actual start and end timestamps there.

Use time-ordered folds with a gap

Here the latest included training index plus five is strictly earlier than the first test index.

import numpy as np from sklearn.model_selection import TimeSeriesSplit sessions = np.arange(252) splitter = TimeSeriesSplit(n_splits=5, test_size=21, gap=5) for train, test in splitter.split(sessions): assert train[-1] + 5 < test[0] print(f"train {train[0]}–{train[-1]}, test {test[0]}–{test[-1]}")

References and next steps