Skip to content

Why Deep Learning Can Underperform Simpler Return Models

Original question

Why do deep learning models often struggle to outperform simpler statistical models when predicting financial asset returns?
Financial return signals are often weak, observations are dependent, and the relationship between inputs and future returns can change. A flexible neural network can fit patterns that happen to recur inside a historical sample. A simpler model can estimate a useful average relationship with fewer observations and less sensitivity to a particular period. Deep learning can add value when the data and task support the additional flexibility.

Make the network earn its extra flexibility

Compare three classes using the same features and chronological split: a simple reference such as zero or a historical mean, a regularised linear model, and the network. Use early stopping and hyperparameter selection only on development folds. Record errors by period and date, along with the performance of a fixed position-construction rule. Vary one source of complexity at a time, such as depth, input history or number of features.

A standard diagnostic is out-of-sample squared-error improvement relative to a reference forecast:

ROOS2=1−∑t(yt−y^t)2∑t(yt−y^tref)2.R^2_{\mathrm{OOS}} =1-\frac{\sum_t(y_t-\widehat y_t)^2} {\sum_t(y_t-\widehat y^{\mathrm{ref}}_t)^2}.
Negative values mean higher squared error than the reference over those observations. Compare repeated seeds, test periods and training-window lengths. A performance gain that appears only for one seed may reflect optimisation variability, while gains limited to a single regime warrant a narrower hypothesis.

Count independent episodes, not just rows. Ten thousand stocks observed during one crash give many cross-sectional observations and only one realisation of that market event. A model can learn to recognise a pattern tied to that episode without seeing enough independent examples to generalise. Compare the network to a regularised linear model on the same eras and with a comparable selection budget. If an advantage disappears when you hold out an entire regime, use that failure to investigate the feature or mechanism before adding more layers.

Suppose a ridge model produces 1.0% out-of-sample R2R^2 against zero. A neural network produces 1.5% using the same inputs and time splits. Inspection shows the network's gain came primarily from one stress period. Its forecast-driven portfolio also trades twice as much, so net performance is weaker at the assumed costs. Those findings suggest investigating whether there is a repeatable stress-period interaction and whether a simpler turnover-aware model can capture it.

Another task might offer longer sequences of order-book events or structured text where a network can exploit input relationships a small linear model lacks. In either case, compare the model with a strong baseline using the same information. If the deep model uses better inputs, test those inputs with the simpler model as well.

Give the two model classes a comparable opportunity to tune. A network selected from dozens of architectures and stopping rules can have a larger effective search budget than one fixed ridge penalty. The observed difference in their best validation scores then includes both architecture and selection. Freeze the inputs, comparison metric, retraining schedule and a final chronological test. A network that wins again on the final dates has a stronger case than one that simply had more chances to win on development data.