Why a Profitable Backtest Can Lose Money Live

Decompose the gap before changing the strategy. Compare the live system with a shadow run on the same days, starting from the same positions. If the signals differ, inspect data, timestamps and the deployed code. If signals match, reconcile submitted orders, fills, costs and exposures. If the two systems agree and both lose, investigate whether the backtest overestimated the edge or the live sample is consistent with the strategy's historical variability.

Reconcile the first place the two runs disagree

For each decision, record the code and model version, decision time, available data, signal, target weights, orders, fills and fees. Replay the same session and build a bridge from intended gross performance to realised net performance. Use consistent prices for marking the portfolio and include financing and borrow where applicable.

For a completed long trade of $q$ shares, start from gross P&L at the decision prices $P_{\text{entry}}^d$ and $P_{\text{exit}}^d$ . Actual P&L is

q(P_{\text{exit}}^d-P_{\text{entry}}^d) -q(P_{\text{entry}}^f-P_{\text{entry}}^d) -q(P_{\text{exit}}^d-P_{\text{exit}}^f) -\text{fees},

where $f$ denotes fill prices. For partial fills, calculate on actual sizes and separately measure the contribution from unfilled exposure. If orders were never submitted, inspect signal eligibility and execution constraints.

If costs and operations explain little, revisit the research design. Check how many strategy variants were tried, whether final test results influenced decisions, and whether performance concentrated in a few assets or market episodes. Compare the live drawdown with historical out-of-sample draws of similar length. With dependent returns, resample contiguous blocks or use a dependence-aware estimate. The sample can still be too short to distinguish regime change from variance.

Start with one live session you can reproduce end to end. Compare positions immediately before the first order, then compare the signal, target, submitted order, fill and accounting entry in that order. If the target differs, tuning the fill model will hide the disagreement. If the orders agree and the fills differ, mark both against the same arrival price to isolate execution. Only after you can reconcile the mechanics should you compare the gross signal on live dates with what its original research promised.

A backtest buys 1,000 shares at a decision mark of $50.00 and sells at $50.10. It reports $100 gross. Live, the entry fills at $50.06 and the exit at $50.07, giving $10 gross. Two fees totalling $15 yield a $5 loss. The bridge explains $60 of entry shortfall, $30 of exit shortfall and $15 of fees. You can now test whether observed execution costs systematically exceed assumptions, while keeping the signal itself fixed.

Suppose the replay also loses $5 after using the realised spreads and fills. That supports investigating the fill model. If the replay submits different orders, fix the replay or deployment discrepancy before interpreting the strategy's returns.

The order book tool makes the difference between a quoted price and the price paid for the whole order visible.

Reconcile the first place the two runs disagree

Find the execution cost that consumes a trading edge