Your backtest is lying to you: walk-forward analysis and the Deflated Sharpe Ratio in plain Python

Backtesting answers what would have happened, but investors care about what will happen. The gap between the two increases with every parameter tried. A simple daily trend-following rule was tested with real money (around ¥24,000). The best parameter combination had a Sharpe ratio of 1.46 on the last three years, while the fixed parameters scored 1.20. Two tools were presented to address this issue: walk-forward analysis and the Deflated Sharpe Ratio. Both fit in a page of Python code.

Walk-forward analysis involves scoring what the optimizer has not seen. By optimizing on a window (in-sample) and trading the winner on the next window (out-of-sample), the stitched out-of-sample curve is the only performance to believe. The stitched OOS Sharpe ratio on daily bars from 2018 to 2026 was 0.85, while the full-period backtest gave 1.11.

The fixed parameters performed as well as or better than the re-optimized ones in four out of six years. Re-optimizing added nothing, indicating the rule is not fragile and one should stop fiddling.

The Deflated Sharpe Ratio is another sanity metric. It accounts for the number of trials and the distribution of returns. By running enough random strategies, one will eventually find a brilliant-looking one by luck alone. Bailey and López de Prado's Deflated Sharpe Ratio calculates the Sharpe ratio one would obtain by luck alone given the number of trials and the statistical properties of the returns.