Overlapping windows will manufacture statistical significance in your backtest
The easiest way to publish a false positive in time-series work, with a worked example from my own project. The result that looked publishable A walk-forward test, 913 causal anchors, direction accuracy at a 30-minute horizon. Benchmark note that matters: the thing to beat is not 50%, it is the majority class . If 51% of moves were up, "always guess up" scores 51%. Scoring against 50% is how a…
A walk-forward test with 913 causal anchors and a 30-minute forecast horizon produced a misleading 55.2% accuracy result, nearly doubling from the actual 51.1% majority class. This occurred due to overlapping windows, where test points shared 23 out of their 30 minutes of outcome. The inflated sample size of 912 was actually a smaller number of distinct market episodes counted repeatedly, leading to artificially small error bars.
Re-testing on non-overlapping anchors reduced the sample size to 163, increasing the edge to +5.5 pp with smaller standard errors. The key trap is that larger effect sizes can be misleading, making noise appear as a stronger real effect. To avoid this, compare anchor spacing to the forecast horizon and either re-score on non-overlapping anchors or space them a full horizon apart.
This issue applies to any sliding-window evaluation where the window is wider than the stride, including sensor data, load forecasting, anomaly detection, and medical monitoring.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
