{
  "id": 15630633,
  "title": "Overfitting in trading strategies: what a million trials taught me",
  "url": "https://urgent.news/2026/10/19/overfitting-in-trading-strategies-what-a-million-trials-taught-me",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-19T17:30:28.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/bitcoinaipro/overfitting-in-trading-strategies-what-a-million-trials-taught-me-3pgc"
  },
  "original_language": "en",
  "account": "In a study of trading strategies, researchers found that running numerous configurations on historical data could lead to \"overfitting,\" where the best-performing model is merely a product of the search, not a genuine discovery. After conducting over one million trials on Bitcoin, only two model generations were deemed suitable for operation. The researchers emphasize that the rejection rate is a more reliable indicator than the most attractive backtest results.\n\nOne model's feature-importance profile revealed that a calendar-derived input was primarily responsible for its decision-making. However, this input was learned from the sample data rather than representing a consistent market state. When chronology intervened, the model's performance deteriorated.\n\nTo mitigate overfitting, the researchers recommend preferring stable regions over isolated peaks. They advise reproducing the exact model, inputs, and trade sequence before making any changes. Moreover, keeping the selection process and subsequent evidence separate is crucial to maintain independence. Retraining should be treated as model risk, as repeated refinements may introduce variance rather than improve adaptivity. It is also essential to audit execution separately from the signal.\n\nThe researchers highlight the difference between in-sample and out-of-sample testing. Using data from January 2018 to July 2022 for in-sample testing and July 2022 to August 2026 for out-of-sample testing, they achieved a simple return of +733.7 points, a 1:1 notional-equivalent research statistic, and a maximum drawdown of -31.04 points. However, they refrain from publishing specific details like the horizon, barrier calibration, threshold, hyperparameters, or trained model to prevent easy replication of the production model.\n\nFull write-ups and additional resources can be found at https://www.bitcoinai.pro/overfitting-trading-strategies/ and https://www.bitcoinai.pro/walk-forward-analysis-for-bitcoin-trading-a-complete-worked-example/. Remember, this research is not investment advice, and historical or simulated results do not guarantee future performance.",
  "summary": "Run enough configurations against the same history and the top row will eventually look exceptional, even if no durable edge exists. That's overfitting: mistaking the winner of a search for a discovery. I've now run 1,000,000+ machine-learning, execution and robustness trials on Bitcoin. Only two model generations ever left research for operation. The rejection rate tells you more than the…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}