Backtesting a Polymarket copy trading bot without fooling yourself

Polymarket copy trading bots did not prove profitable across five sports, as shown by a backtest conducted by the author. The core of the test involved replicating a wallet's first $20 purchase in a market at a price between 0.30 and 0.85, paying the sports taker fee, and holding the position until resolution.

To set up the backtest, the author outlined five steps. First, they defined a copy as a wallet's first $20 purchase in a market at a specific price range. The copy was implemented by buying the same outcome for $10 at the wallet's price plus 1 cent, paying the sports taker fee, and holding the position until the end of the month. The code for this calculation was provided in the source.

Second, the author selected a specific period for backtesting and measured the performance of wallets during the subsequent month. This was done by ranking wallets based on their copy results from the two months preceding the test month, then copying only in the test month. The selection period was fixed beforehand, and during the running of the test, the test month's data was excluded from the selection.

Third, the author checked whether a wallet's rank in one period carried over to the next month. They ran Spearman correlation tests for MLB, NFL, college football, League of Legends, and Valorant, finding that the correlation coefficients were close to zero in most cases. This suggests that past performance is not a reliable predictor of future performance for individual wallets.

Fourth, the author analyzed the baseline performance of wallets before any selection rule was applied. They found that even the largest whale wallets lost 31 to 63 cents per $10 in every test month. This baseline loss was significant, and any selection rule must still perform better than this baseline in order to be considered profitable.

Fifth, the author identified a potential trap in how wallet labels are computed. Analytics sites, including the author's own platform, label wallets based on factors like "smart money" status, number of resolved markets, win rates, and profit factors. However, these labels are calculated from the wallet's full history, including the months being backtested.

This means that wallets deemed "smart money" in hindsight may have already lost money in the test month and were not included in the backtest, leading to an inflated performance. The author advises that any feature used to select a wallet must be computable from data strictly before the pick date.

Lastly, the author urged readers to ensure the reproducibility of their backtest. They provided a script (copy_backtest.py) that runs the walk-forward backtest on the OrcaLayer API for any league. This script requires a Premium API key, but the author also noted that the free OrcaLayer MCP server can be used to verify the results with a standard API key.

In conclusion, the backtest results demonstrate that copy trading bots do not consistently generate profits across multiple sports when tested using strict methodologies. The author emphasizes the importance of avoiding common pitfalls in backtesting, such as using future data to compute wallet labels and relying on past performance to predict future results.