I simulated 2,999 buyers with three AI models. The one answering mattered more than the one that wrote them.

The reporter has constructed an open-source tool called MarketMirror, designed to test ads, prices, messages and sales pitches using simulated buyers before any money is invested in a real campaign. The tool culminates in a simple decision: go, iterate or no-go. Initially, the reporter questioned whether synthetic customers could truly provide meaningful insights, so they ran a market test with three different AI model providers simultaneously.

In this experiment, the advertised offer was a 7-day free trial with no card required, followed by a USD 4.99 monthly fee. The test included five distinct communities across India, the Philippines and Kenya, represented by 2,999 simulated buyers and 2,987 interviewed participants. Each AI model was assigned a specific audience and conducted interviews, with the answers then spoken by the model for each person throughout the process.

The individual personas generated by the models were diverse, with unique backgrounds, habits, and decision-making processes. The reporter utilized various tools to present the ads to the simulated buyers, gather responses through surveys, and analyze the results via pitch and rebuttal sessions. The analysis showed that while all three models expressed trust as the primary concern, their overall recommendations differed based on the specific tool used.

After receiving feedback from analysts, the reporter concluded that the models disagreed on the optimal course of action. The pitch reduced conversion rates, while the rebuttals had varying effects on the participants' decisions. Ultimately, the report highlighted that the model used to answer questions significantly influenced the outcomes of the simulated buyers' behavior.