Urgent.News

What's breaking now, across hundreds of outlets.

Tech

Do Synthetic Users Work? What the Research Found, and Where It Breaks

Synthetic users work for one job: ranking and stress-testing options you've already written, in categories the model knows well. The best study so far got AI consumers to about 90% of real panels' own consistency on purchase intent. They don't work as a stand-in for real buyers. They're too agreeable and too shallow, and they only predict what people say. That's the verdict from the primary…

Synthetic users are a form of artificial intelligence designed to mimic human responses, specifically in the context of ranking and stress-testing options that a model has previously written. A study conducted by PyMC Labs and Colgate-Palmolive, published in arXiv in 2025, tested GPT-4o and Gemini 2.0 synthetic users on 57 personal care concept surveys, comparing their results to those of 9,300 real respondents.

The findings indicate that synthetic users can achieve about 90% of the consistency in purchase intent as real panels. However, synthetic users are not reliable stand-ins for actual buyers, as they tend to provide agreeable and superficial responses, and they only predict what people say rather than what they would actually do. The research highlights that the performance of synthetic users depends on various factors, including the source of the persona information, the method used to collect answers, and the voice of the synthetic user.

While synthetic users can provide valuable insights and help fill data gaps, they should not replace human-centered research, as they may not accurately capture nuanced consumer behavior and preferences.

Rewritten by urgent.news; may contain errors — check the original.

Source: dev.to

Read the original at dev.to →

Report a problem with this story

More in Tech

Why your "free" dynamic QR code stopped working, and how to design one that won't

A QR code is a picture of some text. When that text is a URL, phones open it. That is the whole trick, and it explains most of the problems people run into with QR codes printed on menus, posters and…

  • "Free" dynamic QR codes stop working due to hidden billing rules.
  • QR codes have two types: static (final URL) and dynamic (short link redirect).
  • To avoid issues, separate billing from redirect path in dynamic codes.

More from Sunday 22 November →