I'm about to ask ChatGPT, Perplexity and Gemini the same five product questions. Tear my method apart first.
Everyone has a screenshot of what an AI said about their product. Almost nobody has a method. Before I run mine, here's the protocol, so you can tell me where it's broken. This post has no results. That's deliberate. If I publish the method first, I can't quietly adjust it after seeing numbers I don't like. Think of it as a pre-registration, the way good studies do it, just much smaller. Why…
The author plans to conduct an experiment comparing ChatGPT, Perplexity, and Gemini AI assistants by asking them the same five product-related questions. The purpose is to analyze the sources cited in their responses, rather than simply checking if the AI mentions the product. The protocol involves running each question through each AI five times, with each run logged with precise timestamps and settings.
Factors that can affect the answers, such as personalized responses, model changes, and user screenshots of successful interactions, are acknowledged as limitations. By recording the specific sentences lifted from sources, the author aims to identify patterns in which types of pages are more frequently cited. The experiment will focus on five question types that represent different page formats: listicles, comparison posts, documentation, definitions, and research summaries.
The goal is to determine which page formats are preferred by the AIs for providing definitions or explanations. The author plans to publish the results alongside the raw data and the exact methodology, in order to ensure transparency and accountability in the analysis.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
