{
  "id": 6017432,
  "title": "I pre-registered a study on AI visibility signals. The main result was null.",
  "url": "https://urgent.news/2026/09/06/i-pre-registered-a-study-on-ai-visibility-signals-the-main-result-was",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-06T18:55:47.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/angeo/i-pre-registered-a-study-on-ai-visibility-signals-the-main-result-was-null-1bd3"
  },
  "original_language": "en",
  "account": "A researcher has pre-registered a study examining whether businesses that repeatedly mention AI assistants differ from those that mention them only once, based on observable technical signals. Four signals were considered: crawler access, serving llms.txt, having a product page with JSON-LD structured data, and offering product availability data. The study used data from 458 product-level home-decor purchasing questions across three AI assistants (ChatGPT, Gemini, and Perplexity). Blinding was employed, ensuring the researcher did not see the store list or the analysis frame until the plan was sealed. The analysis found no signal to separate the two groups by more than 15 points. Specifically, llms.txt was more common among businesses mentioned once rather than repeatedly, and this difference was found to be consistent across all five cuts of the data. However, the apparent difference was attributed more to boilerplate content rather than authentic adoption. The researcher stresses the importance of pre-registering failure conditions, separate inspection of content, and considering alternative sampling heuristics to avoid manufacturing schema changes.",
  "summary": "Originally published on angeo.dev . Full tables, p-values and the sealed plan are there. Most claims about AI visibility are untestable by design: publish the signals, wait, attribute anything good that happens to the signals. I wanted a version I could not fudge, so I wrote the analysis plan first, hashed it, and sent the hash to the other party before I had any data. The question Do businesses…",
  "key_points": [
    "Study pre-registered to examine AI visibility signals in businesses",
    "Four technical signals analyzed across 458 product-level questions",
    "No significant difference found between businesses mentioning AI once vs. repeatedly"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}