My AI visibility score was 78%. Then I fixed how I measured it and it was 25%

On August 14, a report revealed that my studio had been named uncued in 14 out of 18 blind responses from ChatGPT, Perplexity, and Gemini. However, the methodology behind this figure was flawed. To rectify this, I conducted a revised test using two distinct, isolated sessions for each engine. One session held only the blind discovery questions, while the other held the fact-checking questions. This adjustment led to a corrected score of nine mentions out of 36 answers, which translates to 25%.

This new result was significantly lower than the initial 14 out of 18, indicating a drastic decrease in visibility. Moreover, the improvements were not uniform across all engines. While my studio was effectively detected in just one engine out of the three, it highlighted that the engines were relying more on external mentions and third-party roundups than the studio's own website.

Interestingly, even the engines that scored zero still had access to correct facts when asked directly about the studio. The issue lay not in the engines' knowledge of the brand, but in their decision-making process to surface it unprompted. Therefore, in tracking AI visibility, it's crucial to conduct blind discovery tests and named fact-checking tests in entirely separate sessions.

This prevents the brand name, website, or identifying details from entering the context window before the blind questions are answered. Lastly, always verify the methodology before celebrating an increase in your score. A high score built on a flawed test is not visibility – it's simply context.