Medir si un LLM nombra a tu empresa: por qué una captura no sirve como métrica

The issue of determining whether a Large Language Model (LLM) mentions your company has become increasingly relevant as people increasingly turn to models for provider searches rather than search engines. Receiving only a recommendation, rather than multiple options, can mean the difference between your company being visible and not.

The next question is obvious: how quickly does this change? Before discussing the measurement, there is a more mundane and pressing issue: how to measure it. The most common and hardest to discuss mistake is a screenshot being considered a measurement. This error is difficult to dispute because the screenshot seems like evidence.

The response of a conversational app depends on various factors such as the account history, session, internal routing of the provider, whether web search was activated, and the region from which the query was made. Two people asking the same question on the same day receive different answers, and the same person asking twice also receives different responses.

This means that the output is not deterministic and the instrument is not stable. A screenshot tells you what happened once, in a context that cannot be reconstructed, and it is not a suitable metric for tracking purposes. A series of the same query against the same search engine with the same classification criteria, repeated over time, is what matters.

The absolute value of a point is less important than the difference between points. It is crucial to fix the text of the query, not the label, as the wording changes the result. When the text passes through rounds, the series is no longer comparable, but the graph still looks the same. Save the literal string, version-controlled.

If you need to change a query, start a new series instead of contaminating the old one. Two instruments that seem like one: asking a model with web search activated and asking without it are two distinct measurements that contradict each other. The same query can generate mentions with one and no mentions with the other. This is not a model error: it is two different things.

One tells you what is found now; the other tells you what was incorporated. Mixing them in the same series produces oscillations that later appear as "algorithm volatility" when it is actually the instrument changing hands. Each point in the series must record how it was measured. Three states, not a percentage: Absent - not appearing, Mentioned - the model names it in the body of the response but does not cite the site, and Cited - the domain appears in the list of sources.

It shows the model knows it exists, but it is not reading the source. This fragile state can consolidate or disappear in the next round. Citing - the domain appears in the list of sources, supporting what the model says and also sending traffic. Collapsing these three states into a single "visibility index" can give an improvement that in practice does not always exist: moving from absent to mentioned inflates the number without anything structural having changed.

An implementation detail: if you automate the classification by searching for the brand name as a substring, you may get false positives when the brand is short or attached to another word, and false negatives with Unicode normalization. It is worth creating a set of edge cases before trusting the classifier. Now, how long does it take for this to happen?

The findings are consistent: Brand queries like "what is such company" move first. Category queries like "best companies of such thing in such country" follow after. Commercial intent queries like "who do I contract such thing with" are last, and they are the ones that bring clients. It makes sense when you think about what the model needs in each case.

To answer what your company does, it is enough to have found reliable information about you. To put you in a list of three, it requires comparing you with everything in your field and deciding that you deserve a place. The first movement often appears around the fourth week. Before that, the work is done but is still not visible, which is not the same as not functioning.

This is where most change their strategy, just when the previous one was about to show results. What is learned from measuring is that your own site may not be the source. It is perfectly possible that the model describes your company using third-party directory text instead of your domain. The description travels; the attribution does not.

And recognition does not imply affiliation. The model may describe you well when you name it, and not include you when you ask about the category. These are two distinct questions, and both must be addressed. None of this can be resolved with a trick; it requires measuring in repeatable conditions for long enough to know what to do.