The Dimension That Predicts Downloads Is Not the One We Rank Highest

Our rubric indicates that trigger quality is the factor that determines whether a skill is utilized. However, the download data reveals a different story. The correlation between each of the six dimensions in our rubric and the download counts from 591 fully-evaluated skills was examined. The results showed that structure had the strongest correlation at 15% (0.56), followed by content and engineering at 10% (0.52 each), security at 20% (0.42), workflow design at 25% (0.27), and trigger quality at 20% (0.15).

Trigger quality, the dimension we have emphasized as the most crucial aspect of a skill, emerged as the weakest predictor of whether the skill gets downloaded. There are two plausible explanations for this. Firstly, trigger quality becomes evident only after the skill has been installed, whereas downloads occur at the moment of discovery, when a person is simply looking at a page, list, or description.

Consequently, structure and content, which can be evaluated at the time of discovery, appear to be better predictors of downloads. Secondly, the downloads might not be driven by human users. A significant portion of the downloads come from automated clients, and their positive correlation with structure and content may be due to the packaging signals of a clean file tree and reasonable size, rather than the actual trigger quality. This would artificially inflate the ratings for structure and content.

It is important to note that the trigger quality argument should not be based on download numbers alone, and we must refrain from implying that trigger quality is the primary factor. The structure number is the most significant statistic, with a correlation of 0.56, and it is the dimension with the widest disparity between top and median skills (2.64 points compared to 0.46 for security).

It is also the most cost-effective to improve: a concise SKILL.md file with necessary details in references/, scripts in scripts/, and templates in assets/. If even a fraction of this correlation is accurate, the most impactful change for a skill author would be to refactor a lengthy 900-line file into the layout that the agent can actually load selectively.

The missing measurement that could resolve this issue is the activation rate in production: the percentage of sessions where a skill is installed and it fires when the user's request matches. This would directly test the trigger-quality hypothesis without the confounding effect of downloads. Currently, we do not possess this data.

Once we have it, this post will be revised accordingly. Until then, it is crucial to acknowledge that while we rank trigger quality highest based on belief, our own adoption data does not substantiate this claim. It is essential to express this truth openly rather than silently omitting the correlation from the analysis.