{
  "id": 5084932,
  "title": "The Skill That Never Fired: How to Test Whether Claude Actually Picks Your Skill",
  "url": "https://urgent.news/2026/09/02/the-skill-that-never-fired-how-to-test-whether-claude-actually-picks",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-02T12:17:53.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/harryfloyd/the-skill-that-never-fired-how-to-test-whether-claude-actually-picks-your-skill-fae"
  },
  "original_language": "en",
  "account": "Two ways a skill can fail: incorrect instructions causing poor performance or Claude choosing not to load the skill at all. The first failure is easy to spot during testing, while the second remains undetected unless Claude selects the skill on its own. When a skill is written and invoked by name, it functions correctly. However, in normal use, the skill may sit idle, never being chosen by Claude to execute. This selection is a routing decision made by matching the request against the skill's name and description before reading any content from the body. Claude's documentation states that the description is crucial for Claude to determine when to load a skill. Anthropic advises testing this decision independently from the skill's output and provides a tool to perform this check. The Skill Creator evaluates a single skill over multiple runs to determine if it fires for the appropriate prompts and remains silent for the ones it should not. However, this score does not reveal what happens when another similar skill is present, leaving open the possibility of the skill losing to a rival. The article explores building a skill, observing the decision-making process, and assessing its performance against a competing skill. The process can be completed within 15 minutes using a terminal. A skill is essentially a folder containing the required SKILL.md file. Additional scripts and reference files may be included but are only loaded when needed. The minimum requirement is the SKILL.md file, which contains metadata such as the skill's name and description. When a skill is auto-invocable, the name and description are placed in Claude's discovery context, enabling Claude to decide if the skill is relevant. The body of the skill file is executed only when the skill is invoked, whether chosen by Claude or invoked manually. Claude examines both the name and description, using the description as the primary indicator for when the skill should run. The description should be written with the router in mind, not as a note for the developer. There are three common methods to use a skill. In claude.ai, enable code execution, navigate to Customize then Skills, and upload the skill folder as a zip file. In Claude Code, place the skill folder in .claude/skills/ for a single project or ~/.claude/skills/ for all projects. Through the Claude API, the skill can be uploaded and referenced by its unique skill_id. The SKILL.md format is consistent across all three platforms, although installation methods and certain frontmatter options, such as disable-model-invocation, are specific to Claude Code. To observe the routing decision, avoid relying solely on the skill's output. Instead, request Claude to format a date and verify that the skill runs when expected and remains dormant when not necessary. Claude Code presents the skill's invocation through a Skill tool in the event stream. By filtering the stream, you can identify skill calls and examine the input. For example, running the command $ claude -p \"Rewrite this date for the customer email: 2026-08-30\" --output-format stream-json --verbose | jq -c \"select(.type== \\\"assistant\\\") | .message.content[]? | select(.type== \\\"tool_use\\\" and .name== \\\"customer-date\\\") | .input\" will isolate the filtered result. If the output is empty, manually search for the Skill call in the stream, as the format may vary between versions. The route decision is determined from the tool call, not inferred from the output. The claude.ai interface does not provide a machine-readable event stream, making verification by machine difficult. Anthropic suggests reviewing Claude's thinking to confirm if a skill loaded, which is feasible for manual inspection but impractical for automated record-keeping. The event stream demonstrates the skills Claude actually invoked, including both when invoked simultaneously, highlighting the difficulty in determining which skill was outperformed. This process allows for grading the decision-making process. Two skills are created with distinct functions: customer-date formats dates for customer emails in long form, while export-date formats dates for CSV exports as DD/MM/YYYY. A labeled prompt set is then constructed, containing four requests for the customer skill, four for the export skill, and four that should not trigger either skill. Each response is labeled as right, wrong, none, or both, enabling a comprehensive evaluation. The failures are particularly important, as they demonstrate the effectiveness of the method in identifying potential issues. Ambiguous requests, such as \"Format this date: 2026-08-30\", can result in varied outcomes, including one skill firing, both firing, or neither firing. This ambiguity is expected due to the lack of clarity in the request, and no definitive answer can be graded against.",
  "summary": "The Skill That Never Fired A skill can fail in two ways. Its instructions can be wrong, so it does the job badly. Or Claude can decide never to load it, so the instructions never run at all. The first failure is obvious when you test the skill by name. The second only shows up when you test whether Claude chooses it on its own. The second failure is the quiet one. You write a skill, you invoke it…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}