How do you unit test an agent skill?
Agent skills are prompts, not code, and there’s no compiler to catch a broken one. Agent skills ship on the honour system. You rewrite one, run it twice, post something convincing in Slack, and that’s the review. Is it faster? More reliable? Going to cost more? This isn’t a strategy. This is astrology for prompts. That bugged me. Not because I thought people didn’t know what they were talking…
Agent skills are not code, but rather prompts that are reviewed on a trust system. To ensure these skills are functioning correctly, tests can be built using skilleval. This tool takes a SKILL.md file, a prompt and a fixture for the agent to use and runs a real agent. The results of the test are then asserted against specific criteria such as cost, activated skills, tools used, tool calls, tool arguments, and the final message.
These tests save artefacts so that any changes to the skill can be seen when re-running the test. Running the test multiple times can also provide a pass rate, but this should not stop a skilled individual from pushing changes. The goal is to have a result.json file that proves the skill is making fewer mistakes. This is one way to test agent skills and the author is open to hearing about other methods.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
