When a Real Website Taught My Agent-Security Scanner Some Manners

This article chronicles an incident involving an AI agent-first website and a security scanner named SkillSentry. The scanner, which is open-source and designed to work locally on a user's machine, was used to analyze the website's files. Despite the website's precautions, such as publishing an llms.txt file that warns agents to treat its content as untrusted and gating its enrichment features behind consent, the scanner flagged 30 low-risk findings.

The bulk of these findings were due to self-referential documentation URLs that were flagged as suspicious. The author of SkillSentry realized that this was a flaw in the scanner's design and implemented a fix to distinguish between URLs used for data exfiltration and those used for documentation. Additionally, a smaller issue was addressed where the --stdin help text incorrectly promised a SKILL.md or MCP config, but in reality, it did not.

These fixes resulted in the scanner correctly identifying the website as clean, with no findings. The author also explained that the scanner operates on static rules with no dependencies on runtime or AI models, and that it offers an optional second opinion from an open-weight model that can be run locally on the user's machine. This design ensures that sensitive data never leaves the user's disk.