AIFeed - signed content permissions for AI web crawlers
I built this because robots.txt kept coming up in conversations about AI crawlers and the answer was always "it's a polite request". AIFeed tries to make it checkable. What it is: a publisher signs a small JSON manifest with Ed25519 and anchors the key in DNS. Agents verify TLS -> domain -> signature -> DNS, then re-check a revocation list. Two markdown profiles for delivery, plus a signed delta…
AIFeed is a system designed to provide signed content permissions for AI web crawlers. It addresses the limitations of robots.txt, which merely serves as a polite request. AIFeed works by having publishers sign a small JSON manifest with Ed25519 and anchor the key in DNS. Agents can then verify the TLS-domain signature and DNS to ensure the content is accessible.
The system also includes a revocation list for added security. Two markdown profiles are available for delivery, and a signed delta index is included so that unchanged pages do not incur any costs. To publish a static site, users only need to run a single command: npx aifeed init --domain example.com --dir ./site. Verifying any domain is as simple as using @aifeed/verify in three lines of code.
Preliminary tests using a local harness show a 68.8% reduction in bytes compared to HTML, and a 0.70 ms per signature check. However, these results are based on a simulation, not a real-world trial. The system has several limitations, including no external crypto review, draft specifications, and the possibility of origin+DNS compromise on first contact being undetectable.
There is an invitation for anyone to ask questions about the design, which relies on JCS+Ed25519, DNS anchoring, revocation semantics, and MAKO compatibility. The AIFeed system can be run using npx -y aifeed-mcp-server, and the registry and repository are linked for further information.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
