{
  "id": 9053001,
  "title": "AIFeed - signed content permissions for AI web crawlers",
  "url": "https://urgent.news/2026/09/22/aifeed-signed-content-permissions-for-ai-web-crawlers",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-22T01:10:00.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/denyn1/aifeed-signed-content-permissions-for-ai-web-crawlers-cj3"
  },
  "original_language": "en",
  "account": "AIFeed is a system designed to provide signed content permissions for AI web crawlers. It addresses the limitations of robots.txt, which merely serves as a polite request. AIFeed works by having publishers sign a small JSON manifest with Ed25519 and anchor the key in DNS. Agents can then verify the TLS-domain signature and DNS to ensure the content is accessible. The system also includes a revocation list for added security. Two markdown profiles are available for delivery, and a signed delta index is included so that unchanged pages do not incur any costs. To publish a static site, users only need to run a single command: npx aifeed init --domain example.com --dir ./site. Verifying any domain is as simple as using @aifeed/verify in three lines of code. Preliminary tests using a local harness show a 68.8% reduction in bytes compared to HTML, and a 0.70 ms per signature check. However, these results are based on a simulation, not a real-world trial. The system has several limitations, including no external crypto review, draft specifications, and the possibility of origin+DNS compromise on first contact being undetectable. There is an invitation for anyone to ask questions about the design, which relies on JCS+Ed25519, DNS anchoring, revocation semantics, and MAKO compatibility. The AIFeed system can be run using npx -y aifeed-mcp-server, and the registry and repository are linked for further information.",
  "summary": "I built this because robots.txt kept coming up in conversations about AI crawlers and the answer was always \"it's a polite request\". AIFeed tries to make it checkable. What it is: a publisher signs a small JSON manifest with Ed25519 and anchors the key in DNS. Agents verify TLS -> domain -> signature -> DNS, then re-check a revocation list. Two markdown profiles for delivery, plus a signed delta…",
  "key_points": [
    "AIFeed provides signed content permissions for AI web crawlers.",
    "Publishers sign JSON manifest with Ed25519 and anchor key in DNS.",
    "System includes revocation list and 68.8% reduction in bytes."
  ],
  "editors_take": "This development gives publishers a more secure way to control AI web crawler access to their content, addressing limitations of the current robots.txt system.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}