{
  "id": 2709239,
  "title": "483 tests passed, but Vestibule RAG framework wasn't installable — lessons from building with AI agents",
  "url": "https://urgent.news/2026/08/23/483-tests-passed-but-vestibule-rag-framework-wasnt-installable",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-23T03:18:54.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/vk032503/483-tests-passed-but-vestibule-rag-framework-wasnt-installable-lessons-from-building-with-ai-b81"
  },
  "original_language": "en",
  "account": "Vestibule, an open-source Python framework for RAG ingestion, was built over two months with the help of AI agents. The framework includes components such as stable document IDs, a state ledger, error classification, and per-vertical governance. Four AI agents worked on designing, reviewing, implementing, and reviewing the code, all through real GitHub pull requests, with the developer signing off at every stage. The framework contains twelve components and three releases, with 878 tests passed.\n\nTwo significant moments defined the experience: catching errors that the developer couldn't, and when green tests lied to the developer. The trickiest component was provisioning vector indexes on first use, which had a race condition issue initially. The first version of the design was rejected and revised five times before any code existed. When the quickstart script was run, the packaging conflict made the whole framework uninstallable, despite 483 tests passing.\n\nThe real issue was what the tests measured; they only proved the code agreed with itself and not the real-world usage. The lasting fix was a CI job that builds a clean virtual environment, does a real installation, and runs the quickstart on every PR. Vestibule addresses issues that typically arise in RAG tutorials, such as retries duplicating chunks, documents silently vanishing mid-pipeline, and one team's config changes corrupting another team's index. Vestibule provides four contracts that other components can plug into, including arrival envelopes, deterministic identity, retries, and a state ledger. Additional features include per-vertical configuration, automatic index provisioning, and pluggable adapters for various parsing, chunking, embedding, and vector store options. The developer recommends knowing the real need before building, using existing tools, and implementing gates to ensure quality before speeding up the process.",
  "summary": "I spent two months building Vestibule, an open-source Python framework for the boring layer of RAG ingestion — stable document IDs, a state ledger, error classification, per-vertical governance. The parts every team struggles with once the demo works and production doesn't. Most of the code wasn't typed by me. Four AI agents did the work — one wrote designs, one reviewed them, one implemented,…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}