{
  "id": 4777210,
  "title": "9 Bugs That All Looked Like a Working System",
  "url": "https://urgent.news/2026/09/01/9-bugs-that-all-looked-like-a-working-system",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-01T03:45:36.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/debashish_ghosal/9-bugs-that-all-looked-like-a-working-system-25mg"
  },
  "original_language": "en",
  "account": "AgentSelfEdit is an open-source tool that automatically iterates on its own system prompt based on execution feedback. It performs A/B testing on edits and only adopts the statistically-proven winners. However, while trying it out, I discovered nine critical bugs that all produced output that looked correct but was actually flawed.\n\nBug 1: The Gate Letting Noise Through as Improvement\nThe \"promotion gate\" was incorrectly checking for statistical significance. Instead of looking for a p-value below 0.05, it was checking for a p-value above 0.95. This meant that edits with even a 90% chance of being random noise were being accepted as improvements. After fixing this, the same edit that previously passed with p=0.9 now only passed with p=0.23, rejecting 23% of improvements as noise.\n\nBug 2: A/B Test Compared Prompt Against Itself\nThe A/B testing logic compared the edited prompt against the original prompt, rather than the edited version against the unedited version. When the A/B test produced a perfect tie (p=1.0), it incorrectly rejected the improvement. This occurred because the system was editing only a fragment of the prompt rather than the full system prompt. After fixing the code to construct the full candidate prompt from the edited fragment, the A/B test generated a correct p-value and rejected the improvement when appropriate.\n\nBug 3: Scoring Mode Marked Everything as Passed\nThere was a \"label\" scoring mode designed for real trace data, which incorrectly marked any trace with any non-empty response as a pass. This meant that even failure traces were scored as 100% correct. To fix this, the label scoring mode was entirely removed from the system.\n\nBug 4: Docker Tests Skipped the Promotion Gate\nThe Docker integration tests were configured to bypass the promotion gate and A/B testing. While running a dry-run, the system failed to promote the actual improvement because these tests didn't go through the proper testing gates. This created a false sense of security that the system was working correctly.\n\nBug 5: Failure Traces Were Fabricated\nThe failure traces were not being generated correctly and were being fabricated, leading to incorrect conclusions about the system's performance. This bug was challenging to identify as it had subtle effects on the system metrics.\n\nBug 6: The Gate Received the Wrong Prompt\nThe promotion gate was receiving an incorrect system prompt instead of the correct one, causing it to evaluate the wrong prompt and produce misleading results.\n\nBug 7: The CLI Talked to a Mock Instead of a Real LLM\nThe command-line interface (CLI) was configured to interact with a mock LLM rather than the actual system, giving a false impression of the system's performance.\n\nBug 8: The Config Silently Ignored the Endpoint\nThe configuration file had a bug that caused it to ignore the endpoint specified for the LLM, leading to the system using a default or improperly configured endpoint.\n\nBug 9: The Field Test Runner Measured the Wrong Thing\nThe field test runner was measuring incorrect metrics, focusing on irrelevant aspects of the system's performance and neglecting the key metrics that truly indicated its effectiveness.\n\nEach of these bugs, while seemingly minor, had significant consequences on the system's accuracy and reliability. They highlighted the importance of thoroughly testing and validating every component of an AI system, especially when it is designed to self-improve. The experience taught me the critical importance of rigorous testing, proper statistics, and careful implementation when building systems on top of large language models.",
  "summary": "AgentSelfEdit is an open-source sidecar that rewrites its own system prompt from execution feedback. It A/B tests edits and promotes only statistically-proven winners. Code: github.com/deghosal-2026/agent-self-edit I built an AI that rewrites its own prompts. It looked like it worked. It didn't. Over a single session, I found and fixed 31 issues. Nine of them were fundamental — each one made the…",
  "key_points": [
    "Promotion gate incorrectly checked for p-value above 0.95 instead of below 0.05",
    "A/B test compared edited prompt against original prompt, not against unedited version",
    "Scoring mode marked any trace with non-empty response as 100% correct"
  ],
  "editors_take": "The discovery of nine critical bugs in AgentSelfEdit highlights the importance of rigorous testing and validation in AI systems, especially those designed to self-improve, to ensure accuracy and reliability.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}