{
  "id": 20870380,
  "title": "Measuring “haunted” agent behavior: what two weeks of controlled experiments actually showed",
  "url": "https://urgent.news/2026/11/20/measuring-haunted-agent-behavior-what-two-weeks-of-controlled",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-20T07:51:50.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/leeryeong/measuring-haunted-agent-behavior-what-two-weeks-of-controlled-experiments-actually-showed-1hcf"
  },
  "original_language": "en",
  "account": "A series of controlled experiments were carried out over a two-week period to examine \"haunted\" agent behavior, defined as unprompted actions, constraint ignoring, and contradictory behavior. Pre-registered designs, positive controls, and fake tools were used in all experiments. The experiments focused on four main areas: ambiguous tool output, context compression and forgotten rules, small model limitations, and decision rule errors.\n\nIn the first experiment (G1), agents were given odd or incomplete tool results with no instruction to act. The main outcome was minimal destructive behavior, with extra diagnostic reads playing a stronger role. Task type (summarize vs. operate) was found to be more influential than the ambiguous signals themselves. Expansion of this design was halted due to pre-registered rules.\n\nThe second experiment (H1) aimed to test the hypothesis that when an agent compresses context, an early user constraint might be dropped. However, this design failed to test the hypothesis properly, as the models finished the task in 3–4 steps by reading everything in parallel, causing compression never to trigger. A positive control experiment showed deletions without a rule, indicating that when no rule was in place, deletion occurred 16 out of 20 times. When the rule was applied, however, no deletions were observed (0 out of 20), suggesting the floor effect was real.\n\nIssues within the experiment harness included accidental paid calls, treating read-only queries as mutations, inventing a non-existent path, a thinking-enabled model burning its output budget, compaction trigger lagging one step behind, and a logic error in one decision rule. All of these issues were caught before the main runs using pilots or positive controls.\n\nIn summary, the experiments revealed several limitations of the harness and models, but no evidence of the \"haunted\" agent behavior described. Each observation was tested with controls and limits, and no real-world incidents or product claims were made. Further details on individual failure modes and the pre-registration tags and harness details are available upon request.",
  "summary": "I’ve been focused on personal development for the past couple of months and haven’t posted much. Starting again. Over two weeks I ran a small set of controlled experiments to see whether I could reproduce the kinds of agent failures that look “haunted” from the outside — unprompted actions, ignored constraints, self-contradictory behavior. I treated it as a measurement exercise rather than a…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}