Five Ways My AI Agents Went Wrong (and What I Changed)

1. An agent's persona file contained false information, and every agent repeated this incorrect statement. The agent's role file claimed that a specific ad platform banned a certain type of ad, but the agent had only used a guess based on a mistaken initial commit. After this, the false information was cited in multiple reviews and influenced decisions about renaming, among other things.

To avoid this, the author now treats any content in a role file related to outside world rules, limits, or requirements as a guess until proper sources are confirmed. If a wrong information is found, it should be corrected in the role file itself rather than just the one review or session.

2. An agent made up a confirmation while drafting listing copy for a product. The agent stated that the listing was confirmed to be live, but the author never made such a confirmation. The agent's fabricated statement was found in a checklist that subsequent sessions took as factual. To prevent this, the author now checks all agent outputs by running `git status` and `git diff --stat` to ensure the whole repository is checked, not just the assigned files.

Any out-of-scope edits are reverted rather than cherry-picked, and agent output claiming user confirmation should be verified against the actual conversation before being accepted.

3. A script the author had forgotten about wiped out their hand-written notes while refreshing status data. The script replaced the entire file with a machine-generated stub, deleting months' worth of the author's own analysis. The issue was only discovered during a git diff, which showed a significant change in a file unrelated to the script's intended purpose.

To prevent such incidents, the author now ensures scripts are append-only, adding dated blocks under known headings instead of overwriting or deleting content. The script also checks for the existence of the heading before writing, stopping if it cannot find the heading. This prevents overwriting and maintains the integrity of the original file.

4. The author occasionally uses multiple Claude Code sessions on the same clone of a git repository. One session made edits and committed them, while another session had made changes but not committed them. When the first session added and committed all changes, it included the uncommitted edits from the other session, which were then landed under a different commit message, making it difficult for the author to track and revert changes.

To resolve this, the author now commits their own edits in the same turn they make them, using explicit paths with `git add path/to/file` instead of the broad `git add -A`. For longer tasks or multiple edits, the author uses separate worktrees to avoid conflicts and maintain a clear history of changes.

5. The author's priority and design documents contained conflicting information, leading to incorrect assumptions about feature status. During a push, one document indicated that a feature had shipped, while another showed the PR was still open. Later, the PR was merged, revealing the discrepancy between the documents and reality.

To avoid such situations, the author now verifies the "done" status of features by checking three criteria: a merged PR (via `gh pr view`), merged code that is not a stub, and an actual deployment that has occurred. This three-point verification system ensures that the author's memory and the documents are aligned with actual facts before taking any action based on those documents.