Leave a Replay Script, Not a Transcript

A backend engineer submitted a pull request after an afternoon of coding. The ticket stated the pagination bug was resolved, tests were passing, and cache keys no longer collided. However, reviewers encountered a build failure even before the first test began. This failure highlights a crucial issue: chat transcripts are not reliable evidence of an agent's work.

Teams that rely on them often find the work only exists within the original session. To combat this, a spike can be conducted using a replay script that the agent must leave behind. The hypothesis for the spike is defined before any prompts are sent. Given a frozen git worktree, failing characterization test, and a 90-minute time limit, the agent must generate a replay script.

If the script cannot be reproduced on another machine without the agent, the spike is abandoned. Think of the replay script as a black-box recorder, capturing the agent's actions rather than its understanding. The goal is to have a script that can be executed on any machine, proving that the work existed independently of the agent's memory.

The proposed layout includes a disposable directory with a hypothesis file, failing test, patch, and replay script. The hypothesis file contains a short statement that can be verified by a shell script. The clock file sets a time limit for the spike. The characterization test should fail on the main branch, as a spike that creates both the bug and the fix is not genuine.

Before the agent starts, the operator clones a clean worktree and records the source tree's fingerprint. If the replay script runs successfully on a second host without relying on the original chat directory or agent artifacts, the spike passes and can be considered evidence of the agent's work.