The Gate Said No. Now What? A Triage Procedure for Rejected Agent Patches

A gate that rejects a patch is only half of a policy. The other half is what happens after a patch is rejected. In most pipelines, a failing agent patch usually leads to one of three outcomes: a human reads the log, the patch is rebuilt blindly, or the test is deleted. These three outcomes are wrong in different ways. This article proposes a decision procedure to classify rejected patches into three failure classes, assign an action to each class, and maintain a quarantine ledger with an expiry date.

The result is a reproducible script called triage_gate.py that re-runs the failing test, compares fixture hashes, and freezes only the flakes.

There are three main types of failures:

Class A - Deterministic regression. The test fails on the first run and every subsequent run with the same input. The agent's code is primarily responsible.

Class B - Fixture drift. The test passes locally but fails on the server. A fixture ID, snapshot, or generated seed changed outside of the patch. The agent patch may be innocent.

Class C - Flake. The test fails intermittently. Re-runs alternate between red and green. Timing, ordering, or shared state are the suspects, not the patch.

The procedure involves re-running the failing test three times with the same seed and command. If all runs are green, the patch is a Class C candidate. Mixed results indicate a Class C flaky test. All red results move the patch to the next step. Fixtures are hashed and compared with the hash recorded at the last green commit. If there's a mismatch, it's a Class B fixture drift issue.

If the hash matches, it's a Class A deterministic regression. The agent patch should be fixed or, better, the failing input should be added to the property test corpus to become a property violation instead of an isolated assertion.

Class C failures are properly froze by appending the test to quarantine.json with a first-seen timestamp, reason, and expiry date. CI skips quarantined tests. The expiry date acts as a guard, ensuring that a freeze without a deadline is not equivalent to deletion with extra steps. Every verdict is logged to the quarantine ledger, which becomes the report. The triage_gate.py script is a minimal implementation of this procedure, taking a test ID and command, running the command three times, and writing the verdict.