Score Fixture Independence Before an Agent Patch Merges

The recommendation is to assess the independence of fixtures prior to an agent patch being merged into the codebase. An agent patch that simultaneously modifies production code and locks fixtures in a single diff has lost its oracle. This is not an isolated flake. The freeze must not absorb this issue. Prioritize fixture independence, ensure an oracle remains untouched by the patch, and subsequently allow a limited freeze for an environmental signal.

The continuous integration (CI) system makes three classes appear similar. A red assertion, timeout, and snapshot mismatch all cause the job to fail. These issues have distinct remedies. Co-modification is considered a review defect. A flawed invariant represents a product defect, while a missed socket is a scheduling defect. Applying a single retry policy to all three problems will obscure the initial two within the third.

Once a fixture can no longer serve as an oracle, it must remain outside the scope of the change being tested. The path condition enabling this verification can be easily checked. It is not a statement about the agent's intent. The typical prompt seeks the agent to ensure the suite passes. The most efficient modification often involves editing JSON files, golden files, or recorded response bodies.

Even if the job appears green, the underlying behavior may still be incorrect. If recent issues have already taught the team to freeze red tests, the subsequent green run will be regarded as a measure of stability. Instead of merely counting paths, focus on individual tests. A diff affecting src/billing/prorate.py and tests/fixtures/prorate/case_18.json does not lock the prorate feature.

It merely duplicates one opinion. Gate 1 requires classifying hunks before any test execution. Create a unified diff from the merge base to the patch tip, disregarding the file list provided by the agent. This diff serves as the record. Assign each path as production, fixture, test, or other using a mapping previously approved by reviewers.

Paths lacking classification are closed off. Pair paths based on their feature key. Paths are deemed co-modified when both the production and fixture lists for that feature are non-empty within this diff. Generate a score: tally co-modified pairs and record unlabeled paths. A boolean embedded within a log line will be disregarded.

This script, proposed for Python 3.9 or later, remains untested against the account's repositories during its creation. Patterns must be exact file paths or a directory prefix (dir/**). This limitation is intentional, as Path.match operates relative to the current directory and could yield false positives for suffixes.