{
  "id": 3743398,
  "title": "Agent Patches Get Three Gates: Property Checks, Pinned Fixtures, and a Flaky Freeze",
  "url": "https://urgent.news/2026/08/27/agent-patches-get-three-gates-property-checks-pinned-fixtures-and-a",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-08-27T13:21:12.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/datacpp_8185/agent-patches-get-three-gates-property-checks-pinned-fixtures-and-a-flaky-freeze-3h0n"
  },
  "original_language": "en",
  "account": "An agent's test suite is green, indicating it appears to pass all tests. However, this green signal is the least informative in modern software engineering. Tests often fail to capture the inputs an agent missed, the data it modified, or the nondeterminism it introduced. To address these failure modes before merging an agent-written patch, three gates have been implemented.\n\nThe first gate, property checks, is the cheapest signal to obtain. It requires only one invariant per function touched by the agent. Using a tool like Hypothesis, the gate generates several examples to test the function's behavior and ensures the function never produces negative results. For example, it tests that \"1h30m\" equals \"1h\" plus \"30m\". This gate runs first because it is fast and deterministic. If a property check fails, the patch is incorrect, not flaky.\n\nThe second gate, pinned fixtures, checks for data drift. Agents often edit tests to make them pass, leading to silent data changes that turn into merge-blocking diffs. This gate compares the agent's modified fixtures against a manifest to ensure no changes occur without a manifest update. If a fixture changes without a manifest update, the gate fails, signaling a need for human approval before the patch can merge.\n\nThe third and final gate, the flaky freeze, handles nondeterministic behavior that passes intermittently. When an agent patch makes a stable test flaky, the gate quarantines the test instead of deleting it or adding a retry. The flaky test is moved to a quarantine list with an expiry date and linked to a ticket. The patch can only merge if the frozen test's path is covered by a property check or a pinned fixture. This gate runs last, after property checks and pinned fixtures have passed.\n\nThese three gates, implemented in a specific order, provide a comprehensive safety net for agent-written patches. They ensure that wrong logic on unseen inputs, data drift due to test data changes, and nondeterministic behavior are all addressed before merging. The gates' structure allows for clear identification of the failure mode when a gate fails, guiding the next steps in debugging and resolving the issue.",
  "summary": "An agent's test suite is green. That is the least informative sentence in modern software engineering. Example tests encode what you already thought of. They do not encode the inputs you missed, the data the agent changed, or the nondeterminism it introduced. Three gates catch those three failure modes before merge. I have been running this stack on agent-written patches for the past few weeks,…",
  "key_points": [
    "Property checks verify function invariants using tools like Hypothesis",
    "Pinned fixtures prevent data drift by comparing modified fixtures to a manifest",
    "Flaky freeze quarantines nondeterministic tests, linking them to tickets for resolution"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}