Shadow-Compare the Agent Patch. Merge Only Classified Divergences.

A green test run does not represent a behavior specification. An agent patch may maintain all current assertions while modifying encodings, error types, empty input handling, or stdout bytes. Compare the candidate against a static baseline on the same dataset through shadow-comparison. Only merge after classifying every divergence into an accepted-delta ledger.

This document outlines a testing workflow, not a model benchmark. The accompanying harness is labeled as a tentative, executable pattern. It does not guarantee production speeds, model names, or pass rates. The green CI may overlook agent patches, as they optimize for visible tests. Hidden behavior exists in unnamed branches, such as trailing newlines, NaN keys, timezone-naive timestamps, and None versus empty lists.

These are easy to modify but difficult to detect post-merge. A dual-run gate uses the old artifact as the standard for unidentified behavior. Specified behavior should remain in regular tests. The ledger records both purposeful and rejected differences. Do not rely on this as a replacement for code review; instead, use it as a tool to minimize manual review.

The contract consists of three files: baseline (a locked copy, wheel, or container hash), candidate (the agent patch applied to the same reference), and delta_ledger.yaml (previous classified output differences, identified by fixture ID). The ideal layout includes shadow/ and corpus/ directories for deterministic fixtures only.