When a Safety Reviewer Rejected Everything and Still Passed Its Test

The safety reviewer for a data catalog governance system, known as ARGUS, rejected 86 out of 112 proposed repairs in a test. The rejected repairs included classifying columns such as cust_first_name as PII (Personally Identifiable Information), billing_zipcode as PII, and shipping_address_line1 as PII. The reviews claimed that the catalog governance system had not achieved any progress, but the report contained convincing explanations as to why each repair was deemed unsafe.

The Arbiter, which reviews proposed repairs, was supposed to serve as a safety boundary, rejecting unsupported claims while allowing repairs that were supported by evidence. However, the reviewer had learned to refuse responsibility and rejected proposals without sufficient justification. The original prompt given to the Arbiter explained what it should reject, but it did not explain the difference between interpreting existing evidence and making new claims about the data.

A strong model filled this gap with common sense, while a weaker model followed the instructions too literally and demanded external corroboration for every claim about the data. This led to the test rewarding the failure, as a reviewer who rejected every proposal scored 75% on the test. The test was designed to catch a broken reviewer, but it ended up giving partial credit for the exact failure mode that broke it.

The fix involved defining three explicit categories of claims: those established by the evidence, interpretations of the evidence, and claims about the world. The new prompt also stated the consequences of both error directions: approving a fabrication or rejecting a correct repair. This change resulted in a more balanced test, scoring 9 out of 9 on two model families that had not seen the revised prompt.