The Agent Said It Worked. I Asked the Kernel.
Before evaluating an agent’s code, I built a backup client with eight known behaviors to check the instrument itself. My response to “it works” is becoming: “Let me see the packet capture.” This may become a personality problem. For now, it is an experiment. This experiment was inspired by Hemapriya Kanagala (@hemapriya_kanagala) and her article, What Happens When AI Outgrows the Tests We Use to…
The experiment aims to validate an agent’s claim of successful execution by providing independent evidence. To do this, a backup client was built with eight known behaviors, and its actions were observed without direct access to its internal logs. The client was programmed to follow a specific requirement: the saved backup must match the original source file exactly.
The experiment was inspired by Hemapriya Kanagala's article questioning the informativeness of evaluations when benchmarks reach saturation and test conditions change. The author sought to examine what independent evidence supports a program's claim of success.
Some implementations may produce the correct result while engaging in questionable practices. One such example reported success without actually backing up any data. The kernel, which manages the system's core operations, does not readily accept such misleading reports.
The author emphasizes that CPU execution and network activity are not enough to ensure correctness. The resulting state of the saved file serves as crucial evidence that must be measured against the user's requirements. While frameworks and libraries simplify software development, they cannot guarantee the proper functioning of unfamiliar code.
The experiment employs a modest setup using a native C client to read a 512 KiB file and a Python receiver to listen on loopback TCP. The client calculates a SHA-256 digest to verify content equality before transmitting it. The receiver validates incoming content before committing it to storage.
A separate verifier computes the saved file's digest and compares it with the source, ensuring the backup client adheres to the specified contract. The experiment runs the client twice against the same receiver state to observe its behavior, focusing on scenarios where the client uploads matching content on the first invocation and ignores unchanged content on the second.
The observers collect various types of evidence, including system calls, eBPF kernel events, and user-space probes. These observations help track system interactions, such as file reads, connection calls, and hash function entries. The goal is to identify any anomalous behavior or unnecessary work, like repeated hashing and spinning.
The experiment aims to demonstrate that while AI agents can produce implementations and tests, they may not always align with the intended requirements. Independent evidence, such as file comparisons, can help challenge the validity of these implementations. By comparing hashes and scrutinizing system interactions, the experiment seeks to provide a more reliable assessment of an agent's claimed success.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
