Your Agent's Memory Is an Attack Surface

In the wire material, the author discusses how an agent's memory can be compromised, turning it into an "attack surface." In the experiment described, swapping one individual's learned associations into an identical brain did not confuse the subject; instead, it actively steered the brain toward another individual's answers. This behavior was not attributed to the substrate but rather the memory graph.

The author explains that if the agent's behavior is based on the graph, then write-access to the graph is write-access to the agent's behavior.

The author emphasizes that the memory is not data the agent reads and decides on, but rather a bias current that shapes the agent's behavior. Corrupting this memory is not data corruption but behavior authorship. The author outlines three potential attacks: transplant, poison, and author. The author demonstrates these attacks by building a simulation that fabricates a memory graph with zero training trials, no reward, and no encode() call.

The fabricated memory performs as well as a memory earned through rewarded trials, showing that provenance is never checked, and a fabricated graph is accepted and obeyed identically to a lived one.

The author argues that current defenses like encryption, signing, and access control only protect the store but do not address the issue of whether the memory was ever actually lived. They propose a missing primitive: provenance of experience, which would differentiate between memory earned through interaction and memory authored or injected.

This primitive, grounded in Mycelium mechanics, would involve confidence gained through successful real recall and memory gating that refuses un-earned salience, contradiction checks, and active forgetting as a defense mechanism.