Recursive Governance: When Agents Write the Rules They Execute
We almost lost forty modules to a file that never changed. The sync job ran every six hours, mirroring our shared working directory into a test workspace. At 21:47 on a Tuesday, it removed about forty private modules that should have been ignored. The filter file was missing one pattern. That's it. One pattern. A colleague had restructured the source tree three days earlier, added a directory,…
In a case where forty modules were lost due to an uncannily accurate file sync job, a software team learned an important lesson about governance of multi-agent systems. The sync job ran every six hours, mirroring a shared working directory into a test workspace. However, a colleague had restructured the source tree three days prior, adding a directory without updating the filter file.
Consequently, the script mirrored the state it was told to mirror, without warning, asking, or failing. Restoring the modules took almost two hours, done manually from a backup box.
This incident highlights how static policy files fail to keep pace with changes in a running system, as they lack the ability to understand and adapt to the evolving environment. To address this issue, the team gave agents the capability to write their own rules. Initially, the rule store accepted new rules by allowing any agent to append to it. Early on, this approach led to successful resolutions of tool-access conflicts and archiving of stale patterns without human intervention.
However, as the rule store grew in size, inconsistencies emerged—rules referencing other overwritten rules, conflicting rules, and an overall lack of clarity on which rule was actually in effect. The rule history provided some visibility into the rot, but no alerts or metrics were in place to draw attention to the issue. The team realized that each rule should have a birth date, parent, and gravestone, and if a rule can't be archived, it keeps counting toward quorum and confuses the live ones.
The team now employs a governance system comprising five non-removable MCP tools within the agent runtime. These tools enable submitting, amending, ratifying, vetoing, and checking the consistency of rules. The proposal, amendment, ratification, and veto processes incorporate per-shard quorum, ensuring a fair and auditable decision-making process. A consistency check scans the existing rule graph, applying a short list of human-maintained allow/deny rules to prevent obvious category errors and near-misses.
By integrating governance as a part of the runtime contract and making the observation window runtime-configurable, the team mitigates the risk of agents enforcing conflicting rules. The rules live as structured memory events in an append-only namespace, ensuring transparency and the ability to replay the rule state at any past timestamp. By storing rule hashes in cold storage and requiring a quorum to restore from backups, the team prevents corrupted or wrongly pruned governance memory from silently rewriting history.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
