A decision you didn't write down isn't a decision

In the realm of artificial intelligence, a decision made during a conversation is not enough to ensure its persistence. For a decision to truly stick, it must be documented in a file accessible by both humans and AI models. This concept is at the core of architecture decision records (ADRs), which are written records of decisions made during the development process.

The author, Derek Wang, has witnessed this failure mode multiple times in his experience with AI coding. He explains that a decision made during a conversation may seem securely locked in, but the moment a new session begins, the AI model starts anew without any memory of the previous decision. Consequently, the deviation goes unnoticed until it has spread across numerous files, causing significant issues.

Wang identifies three main failures that arise from decisions made only in conversations: memory drift, architecture drift, and repeated-argument drift. Memory drift occurs when the humans remember a decision, but the AI fails to recall it, leading to repeated arguments about the same issue. Architecture drift manifests as small, innocent-looking deviations from the design that accumulate over time, resulting in a codebase that is significantly different from the original intention.

Repeated-argument drift occurs when the same debate happens again with each new session, often leading to conflicting decisions within the same team.

To address these issues, Wang introduces the concept of a single source of truth, a written decision that acts as the ultimate authority for subsequent decisions. This is where ADRs come into play. An ADR is a short document containing essential information about a decision, such as its status, decision-maker, context, decision itself, rationale, consequences, and mitigation strategies.

By providing this self-contained slice of history, ADRs ensure that future engineers or models can understand why a particular decision was made, preventing the need for re-litigating the question.

ADRs have proven effective in real-world projects, as demonstrated by Wang's methodology project. In this project, 20 ADRs were recorded for trade-domain rules, and another 11 were created for the architecture as SmartQuant started up. These ADRs effectively capture the decision-making process and provide a clear record of the choices made.

In the AI era, ADRs have evolved from a top-down constraint to a living record of resolved problems. The Swarm-Fed ADR takes this concept further by incorporating decisions made by multiple agents and models during the development process. These resolutions are then formalized as ADRs and integrated into the single source of truth, ensuring that the accumulated knowledge is preserved and accessible to future generations.

In conclusion, for a decision made during a conversation to truly stick, it must be documented in an ADR. By providing a clear and concise record of the decision-making process, ADRs help prevent memory drift, architecture drift, and repeated-argument drift, ultimately ensuring that the AI system evolves in a coherent and predictable manner.