The wrong question when an AI breaks is "where"; the right one is "which layer"

When an AI system produces a malfunctioning output, the first reaction is often to ask "where" the issue lies. This focus on a specific location within the system is a dangerous trap, as it assumes the problem is a concrete spot that can be easily identified and fixed. The right question to ask instead is "which layer" of the system is responsible for the problem.

Breaking down the layers, the first (L1) is the phenomenon itself - the error or malfunction observed. This description of what went wrong almost never provides insight into why the issue occurred. The second layer (L2) examines the structure of the code. Is the code itself flawed in some way? For example, if the error is an "interface timeout," the structure layer might question whether the service never set a timeout configuration, or if the default timeout is set too low.

The third layer (L3) delves into the design of the system. Is the overall plan or assumption flawed? Continuing the example of an "interface timeout," the design layer might ask whether the system was designed to account for the possibility that another service could be slow or unresponsive.

The key lesson here is that fixing the symptoms at layer L1 rarely resolves the underlying issue. Instead, a more thorough investigation is needed to determine the root cause. This involves probing deeper into the structure and design layers, asking questions such as "why is the data wrong?" or "did the upstream feed bad data?" Often, what appears to be a data problem is actually a combination of structure and design problems.

In addition to focusing on the layers, another crucial principle is to avoid jumping to conclusions that a root cause has been found and then proceeding to code the fix. This is where the concept of "ripple analysis" comes in. Before making any changes, it is essential to scan the blast radius of the change - both directly and indirectly - to understand how it might affect other parts of the system. This means accounting for all potential downstream effects before altering any code.

The author recounts a real-world example of a bug that took three rounds to fix. In the first round, a change was made to an unpacking field, but the regression only partially passed. In the second round, it became clear that the field and unpack were both actually correct. The real problem lay in another microservice's interface timeout. In the final round, the true issue was revealed to be the remote call lacking a timeout parameter altogether, with the default timeout being too small to handle any slowness.

This story emphasizes the importance of fully understanding the "seven-eighths below the water" - the deeper issues that often lie hidden beneath surface-level symptoms. A diagnosis that stops at the surface level only provides temporary relief, while a fix that reaches the root cause is a true cure. The craft of debugging is not about speed, but about ensuring causation is traceable and preventing a recurrence of the incident.