How Complex Systems Fail

All intricate systems, such as transportation, healthcare, and power generation, inherently possess hazardous elements. The likelihood of encountering hazards can be altered, but the inherent hazardous nature of the processes themselves is intrinsic and cannot be eliminated. This presence of hazards compels the development of defenses against potential failures, which are characterized by multiple layers of protection.

These safeguards encompass both technical components, such as backup systems and safety features, and human elements like training and knowledge. They also include organizational, institutional, and regulatory measures like policies, procedures, certification, work rules, and team training. These collective defenses effectively divert operations away from accidents, ensuring the successful operation of the system.

Catastrophic failure occurs when seemingly harmless failures combine to create opportunities for systemic accidents. Although numerous such failure opportunities exist, overt system accidents are relatively rare. Most initial failure trajectories are prevented by the designed safety components within the system. However, when failures reach the operational level, they are typically blocked by practitioners.

The complexity of these systems necessitates the presence of numerous flaws, which are considered minor during operations and are regarded as insufficient to cause failure individually. The failures are dynamic due to changing technology, work organization, and ongoing efforts to eliminate potential failures, making it economically challenging to eradicate all latent failures before they contribute to an accident.

As a consequence, complex systems operate as broken systems, functioning due to redundancies and human adaptability. After accident investigations, it is often noted that the system has a history of prior "proto-accidents" that almost led to catastrophe. The belief that these degraded conditions should have been recognized before the overt accident is often based on the assumption that system performance was not compromised before the failure.

Complex systems have the potential for catastrophic failure, and human practitioners frequently operate in close proximity to these potential failures. Disaster can occur at any time and in any location due to the continuous replacement of components (organizational, human, technical) within the system. This inherent potential for catastrophic failure is a defining characteristic of complex systems.

It is impossible to eradicate this potential, as it is a fundamental aspect of the system's nature. No single isolated cause can be identified for an accident, as multiple contributors must combine to create the circumstances necessary for failure. Each contributor is insufficient on its own to generate an accident. The linking of these causes is what ultimately leads to the accident.

Therefore, root cause analysis is not feasible, as it fails to capture the technical understanding of failure but rather reflects the social and cultural need to assign blame to specific, localized forces or events. Anthropological field research demonstrates the social construction of the notion of 'cause' (Goldman L, 1993; Tasca L, 1990).

Practitioners operate the system to produce desired outputs while also working to prevent accidents, creating a dynamic balance between these two objectives. Outsiders often fail to recognize this dual role, emphasizing either the production aspect during non-accident-filled periods or the defense against failure aspect after accidents.

After accidents, the overt failure may appear inevitable, and practitioner actions may be perceived as blunders or deliberate disregard for impending failure. However, all practitioner actions are, in fact, gambles, taken in the face of uncertain outcomes. The level of uncertainty may fluctuate from moment to moment, and practitioner actions become clear after accidents.