Normal Accidents

1. Complex systems are prone to "normal accidents" due to unexpected interactions

Normal accidents are inevitable in complex, tightly-coupled systems with catastrophic potential.

Interconnected components. Complex systems like nuclear power plants, chemical facilities, and aircraft have numerous interconnected parts that can interact in unforeseen ways. These unexpected interactions can lead to cascading failures that are difficult to predict or prevent.

Incomprehensible failures. In complex systems, operators may not fully understand all potential failure modes or system behaviors. This lack of comprehension can lead to incorrect diagnoses and responses during emergencies, potentially exacerbating the situation.

• Examples of complex systems:
  • Nuclear power plants
  • Chemical processing facilities
  • Modern aircraft
  • Air traffic control systems

2. Tight coupling in systems increases the risk of catastrophic failures

Tightly coupled systems will respond more quickly to these perturbations, but the response may be disastrous.

Time-dependent processes. Tightly coupled systems have little slack or buffer between components. When one part fails, it quickly affects other parts, leaving little time for intervention or recovery.

Invariant sequences. In tightly coupled systems, processes must occur in a specific order with little flexibility. This rigidity can make it difficult to isolate problems or implement alternative solutions during emergencies.

• Characteristics of tightly coupled systems:
  • Limited slack or buffers
  • Time-dependent processes
  • Invariant sequences
  • Little substitutability of resources

3. Technological fixes often introduce new risks while addressing old ones

Fixes, including safety devices, sometimes create new accidents, and quite often merely allow those in charge to run the system faster, or in worse weather, or with bigger explosives.

Unintended consequences. New technologies designed to improve safety can introduce unforeseen risks or complications. These additions may increase system complexity, making it harder for operators to understand and manage.

Risk compensation. Safety improvements often lead to increased risk-taking behavior, as people feel more protected. This phenomenon, known as risk homeostasis, can negate the intended safety benefits of technological fixes.

• Examples of technological fixes with unintended consequences:
  • Radar in marine navigation leading to "radar-assisted collisions"
  • Automated systems in aircraft reducing pilot situational awareness
  • Safety valves in chemical plants creating new failure modes

4. Human error is frequently blamed, but system design is often the root cause

If interactive complexity and tight coupling—system characteristics—inevitably will produce an accident, I believe we are justified in calling it a normal accident, or a system accident.

System-induced errors. While human error is often cited as the cause of accidents, many mistakes result from poorly designed systems that set operators up for failure. Complex interfaces, ambiguous information, and time pressure can lead to incorrect decisions.

Hindsight bias. After an accident, it's easy to identify what operators should have done differently. However, this ignores the reality of decision-making under uncertainty and stress in complex systems.

• Factors contributing to operator errors:
  • Incomplete or ambiguous information
  • Time pressure and stress
  • Complex interfaces and control systems
  • Conflicting goals (e.g., safety vs. productivity)

5. Production pressures can compromise safety in high-risk industries

The vast majority of collisions occur in inland waters in clear weather with a local pilot on board. Often the radar is not even turned on.

Economic incentives. In many high-risk industries, there are strong economic pressures to maximize productivity and efficiency. These pressures can lead to decisions that prioritize speed or cost-cutting over safety considerations.

Safety-productivity trade-offs. Operators and managers often face difficult choices between maintaining safe operations and meeting production targets. Over time, these pressures can erode safety margins and lead to normalized deviance.

• Examples of production pressures:
  • Ships sailing in dangerous weather to meet schedules
  • Nuclear plants deferring maintenance to maximize uptime
  • Airlines pushing for faster turnaround times

6. Redundancy and safety features can paradoxically increase system complexity

With each bit of automation, more difficult performance in worse weather or traffic conditions is demanded.

Increased complexity. Adding redundant systems and safety features often makes the overall system more complex. This increased complexity can introduce new failure modes and make the system harder to understand and manage.

False sense of security. Redundancy and safety features can create a false sense of security, leading operators and managers to push systems closer to their limits. This behavior can negate the intended safety benefits.

• Paradoxical effects of safety features:
  • More complex systems to monitor and maintain
  • Increased operator workload to manage multiple systems
  • New failure modes introduced by safety systems
  • Potential for overreliance on automated safety features

7. Effective accident prevention requires understanding the entire system

Probably many production processes started out this way—complexly interactive and tightly coupled. But with experience, better designs, equipment, and procedures appeared, and the unsuspected interactions were avoided and the tight coupling reduced.

Holistic approach. Preventing accidents in complex systems requires a comprehensive understanding of how all components interact. This includes technical aspects, human factors, organizational culture, and external pressures.

Continuous learning. Industries must continuously analyze near-misses and minor incidents to identify potential system weaknesses before they lead to major accidents. This requires a culture of open reporting and non-punitive investigation.

• Key elements of effective accident prevention:
  • Systems thinking approach
  • Robust incident reporting and analysis
  • Regular system audits and risk assessments
  • Emphasis on organizational culture and human factors

8. Marine transport exemplifies an error-inducing system with perverse incentives

I do not see any single failure as responsible for an error-inducing system such as this. Socialist countries are a part of this system, so private profits are not the primary cause of the rise in accidents and the increase risk of creating third-party victims.

Fragmented industry. The marine transport industry is highly fragmented, with many small operators and complex ownership structures. This fragmentation makes it difficult to implement and enforce consistent safety standards.

Perverse incentives. The current structure of marine insurance and liability often fails to incentivize safety improvements. Ship owners may find it more economical to operate older, less safe vessels and simply pay higher insurance premiums.

• Factors contributing to marine transport risks:
  • Flags of convenience allowing regulatory avoidance
  • Inadequate international oversight and enforcement
  • Competitive pressures leading to corner-cutting on safety
  • Difficulty in assigning clear responsibility for accidents

9. Linear systems like dams are less prone to system accidents but still have risks

Dam failures are quite rare, and catastrophic, or even serious consequences are much rarer.

Simpler interactions. Dams and other linear systems have more straightforward cause-and-effect relationships between components. This makes their behavior more predictable and easier to manage compared to complex systems.

Catastrophic potential. While dam failures are rare, they can still have devastating consequences when they do occur. The potential for catastrophic failure requires ongoing vigilance and maintenance.

• Key considerations for dam safety:
  • Regular inspections and maintenance
  • Monitoring of geological and hydrological conditions
  • Emergency preparedness and evacuation planning
  • Long-term effects on local ecosystems and geology

10. Organizational failures often contribute more to accidents than technical issues

Nor was this due to a stodgy industry "boiler business" mentality. The utility industry had been one of the great growth areas in the postwar American economy.

Cultural factors. Organizational culture, decision-making processes, and communication patterns often play a crucial role in major accidents. Technical failures are frequently symptoms of deeper organizational issues.

Normalization of deviance. Over time, organizations can become accustomed to operating outside of safe parameters. This gradual acceptance of risk can lead to major accidents when conditions align unfavorably.

• Common organizational failure modes:
  • Poor communication between departments or levels
  • Prioritization of production over safety
  • Inadequate training or resources for safety management
  • Failure to learn from past incidents or near-misses

Last updated: October 1, 2024