A Systemic Review of Safety Science: Evolving Paradigms, Complex Systems, and the Pursuit of Resilience

Many thanks to our sponsor Focus 360 Energy who helped us prepare this research report.

Abstract

Safety science, once primarily concerned with individual failures and linear cause-and-effect relationships, has undergone a significant paradigm shift towards recognizing the inherent complexity of sociotechnical systems. This research report provides a systemic review of safety science, exploring its evolving paradigms, focusing on the challenges posed by complex systems, and examining the crucial role of resilience engineering. It critically evaluates the limitations of traditional approaches, such as root cause analysis, and advocates for proactive strategies that embrace emergent behavior and adapt to dynamic environments. The report also investigates the ethical considerations inherent in safety management, particularly regarding the allocation of resources and the acceptance of residual risk. Furthermore, the report considers future directions for safety science, including the integration of artificial intelligence and machine learning to enhance risk assessment and incident prediction, while simultaneously emphasizing the need for human-centered design and participatory approaches to ensure effective implementation and acceptance of safety interventions.

Many thanks to our sponsor Focus 360 Energy who helped us prepare this research report.

1. Introduction: The Shifting Landscape of Safety

Safety, traditionally defined as the absence of unacceptable risk of harm, has evolved from a reactive discipline focused on preventing accidents to a proactive science aiming to understand and manage systemic vulnerabilities. Early approaches to safety were rooted in a linear model of causality, attributing accidents to individual errors or equipment failures. This ‘linear thinking’ (Reason, 1990) dominated safety management for decades, leading to the widespread adoption of root cause analysis and corrective action systems. However, the limitations of this approach became increasingly apparent as technological systems grew more complex and interconnected, and safety issues grew more complex, interconnected and frequent.

The realization that accidents often arise from the confluence of multiple contributing factors within a complex system marked a pivotal shift in safety science. This shift emphasized the need to move beyond individual blame and embrace a systemic perspective. Complex systems, characterized by interconnectedness, feedback loops, and emergent behavior, pose unique challenges to safety management (Dekker, 2006). In such systems, seemingly minor events can cascade into major incidents, and the behavior of the system as a whole cannot be predicted simply by analyzing its individual components. It is therefore of paramount importance to move from a reactive stance to a proactive approach that embraces complexity in order to increase safety.

The increasing relevance of resilience engineering, which focuses on enhancing a system’s capacity to anticipate, adapt to, and recover from unexpected events, reflects this paradigm shift (Hollnagel et al., 2006). Resilience engineering acknowledges that systems will inevitably encounter disturbances and stresses, and seeks to build in mechanisms for coping with these challenges. Furthermore, the ethical dimensions of safety management, particularly regarding the allocation of resources and the acceptable level of residual risk, are also being increasingly scrutinized. As we develop our understanding of risk and human factors, we are faced with the moral dilemmas of which risks we are willing to accept in the process of innovation and industry, and where we are willing to compromise on safety.

Many thanks to our sponsor Focus 360 Energy who helped us prepare this research report.

2. From Linear Causality to Systemic Thinking

The traditional approach to accident investigation, centered on identifying the ‘root cause,’ often oversimplifies the complex interplay of factors that contribute to incidents (Woods, 2000). Root cause analysis typically seeks a single, definitive cause that, once eliminated, will prevent recurrence. While this approach can be useful for addressing simple, isolated problems, it fails to capture the systemic nature of many accidents. It is also a backward-looking approach, and as such will always be less effective in preventing issues than a well-designed, proactive solution. It fails to address the fundamental nature of how errors and failures arise in systems. The ‘root cause’ is often a symptom of deeper systemic issues. This is partially why this approach often results in treating the human factors element of errors as the root cause, instead of focusing on the systemic issues that allow for those errors to occur. The temptation to point the finger will always be a more tempting option than redesigning the system.

Systemic thinking, in contrast, emphasizes the interconnectedness and interdependence of elements within a system. It recognizes that accidents are rarely the result of a single cause but rather arise from the interaction of multiple factors, including organizational culture, management practices, technological design, and human behavior (Leveson, 2011). This perspective highlights the importance of understanding the relationships between different parts of the system and how they contribute to overall performance. Systemic thinking recognizes that a complex network of factors can contribute to an accident, and focuses on how they relate to each other. It also takes into account that the system in question will change and adapt over time.

A key concept in systemic safety is the notion of ‘drift’ or ‘normalization of deviance’ (Snook, 2000). This refers to the gradual erosion of safety margins over time as individuals and organizations adapt to operational pressures and become accustomed to accepting minor deviations from established procedures. Such drift can be insidious and difficult to detect, leading to a gradual increase in risk exposure. This is in part due to human beings becoming desensitized to certain hazards when they are frequently exposed to them. When we no longer perceive hazards, we are less able to protect ourselves and others from them.

Many thanks to our sponsor Focus 360 Energy who helped us prepare this research report.

3. The Challenges of Managing Safety in Complex Systems

Complex systems exhibit several characteristics that pose unique challenges to safety management. First, they are often characterized by emergent behavior, meaning that the behavior of the system as a whole cannot be predicted simply by analyzing its individual components (Checkland, 1999). Emergent properties arise from the interactions between different parts of the system, and can be difficult to anticipate or control. Emergent behaviour is the result of many different factors interacting in a system and producing results that cannot be foreseen with complete accuracy. Attempting to control emergent behaviour is futile, and will often produce additional unintended effects. The alternative is to design the system in such a way that the behaviour is predictable and within acceptable limits.

Second, complex systems are often tightly coupled, meaning that events in one part of the system can quickly cascade to other parts, potentially leading to catastrophic failures (Perrow, 1984). Tight coupling increases the potential for propagation of errors and makes it more difficult to contain incidents once they occur. This is compounded by the fact that many systems are also increasingly complexly interactive. This term is used to describe systems where it is difficult to predict the ways in which components will interact, even if they are relatively simple. In a tight and complex system, the potential for small errors to quickly escalate into large-scale catastrophes is increased exponentially.

Third, complex systems are often dynamic and constantly evolving, making it difficult to maintain a stable understanding of their risk profile. Changes in technology, organizational structure, and operating procedures can all affect the system’s safety, and these changes must be continuously monitored and assessed. If they are not, it is possible that the system may reach a critical point where it is no longer able to cope with stress or disturbance. These critical points can be difficult to predict and even harder to reverse once they have been reached. Systems must be actively monitored and adapted to avoid catastrophic failure.

Finally, managing safety in complex systems requires a high degree of situational awareness on the part of operators and decision-makers (Endsley, 1995). This involves being aware of the current state of the system, understanding its potential vulnerabilities, and anticipating future events. Situational awareness can be enhanced through training, simulation, and the use of advanced decision support tools. However, it is also important to recognize the limitations of human cognition and to design systems that minimize cognitive workload and promote effective communication. Humans are not computers, and are susceptible to numerous biases and distractions that can affect our perception of risk. Safety systems should be designed to compensate for these human factors, and not to rely on humans to be infallible.

Many thanks to our sponsor Focus 360 Energy who helped us prepare this research report.

4. Resilience Engineering: Building Capacity to Adapt

Resilience engineering offers a promising approach to managing safety in complex systems by focusing on building a system’s capacity to adapt to unexpected events (Hollnagel et al., 2006). Unlike traditional safety management, which seeks to eliminate all potential hazards, resilience engineering recognizes that systems will inevitably encounter disturbances and stresses, and seeks to build in mechanisms for coping with these challenges. These mechanisms can include: the ability to learn from past events, to self-organize in response to new challenges, and to maintain a flexible and adaptable approach to operations. In the long term, the most effective resilience engineering measures should be designed into the system from the outset, so that the systems are inherently adaptable.

Four key capabilities are central to resilience engineering: monitoring, anticipating, responding, and learning (Hollnagel, 2017). Monitoring involves continuously tracking the system’s performance and identifying potential vulnerabilities. Anticipating involves predicting future events and developing plans for responding to them. Responding involves taking appropriate action when unexpected events occur. Learning involves analyzing past events and using the lessons learned to improve future performance. These four elements must work in unison to create a system capable of adjusting to unforeseen situations. For instance, monitoring can alert decision-makers to an imminent issue, anticipation allows them to consider the potential consequences, response allows them to take appropriate action, and finally, learning can ensure that future responses are even more effective.

Resilience engineering also emphasizes the importance of distributed cognition (Hutchins, 1995), which recognizes that cognitive processes are not confined to individual minds but are distributed across individuals, artifacts, and the environment. This perspective highlights the importance of designing systems that support effective communication and collaboration among different actors. In order to foster distributed cognition, systems must be designed in such a way that information is easily accessible and understandable to all stakeholders. It is also necessary to ensure that all actors understand their roles and responsibilities, and are able to effectively coordinate their actions with others.

Many thanks to our sponsor Focus 360 Energy who helped us prepare this research report.

5. Ethical Considerations in Safety Management

Safety management is inherently intertwined with ethical considerations, particularly regarding the allocation of resources and the acceptance of residual risk. Decisions about how to allocate resources to safety initiatives often involve difficult trade-offs between competing priorities. For example, investing in safety improvements may come at the expense of other organizational goals, such as profitability or productivity. Furthermore, the acceptance of residual risk raises ethical questions about the level of risk that is considered acceptable, and who bears the burden of that risk. These ethical questions should be at the forefront of the system design and operation, and should be considered at every stage of the process.

A key ethical principle in safety management is the principle of proportionality, which states that the benefits of an activity should outweigh the risks associated with it (Shrader-Frechette, 1991). This principle requires a careful assessment of the potential benefits and risks of different courses of action, and a willingness to accept only those risks that are proportionate to the benefits. It also implies a responsibility to minimize risks to the extent reasonably practicable. Furthermore, the allocation of risk should be equitable, ensuring that those who benefit from an activity also bear a fair share of the associated risks. This principle must apply to all levels of the organization, and should inform all safety-related decisions.

Another important ethical consideration is the principle of informed consent, which states that individuals should be fully informed about the risks they face and given the opportunity to make autonomous decisions about whether or not to accept those risks (Beauchamp & Childress, 2019). This principle is particularly relevant in situations where individuals are exposed to risks that they may not fully understand, such as in the context of occupational safety. The principle of informed consent also implies a responsibility to provide individuals with the information and resources they need to make informed decisions about their safety.

Many thanks to our sponsor Focus 360 Energy who helped us prepare this research report.

6. Future Directions: Embracing Technology and Human Factors

Safety science is poised to undergo further transformation in the coming years, driven by advances in technology and a growing understanding of human factors. Artificial intelligence (AI) and machine learning (ML) offer promising opportunities to enhance risk assessment and incident prediction. AI/ML algorithms can analyze large datasets to identify patterns and anomalies that might otherwise go unnoticed, providing early warnings of potential safety hazards. These technologies can also be used to automate routine safety tasks, freeing up human resources for more complex and strategic activities. However, it is important to recognize the limitations of AI/ML and to ensure that these technologies are used in a responsible and ethical manner. The use of AI in safety must be carefully evaluated to ensure that it does not create new biases or exacerbate existing inequalities. The best approach is to implement these technologies as a tool to assist in risk assessment, and not as a complete replacement of human factors, analysis and decision-making.

Human-centered design will continue to play a crucial role in improving safety. This involves designing systems that are intuitive, easy to use, and that minimize the potential for human error. Human-centered design also emphasizes the importance of user feedback and participation in the design process. By involving users in the design of safety systems, it is possible to ensure that these systems are effective, acceptable, and sustainable.

Participatory approaches to safety management, which involve engaging workers and other stakeholders in the identification and mitigation of hazards, are also gaining traction. These approaches recognize that workers are often the best source of information about potential safety problems, and that their involvement is essential for creating a safe and healthy work environment. Participatory approaches can also foster a culture of safety, where individuals feel empowered to speak up about safety concerns and to take proactive steps to prevent accidents. If organizations focus on a just culture where errors can be identified and solved instead of hidden, then safety will be greatly increased. The only way a just culture can be effectively instilled is through participatory approaches from stakeholders at every level.

Many thanks to our sponsor Focus 360 Energy who helped us prepare this research report.

7. Conclusion

Safety science has evolved from a reactive discipline focused on preventing accidents to a proactive science aiming to understand and manage systemic vulnerabilities. The limitations of traditional approaches, such as root cause analysis, have become increasingly apparent in the face of complex sociotechnical systems. Resilience engineering offers a promising alternative by focusing on building a system’s capacity to adapt to unexpected events. Ethical considerations, particularly regarding the allocation of resources and the acceptance of residual risk, are also central to safety management. Future directions for safety science include the integration of AI/ML and human-centered design, as well as the adoption of participatory approaches to safety management. By embracing these new approaches, we can create safer, more resilient, and more ethical systems that protect individuals and communities from harm.

Many thanks to our sponsor Focus 360 Energy who helped us prepare this research report.

References

• Beauchamp, T. L., & Childress, J. F. (2019). Principles of biomedical ethics. Oxford University Press.
• Checkland, P. (1999). Systems thinking, systems practice: Includes a 30-year retrospective. John Wiley & Sons.
• Dekker, S. (2006). The field guide to understanding human error. Ashgate Publishing.
• Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32-64.
• Hollnagel, E. (2017). Resilience engineering: Guiding the journey from hindsight to foresight. CRC press.
• Hollnagel, E., Woods, D. D., & Leveson, N. (2006). Resilience engineering: Concepts and precepts. Ashgate Publishing.
• Hutchins, E. (1995). Cognition in the wild. MIT press.
• Leveson, N. G. (2011). Engineering a safer world: Systems thinking applied to safety. MIT press.
• Perrow, C. (1984). Normal accidents: Living with high-risk technologies. Basic Books.
• Reason, J. (1990). Human error. Cambridge University Press.
• Shrader-Frechette, K. S. (1991). Risk and rationality: Philosophical foundations for populist reforms. University of California Press.
• Snook, S. A. (2000). Friendly fire: The accidental shootdown of Black Hawks over northern Iraq. Princeton University Press.
• Woods, D. D. (2000). The alarm problem and directed attention. Ergonomics, 43(3), 237-262.