1. The Measurement and Monitoring of Safety: Drawing together academic evidence and practical experience to produce a framework for safety measurement and monitoring April 2013 A Report from The Health Foundation based on lessons from the UK’s National Health System

2. BACKGROUND In 2000 the paper, “An Organization with a Memory” shed light on the prevalence of harm, medical errors and safety 10 years later, current metrics still cannot accurately illustrate if we are safer today than we were ten years ago THIS PAPER is an effort to do just that – to develop a broad framework to measure safety, with three main goals: 1.Address differing technical and conceptual issues inherent to measuring safety 2.Understand how metrics can be used to improve safety 3.Disseminate this information Findings draw on published research, case studies, public data and interviews

3. DEFINING SAFETY “The avoidance, prevention and amelioration of adverse outcomes or injuries stemming from the process of healthcare” Safety has been previously narrowly focused – need to account for risk, harm, structural components, system reliability and both system and staff resiliency to process deviation (error) Safety vs. Quality: Both are deviation from standard processes – safety issues are deviations that result in harm Safety issues are also those that are considered “unacceptable” – this emotional label, while helpful in raising expectations, has placed the focus on never events as opposed to systems issues Therefore, when measuring must be careful not to separate quality and safety too much – many errors don’t result in harm and harm is often not the result of an error Need to look towards likelihood of harm and hazards

4. DEFINING SAFETY “The avoidance, prevention and amelioration of adverse outcomes or injuries stemming from the process of healthcare” The perils of safety as self-reported events: even when consistently and precisely reported, every person, unit and institution has varying standards for what an event is Helpful for self-contained improvement, but not across the board comparison EXAMPLE: Matching Michigan A study on the sources of CLI’s reproduced by the NHS shows that differing self-reports don’t necessarily come from the desire for better results, but simply the infinite difference in individual perceptions – it is as much a social practice as a technical practice

5. LESSONS from NHS History The 2000 report “An organization with a memory” incited an evolution of safety interventions, measures and reporting similar to that in the US – ranging from public reporting and hospital rating to never events with financial incentives embedded into the delivery system The NHS has also developed A Safety Thermometer and Outcomes Framework to measure the prevalence and incidence within four areas: pressure ulcers, falls, urinary tract infections, and venous thromboembolism BUT, What has been neglected? Measurement and definition of harm Behaviors Risk and Hazard

6. KEY LESSONS from OTHER INDUSTRIES and SAFETY FRAMEWORKS Incident report data should be used as only a corollary to more automated system metrics The primary purpose of incident data should be to uncover the systemic hazards in place, at which point these more leading indicators become the primary focus of measurement Conceptual safety models echo this approach - events are simply evidence of a more deeply embedded (latent) system problem Resilience is a critical factor in the dynamic “state” of safety and can be considered with or separate from broader systems measures that will make themselves apparent through event monitoring

7. Safety Measuring and Monitoring: Five Classes of Information

8. 1) Measurement of Harm: Has patient care been safe in the past? both physical and psychological measures physical harm requires typography: treatment specific, over-treatment, general harm, failure to provide treatment or missed/delayed diagnosis suggestion: global trigger tool

9. Safety Measuring and Monitoring: Five Classes of Information 2) Reliability: are our clinical systems and processes reliable? Both systems and behavior measures Hard wiring processes and behaviors – process measures and compliance fall here

10. Safety Measuring and Monitoring: Five Classes of Information 3) Sensitivity to Operations: Is Care Safe Today? The information and capacity to monitor safety on an hourly/day to day basis Staff awareness – collective mindfulness Alert and knowledgeable of changing risk (safety walk- rounds) Use Patient Safety interviews, designated officers – these are structural metrics Continuous feedback

11. Safety Measuring and Monitoring: Five Classes of Information 4) Anticipate and Preparedness: Will care be safe in the Future? The ability to both anticipate and prepare for problems Human reliability analysis or failure modes and effect analysis Safety Culture NOTE – acknowledged this area needs most follow up research

12. Safety Measuring and Monitoring: Five Classes of Information 5) Integration and Learning: Are we responding and improving? The ability to respond to and learn from safety information Integrating safety information/data into day to day operations – either through huddles or more automatic processes

13. Safety Measuring and Monitoring: TEN GUIDING PRINCIPLES 1.A single measure of safety is a fantasy 2.Safety monitoring is critical and does not receive enough attention 3.Anticipation and proactive approaches to safety 4.Integration and Learning: invest in technology and data analysis expertise 5.Mapping safety measurement and monitoring across organization 6.A blend of externally required metrics and local development 7.Clarity of purpose is needed when developing safety measures 8.Empowering and devolving responsibility for the development and monitoring of safety metrics is essential 9.Collaboration between regulators and regulated is critical 10. Beware of perverse incentives

14. Questions and Discussion Link to Full Report: http://www.health.org.uk/publications/the-measurement-and- monitoring-of-safety/