Is healthcare getting safer ? The challenge of measurement Charles Vincent Department of Psychology & Oxford Academic Health Science Network

Evangelists & snails ‘Run don’t walk’ ‘The correct question is whether there is a rationale for withholding critical care resources from critically ill patients outside the intensive care unit. The answer is obvious. No’ Walk, don’t run ‘In view of the limitations of the evidence and the heterogeneity of study results it seems premature to declare Rapid Response Teams as the standard of care’. Davidoff, 2011

Table 4.3Adverse events in acute hospitals in ten countries

Cutting error and harm by 50% within 5 years

UK National Reporting & Learning System Hospital Episode Statistics: 11.8M hospital admissions in England 2004/5

But incident reporting only detects 5% of harmful events

Safety interventions The challenge of scaling up

Cumulative incidence radiologically confirmed thrombosis Kreckler et al, 2010

Intensive care Operating theatre Major complication rate decreased 36% Mortality decreased 47% Post-op infection decreased 48% Central line infection rates decreased 66% Quarterly infection rate in most ICU’s <1% Estimated saving of $175 million Potentially more than 1500 lives saved Major successes in focal clinical areas

Team training and surgical mortality After controlling for baseline differences the 74 trained facilities experienced significant decrease in mortality of 18% as compared with 7% in the non trained facilities Neilly et al, JAMA 2010

Safer Patients Initiative To reduce adverse events by 50% in 24 hospitals Programme model Change elements Process measurement QI methodology Safer Patients Initiative Participating hospital site Collaborative learning Expert support

SPI programme elements: Change package

Commentaries on patient safety in the United States five years after the publication of to key reports on patient safety in 2000 were characterised by some despair at an apparent lack of progress. Our data suggest that a more encouraging story on patient safety in the NHS can now be told Benning et al, 2011

The Achievements of SPI u Inspirational and important legacy u Objectives over ambitious u Organisations in different states of readiness u First major UK safety initiative that took evaluation seriously u Simply getting basic clinical data and measures was a major challenge

Assessing safety interventions at population level

We do not know whether we are making progress or not

Temporal trends in rates of patient harm: |United States Landrigan et al, NEJM 2011

u Making Care Safer. Preventable hospital-acquired conditions would decrease by 40% compared to u This would mean 1.8 million fewer injuries to patients. u Improving Care Transitions. Preventable complications during a transition be decreased so that hospital readmissions would be reduced by 20% compared to u This would mean 1.6 million patients recovering without suffering a preventable complication requiring re- hospitalization.

Did Hospital Engagement Networks Actually Improve Care? u ‘Weak study design and methods, combined with a lack of transparency and rigour in evaluation …’ u ‘These numbers appear impressive but given the publicly available data and the approach CMS used it’s nearly impossible to tell whether the PPP actually led to better care’ (Pronovost & Jha, NEJM 2014)

u Focus on improvement u Menu of national priorities u Capacity and capability building u Measurement u Leadership u Evaluation 15 Patient Safety Collaboratives Hosted by Academic Health Science Networks

Aspiration and realism. The pace of change?

Measurement & Evaluation in Patient Safety Programmes u Our major challenge will be to demonstrate change (rather than activity) u This has bedevilled all safety programmes in NHS u Measurement is therefore our number 1 priority because: – It focuses minds and priorities – It has been the major headache for all safety programmes – The time taken to get measures in place has been consistently underestimated – It is essential for the programme teams to function effectively – It is fundamental to evaluation u We owe this to patients and carers

Information should include the perspective of patients and their families; measures of harm; measures of the reliability of critical safety processes; on practices that encourage the monitoring of safety; on the capacity to anticipate safety problems; on the capacity to respond and learn from safety information.