Presentation on theme: "Promise & limitations of surgical checklists: How can we effectively use them to improve the quality of surgical care? Shawn J. Rangel, MD, MSCE & J. William."— Presentation transcript:
Promise & limitations of surgical checklists: How can we effectively use them to improve the quality of surgical care? Shawn J. Rangel, MD, MSCE & J. William Sparks, MD Northeast Regional Patient Safety & Quality Improvement Conference February 5 th, 2011
Reality check- IHI map
Even the stars are using it!
So, will the WHO checklist save mankind?
Reality check: one size does not fit all…
Review evidence supporting surgical safety checklists Attitudes toward the safety checklist at (CHB survey) Strategies for improving checklist utilization & relevance Next steps: IT, custom checklists & beyond… Outline of today’s discussion
London, UK EUROEMRO WPRO I SEARO AFRO PAHO I Amman, Jordan Toronto, Canada New Delhi, India Manila, Philippines Ifakara, Tanzania WPRO II Auckland, NZ PAHO II Seattle, USA 8 Evaluation Sites
Methods 1 to 4 operating rooms targeted at each site 18-item checklist implemented (sign-in, time-out, sign-off) Pre-post intervention study design (general surgery cases) Primary outcome measure: aggregate 30-day major complication rate (NSQIP* defined) *National Surgical Quality Improvement Project
Results: impact on morbidity & mortality (3 month comparison periods) BaselineChecklistP value Cases3,7333,955- Death1.5%0.8%0.003 Any complication11.0%7.0%<0.001 Surgical site infection6.2%3.4%<0.001 Unplanned Reoperation2.4%1.8%0.047
Limitations of the study Unknown influence of the Hawthorne effect Unable to prove causality (non-randomized design) Effect size may be exaggerated (developing nations) Only one of the eight centers was in the U.S. Pediatric patients not included in analysis
Intervention The comprehensive “SURgical PAtient Safety System”:
Methods SURPASS implemented at 6 tertiary-care hospitals Pre-post intervention study design (3 month periods) 12 adverse event categories audited Outcomes compared with five “control” hospitals
Results: impact on morbidity & mortality (3 month comparison periods) Baseline (n=3,760) Checklist (n=3,820) P value Mortality1.5%0.8%- Any complication15.4%10.6%<0.001 Complications/100 cases27.3%16.7%<0.001 Use of the checklist was associated with a significant reduction in complication rates for 10 of the 12 (82%) event categories in the study
But, kids are not small adults!
Pediatric Safe Surgery Collaborative Children’s Hospital Boston Shawn J. Rangel (Study PI) Beth K. Norton (Co-study PI) Jessica Baxter Texas children’s Hospital Thomas Luerssen (site PI) Carrie Smith-Bruce Riley Children’s Hospital Fred Rescorla (Co-site PI) Charles Leys (Co-site PI) Margo Regas Denver Children’s Hospital Tammy Woolley (Site PI) Jenae Nieman Children’s Healthcare Atlanta Kurt Heiss (Site PI) Kawana Mitchell Children’s National Med Center Kurt Newman (Co-site PI) Rahul Shah (Co-site PI) Andrea Ewing-Thomas Children’s Hospital of Philadelphia Peter Mattei (Site PI) Lisa Czyzewski
Collaborative process Identification & recruitment of checklist champions Development of site-specific checklist Plan for piloting on small scale Obtaining buy-in from hospital leadership & peers Full implementation OR-wide Develop internal plan for auditing compliance
General study design Seven hospitals included as “checklist” implementers All inpatient procedures from 7 surgical services included Cardiac, General, Neuro, Ortho, Plastics, Oto & Urology Pre-post intervention comparison design (9 months) Primary endpoint: Aggregate 30-day adverse event rate PHIS database used to identify events Results compared against seven control hospitals
Patient characteristics (demographics) Checklist hospitals (n=7)Control Hospitals (n=7) Pre-intervention (n=19,867) Post-intervention (n=18,850) Pre-intervention (n=15,616) Post-intervention (n=15,298) Age (mean years) Sex (% male) Insurance status : Government (%) Private (%) Other (%) Race/ethnicity: African American(%) Hispanic (%) Case Mix Index Acuity of procedure (% emergent)
Rate of any adverse event Incidence (%) P=0.064 P=0.364
MORTALITY RR of death with checklist utilization: 0.73 (95%CI: ) Incidence (%) p=0.758 p=0.018
MORTALITY (emergent procedures) RR of death with checklist utilization: 0.58 (95%CI: ) Incidence (%) p=0.724 p=0.029
MORTALITY (ICU admissions) RR of death with checklist utilization: 0.57 (95%CI: ) Incidence (%) p=0.731 p=0.002
Limitations of the study Unknown influence of the Hawthorne effect Unable to prove causality (non-randomized design) Reliance on administrative data for outcomes analysis Variation/degree of checklist compliance unknown
What can be concluded from the available data? USE OF A SURGICAL SAFETY CHECKLIST CAN SAVE LIVES !!!!!!
So then, how do people feel about using a surgical safety checklist? --CHB Checklist Survey-- Multidisciplinary targeting (3-headed monster!) Assess attitudes towards the checklist Gain insight on CHB’s current safety culture Obtain feedback for improving checklist utility
Checklist survey: Responses by specialty (n=177)
Has the checklist improved safety? (response = yes) Proportion of responders (%) Chi 2, p=0.948
How has the checklist improved safety? Proportion of responders (%)
Have you witnessed an error or complication prevented by the checklist? (response=yes) Proportion of responders (%) Chi 2, p=0.048
Has the checklist improved efficiency? (response=yes) Proportion of responders (%) Chi 2, p=0.110
Would I want the checklist used for my child? (response=yes) Proportion of responders (%) Chi 2, p=0.122
Content adequacy of current checklist? Proportion of responders (%) Chi 2, p=0.987
So, is everyone in love with the checklist?? “This checklist is bullsh&! and just reinforces the Betty Crocker approach to medicine !!” “This is probably the most important surgical safety intervention we could ever implement !!”
“This is stupid- we do this all the time anyway” “This is stupid- the checklist does not apply to my cases” Blood products available? Imaging reviewed? Special equipment available? DVT prophylaxis considered? IV access adequate? Root causes of “checklist fatigue”
So then, how can we improve the effectiveness of our checklist? Implementation of forcing cues into work flow Incorporation of a more effective auditing system Transition to a “quality”-centered checklist paradigm Development of customized checklists
Change in the checklist paradigm: transitioning from “safety” to “quality” Surgical Quality Surgical Quality Value-based Efficient Effective Safe
Variation in the use of surgical antibiotic prophylaxis for common pediatric procedures
How can we accomplish these goals?
Take home lessons…. EFFECTIVE use of surgical checklists CAN SAVE LIVES! Checklist MUST be team-based and emphasize communication! Checklists HAVE to be developed with input from ALL stakeholders Leadership ABSOLUTELY has to be on board!
YOU ARE THE FUTURE OF SAFETY CULTURE!!! And finally….