1. ©2014 MFMER | slide-1 So many questions, So much data, So little time A Journey with Surgical Outcome Data Alison M. Knight, P.E. IIE 2014 National Conference

2. ©2014 MFMER | slide-2 Agenda Tutorial in quality databases Background information on project Strategy and process for report creation Example of report and metrics chosen

3. ©2014 MFMER | slide-3 Quick Tutorial in Surgical Quality Databases The dirty secrets THEY don’t tell you

4. ©2014 MFMER | slide-4 Lesson #1: Quality databases are NOT created equal Clinical Abstracted vs Administrative Data Outcome Definition Data Audit ProcessSample vs. Population Database Attributes

5. ©2014 MFMER | slide-5 Lesson #2: Use all databases, but choose metrics carefully Mortality & Length of Stay Administrative Data & All Cases Surgical Site Infection & Urinary Tract Infection Clinically Abstracted Data & Sample of Cases Example #1: Example #2:

6. ©2014 MFMER | slide-6 Lesson #3: Data denial will happen, Be ready! Five Stages of Data Denial* 1.The data is wrong 2.The data is old 3.We have changed already 4.This is a different place 5.We tried that already *Borrowed from Dr. Brian Postl, CEO of the Winnipeg Regional Health Authority Presentation

7. ©2014 MFMER | slide-7 Let’s set the stage

8. ©2014 MFMER | slide-8 The surgical practice is frustrated Each surgical specialty pays a portion New costs Most surgical specialties believed this quality data provided little value No perceived value Surgical specialties didn’t believe the data Data is not trustworthy

9. ©2014 MFMER | slide-9 Some background facts…

10. ©2014 MFMER | slide-10 What is NSQIP? American College of Surgeons (ACS)’s National Surgical Quality Improvement Program History Started in the Veterans Administration(1994)* Open to other hospitals (2004)* Mayo Clinic joined in 2006 Clinically abstracted random sample of cases *ACS NSQIP website: http://site.acsnsqip.org/program-specifics/nsqip-history/

11. ©2014 MFMER | slide-11 What types of information is included and how is it collected? Uniform Operational Definitions 135 Patient Variables 30 day Post-op Assessment Audit Process

12. ©2014 MFMER | slide-12 Data Delivery to the Practice Twice a year the practice chair received a “canned report” from NSQIP Each specialty had risk-adjusted odds ratios Ranged in granularity depending on specialty Highlighted the specialty’s decile rankings (1-10) Decile*: divides the distribution of the variable into ten groups having equal frequencies. *Source: dictionary.com

13. ©2014 MFMER | slide-13 The Project

14. ©2014 MFMER | slide-14 Project Goals Educate Educate providers on database facts Work through the stages of denial Provide Provide data at more detailed level Point providers to “hot spots” Change Aid specialty in developing data- driven quality projects

15. ©2014 MFMER | slide-15 Identified the Challenges NSQIP costs money, value not recognized “Why should I pay for NSQIP if I am not using it?” Lack of understanding of data collection process: “Documentation errors are probably driving this data” Decile rankings are the focus “These rankings can’t possibly be right, our patients are sicker and we do more complicated procedures” Specialty practices only were recognized for negative quality “Leadership just doesn’t understand our practice”

16. ©2014 MFMER | slide-16 The Plan to Create a Meaningful Dashboard Goals Practice Input Use in Research Address Concerns De-emphasize Rankings Focus on Good & Bad Make Data Actionable Dispel Myths

17. ©2014 MFMER | slide-17 The process of development: PDSA-Like Cycle One specialty at a time Analysis done by hand Review report for completeness Obtain feedback from each specialty for improvements CreateReviewPresentRevise

18. ©2014 MFMER | slide-18 A Sample Report This data has been created for example purposes only and does not reflect real Mayo Clinic data.

19. ©2014 MFMER | slide-19 Components of the Report Generic information from NSQIP Data collection and risk-adjustment information Specialty specific information Without risk adjustment Example research articles using NSQIP data Recommendations to practice

20. ©2014 MFMER | slide-20 Demographic Data Example* *Data is fictitious and does not reflect Mayo Clinic performance.

21. ©2014 MFMER | slide-21 Volume Analysis by CPT Code Example* *Data is fictitious and does not reflect Mayo Clinic performance.

22. ©2014 MFMER | slide-22 Occurrences: Raw Rates Example* *Data is fictitious and does not reflect Mayo Clinic performance.

23. ©2014 MFMER | slide-23 Case Volume versus Occurrence Volume* *Data is fictitious and does not reflect Mayo Clinic performance.

24. ©2014 MFMER | slide-24 Rankings Example* *Data is fictitious and does not reflect Mayo Clinic performance.

25. ©2014 MFMER | slide-25 Recommendations and Next Steps Based on the data, Outcome #1 and Outcome #3 should be addressed. CPT Code Category #2 shows opportunity for improvement Additional detailed information by request All case data pulled for specific CPT code categories and/or cases with specific outcomes Logistic regression analysis to recognize factors that contribute certain outcomes

26. ©2014 MFMER | slide-26 Current State and Next Steps

27. ©2014 MFMER | slide-27 Where are we now? Stable report that practices find helpful Worked through 5 stages of data denial Creating automated process to provide information in a timely manner Creating processes for follow-up and practice accountability

28. ©2014 MFMER | slide-28 Summary All databases are not created equal Use the correct type of database depending on the information needed Address data denial De-emphasize rankings Create a report that magnifies the “hot spots” for practices to launch quality projects

29. ©2014 MFMER | slide-29 Questions & Discussion