) Benchmarking Critical Care Outcomes: Using data to drive effectiveness and efficiency Thomas L. Higgins MD MBA Vice Chair for Clinical Affairs, Department of Medicine, Baystate Medical Center, Springfield MA Professor of Medicine, Surgery & Anesthesiology, Tufts University School of Medicine Resource Utilization Graph from Nathanson et al, Crit Care Med 2007; 35:1853 Project IMPACT Data Hospitals within control limits Better than expected Mortality and LOS Driving Change Normalized ratios can be created for any outcome: e.g.: ventilator days In this example, ventilator days are higher than predicted, indicating an opportunity for improvement Interventions could include education, institution of “daily wake-up”, attention to VAP and CLABSI, respiratory therapy protocols, or twice- daily weaning trials Driving Change Using ICU Benchmarking Tools Morbidity and mortality –Evidence-based bundles / ordersets; CPOE, medication scanning; alerts, early warning –Excess length-of-stay –Admission, discharge, triage policies –Open versus closed units –Ventilator weaning and sedation practices Ventilator-associated Pneumonia –Ventilator “bundle” of care including HOB elevation –Respiratory therapy equipment and change-out policies Catheter-related Bloodstream Infections –Attention to technique and tools –Operator training restrictions Length of Stay Reduction MICU + SICU Patients, BMC, excludes Heart & Vascular (CVICU, CCU) Central Line Associated Blood Stream Infections (CLABSI) Current ICU Benchmarking Tools Summary Measuring ICU performance requires a balanced scorecard Outcomes must be severity-adjusted –Tools include APACHE, MPM, SAPS –Endpoints include mortality, LOS –Normalized ratios/benchmarking can drive change Readmission rates must also be severity-adjusted but once adjusted do not correlate with case-mix adjusted mortality or other quality measures, raising questions about CMS use of metric –Kramer et al, Crit Care Med 2013; 41:24-33 Quality metrics also include CLABSI, VAP, complications and patient satisfaction Track employee engagement as well as family satisfaction Academic institutions may also track research productivity, teaching evaluations, publications Other Domains of Interest Clinical Quality –Patient and family satisfaction – H-CAHPS Scores Human Capital –Engagement, turnover, morale – Gallup EmployeeSurvey Financial Performance –Revenue and Costs (Part A and Part B) – Income Statement –Resource Utilization by provider – Premier Database Academics: Research and Education – –Grant funding, number of publications, faculty teaching evaluations (New Innovations) ModelnAUROCHLGOF, p APACHE-II (1985)5, nr APACHE-IV (2002-3)110, ICNARC ( )216, <0.001 MPM 0 -II (1993)12, MPM 0 -III (2001-4)124, SAPS-II (1993)13, SAPS-III (2002)16, Standardized Mortality Ratio (SMR) Observed Risk-Adjusted Mortality SMR = Expected Risk-Adjusted Mortality Values 2 SD > 1.0 may indicate poor performance Values 2 SD < 1.0 indicated superior performance Patient DiagnosisPredictedActual DKA2%0 Pneumonia12%0 Asthma10%0 Acute MI24%0 Septic Shock30%1 Pneumonia12%0 Heart Failure15%0 Septic Shock30%0 Ruptured AAA65%1 Heart Failure15%0 AVERAGE: Example of calculating SMR for a hypothetical ICU One patient each; 10 diagnoses SMR for this ICU= Observed (20%) Predicted (21.5%) = 0.93 Major Domains of Interest Clinical Quality –Standardized mortality rate (observed/expected) –ICU and hospital lengths of stay –Complications (CR-BSI, VAP, “never” events) –Patient and family satisfaction Human Capital –Engagement, turnover, morale Financial Performance –Revenue and Costs (Part A and Part B) –Resource Utilization by provider Academics: Research and Education Critical Care Medicine in the US: Big business, and growing 93,955 CCM beds in 3,150 Hospitals (increasing 1%/yr ) 23.2 million patient days (10.6% increase over 5 years) Cost per day: $3518 (30.4% increase over 5 years) Total costs: $ 81.7 Billion (44.2% increase over 5 years) –Critical Care accounts for 13.4% of hospital costs –4.1% of national health expenditures –0.66% of GDP (rate of increase = 3.6% per year) Halpern and Pastores, Crit Care Med 2010; 38:67-71 Hospital Mortality Rate for ICU patients: ~6 to 19% –13.8% in 124,855 patients, Project IMPACT ( ) Higgins et al, Crit Care Med 35:827, 2007 –13.5% in 44,288 patients, APACHE-IV validation ( ) Zimmerman et al, Crit Care Med 34:1297, 2006 Accurate Risk Stratification Needed Mortality outcomes are highly dependent on presenting patient condition –Unadjusted results misleading Mortality rate for DKA <2% Mortality rate for septic shock ~30% Case-mix thus affects unadjusted overall mortality rate Adjusted data is required for internal Quality Improvement efforts Risk stratification helpful (but not infallible) for individual patient prognosis Risk-adjustment models must meet criteria for discrimination (area under ROC >0.80) and calibration (non-significant HL-GOF) Who wants to know? Patients, Families, Physicians, Administrators, Insurers, the media….. AUROC = area under receiver operating curve, ideally >0.80) HLGOF = Hosmer-Lemeshow Goodness of Fit, ideally >0.05 Worse than expected resource utilization (Length of Stay)