Ilona Verburg Nicolette de Keizer Niels Peek Comparison of different statistical methods to predict Intensive Care Length of Stay Ilona Verburg Nicolette de Keizer Niels Peek Dept. Of Medical Informatics Academic Medical Center University of Amsterdam The Netherlands ESCTAIC 2012,Timisoara

Background and objective Intensive Care Units (ICUs) assess their performance to improve quality and reduce costs Background Efficiency of care Effectiveness of care Case mix mortality length of stay 4-10-2012

Background and objective ICU Length of stay is influenced by case mix. Example: Length of stay (mean) 10 days 5 days Age (mean) 68 57 Medical vs surgical 80% medical 40% medical admission type (%) 20% surgical 60% surgical 4-10-2012

Background and objective ICU Observed outcome Compare Case mix Predictive model Case mix Expected outcome 4-10-2012

Background and objective Models exist to predict ICU mortality (example APACHE IV) Few models exist to predict ICU Length of Stay (LoS) No consensus about best modelling method Objective Compare the performance of different statistical regression methods to predict ICU LoS. 4-10-2012

Data NICE registry Dutch National Intensive Care Evaluation (NICE) Registry of ICU admissions in the Netherlands (since 1996) All admissions from (voluntary) participating ICUs (>90%) Evaluating (systematically) the effectiveness and efficiency of ICUs in the Netherlands Identifying quality of care problems Quality assurance Database 4-10-2012

Data Data Patients admitted to ICUs participating NICE 2009 - 2011 Included patients 94,251 (42.4%) admissions Exclusion criteria APACHE IV exclusion criteria elective surgery 81,190 (86.1%) survivors 13,061 (13.9%) non-survivors 4-10-2012

Length of stay Distribution of Length of Stay in fractional days ICU non-survivors (n= 13,061) ICU survivors (n= 81,190) Median: 2.4 (days) Mean: 5.9 Standard deviation: 10.2 Maximum: 139.0 Median: 1.7 (days) Mean: 4.2 Standard deviation: 8.2 Maximum: 326.6 Plaatjes nog vervangen en mediaan en mean etc nog invullen…. 4-10-2012

ICU Length of Stay Distribution of discharge time

Modeling ICU length of stay Different methods to model ICU length of stay (in fractional days) Ordinary least square (OLS) regression LoS and Log-transformed LoS Most frequently used method in literature 4-10-2012

Modeling ICU length of stay Different methods to model ICU length of stay (in fractional days) Ordinary least square (OLS) regression LoS and Log-transformed LoS General linear models (GLM) Gaussian - difference with OLS is the log link function Gamma - LoS time until discharge - depending on chosen parameters positively skewed Poisson - LoS count data `-depending on chosen parameters positively skewed - property: expectation = variance → overdispersion Negative binomial - count data -depending on chosen parameters positively skewed - generalisation of poisson 4-10-2012

Modeling ICU length of stay Different methods to model ICU length of stay (in fractional days) Ordinary least square (OLS) regression LoS and Log-transformed LoS General linear models (GLM) 4 different families Gaussian Gamma Poisson negative binomial Cox proportional Hazard (Cox PH) regression No assumptions on the shape of the distribution Omits the need of transform the outcome 4-10-2012

Modeling ICU length of stay Selection of covariates Starting with large set of variables Known relationship with LoS (literature) Stepwise backwards elimination of variables Included case mix Demographics Age Gender Admission type Diagnoses (APACHE IV) Severity of illness (APACHE IV severity-of-illness score) Different comorbidities (21) 4-10-2012

Validation Performance measures Good prediction Squared Pearson correlation = R2 = High ↑ Root Mean squared prediction error (RMSPE) = Low ↓ Low ↓ - or + Relative BIAS = Relative mean absolute prediction error (MAPE) = Low ↓ 4-10-2012

Validation Validation Performance measures calculated on original data Correcting for optimistic bias 100 bootstrap samples 4-10-2012

Results coefficients Covariates survivors OLS reg los OLS reg log los GLM: gaussian GLM: poisson GLM: negative binomial GLM: Gamma Cox PH chronic dialysis -1.04 -0.16 -0.25 -0.26 -0.28 0.31 cva 0.74 0.1 0.13 0.18 0.26 -0.3 diabetes -0.34 -0.01 -0.07 -0.06 -0.04 0.03 resperatory insufficient 0.38 0.06 0.09 0.15 -0.11 spline Aps (1) 5.55 0.64 1.74 1.65 1.61 -1.52 spline Aps (2) 11.07 1.09 3.16 2.78 2.64 -2.57 spline Aps (3) 15.98 0.99 2.07 2 2.08 -1.79 Covariates non-survivors OLS reg los OLS reg log los GLM: gaussian GLM: poisson GLM: negative binomial GLM: Gamma Cox PH chronic dialysis 0.15 0.08 cva -0.68 -0.18 -0.15 -0.12 0.09 diabetes 0.35 0.03 0.05 0.06 -0.05 resperatory insufficient -0.51 -0.03 -0.11 -0.1 -0.09 0.07 spline Aps (1) -5.59 -0.43 -0.94 -0.84 -0.8 0.7 spline Aps (2) -6.08 -0.73 -1.09 -1.26 -1.53 -1.55 1.54 spline Aps (3) -6.47 -1.64 -1.76 -1.87 -1.88 1.83 4-10-2012

Results validation ICU survivors Mean observed > mean expected   R2 RMSPE Relative BIAS Relative MAPE OLS regression (LoS) 0.174 7.448 0.008 0.812 OLS regression (log(LoS)) 0.183 7.714 -0.400 0.674 GLM Gaussian 0.197 7.335 0.001 0.771 GLM Poisson 0.194 7.349 0.000 0.769 GLM Negative Binomial 0.186 7.388 0.005 0.773 GLM Gamma 0.184 7.407 Cox PH regression 0.097 9.002 -0.693 0.938 Mean observed > mean expected Underestimation of mean LoS 4-10-2012

Results validation ICU non-survivors R2 RMSPE Relative BIAS   R2 RMSPE Relative BIAS Relative MAPE OLS regression (LoS ) 0.107 9.618 0.005 0.891 OLS regression (log(LoS)) 10.213 -0.510 0.762 GLM Gaussian 0.134 9.462 -0.009 0.868 GLM Poisson 0.128 9.504 0.000 0.872 GLM Negative Binomial 0.12 9.545 -0.001 GLM Gamma 0.112 9.602 0.877 Cox PH regression 0.075 11.388 -0.808 0.906 4-10-2012

Conclusion and discussion Difficult to predict ICU LoS Influenced by admission and discharge policy Seasonal pattern for admission and discharge time Skewed to the right GLM models shows best performance Poorest performance found for Cox PH regression Large relative bias was found for OLS regression of log-transformed LoS Differences in performance between models not statistically tested 4-10-2012

Conclusion and discussion Similar study for CABG patients (Austin et al.), with comparable results Different patient type Different distribution of length of stay Future research Different models for survivors and non-survivors combining with mortality in one prediction Statistical methods to predict ICU LoS developing a model for benchmarking purposes 4-10-2012

Thank you for your attention! Questions? 4-10-2012

APACHE IV Exclusiecriteria Age < 16 ICU admission < 4 hours Hospital admission >365 days Died during admission Readmissions Admissions from CCU/IC other hospital No diagnose Burns Transplantations Missing hospital discharge 4-10-2012