1. Ilona Verburg Nicolette de Keizer Niels Peek
Comparison of different statistical methods to predict Intensive Care Length of Stay
Ilona Verburg
Nicolette de Keizer
Niels Peek
Dept. Of Medical Informatics
Academic Medical Center
University of Amsterdam
The Netherlands
ESCTAIC 2012,Timisoara

2. Background and objective
Intensive Care Units (ICUs) assess their performance to improve quality and reduce costs
Background
Efficiency
of care
Effectiveness
of care
Case mix
mortality
length of stay

3. Background and objective
ICU Length of stay is influenced by case mix.
Example:
Length of stay (mean) 10 days 5 days
Age (mean)
Medical vs surgical 80% medical 40% medical
admission type (%) 20% surgical 60% surgical

4. Background and objective
ICU
Observed outcome
Compare
Case mix
Predictive model
Case mix
Expected outcome

5. Background and objective
Models exist to predict ICU mortality (example APACHE IV)
Few models exist to predict ICU Length of Stay (LoS)
No consensus about best modelling method
Objective
Compare the performance of different statistical regression methods to predict ICU LoS.

6. Data NICE registry Dutch National Intensive Care Evaluation (NICE)
Registry of ICU admissions in the Netherlands (since 1996)
All admissions from (voluntary) participating ICUs (>90%)
Evaluating (systematically) the effectiveness and efficiency of ICUs in the Netherlands
Identifying quality of care problems
Quality assurance
Database

7. Data Data Patients admitted to ICUs participating NICE 2009 - 2011
Included patients
94,251 (42.4%)
admissions
Exclusion criteria
APACHE IV exclusion criteria
elective surgery
81,190 (86.1%)
survivors
13,061 (13.9%)
non-survivors

8. Length of stay Distribution of Length of Stay in fractional days
ICU non-survivors (n= 13,061)
ICU survivors (n= 81,190)
Median: 2.4 (days)
Mean: 5.9
Standard deviation: 10.2
Maximum: 139.0
Median: 1.7 (days)
Mean: 4.2
Standard deviation: 8.2
Maximum:
Plaatjes nog vervangen en mediaan en mean etc nog invullen….

9. ICU Length of Stay
Distribution of discharge time

10. Modeling ICU length of stay
Different methods to model ICU length of stay (in fractional days)
Ordinary least square (OLS) regression
LoS and Log-transformed LoS
Most frequently used method in literature

11. Modeling ICU length of stay
Different methods to model ICU length of stay (in fractional days)
Ordinary least square (OLS) regression
LoS and Log-transformed LoS
General linear models (GLM)
Gaussian - difference with OLS is the log link function
Gamma - LoS time until discharge
- depending on chosen parameters positively skewed
Poisson - LoS count data
`-depending on chosen parameters positively skewed
- property: expectation = variance → overdispersion
Negative binomial - count data
-depending on chosen parameters positively skewed
- generalisation of poisson

12. Modeling ICU length of stay
Different methods to model ICU length of stay (in fractional days)
Ordinary least square (OLS) regression
LoS and Log-transformed LoS
General linear models (GLM) 4 different families
Gaussian
Gamma
Poisson
negative binomial
Cox proportional Hazard (Cox PH) regression
No assumptions on the shape of the distribution
Omits the need of transform the outcome

13. Modeling ICU length of stay
Selection of covariates
Starting with large set of variables
Known relationship with LoS (literature)
Stepwise backwards elimination of variables
Included case mix
Demographics
Age
Gender
Admission type
Diagnoses (APACHE IV)
Severity of illness (APACHE IV severity-of-illness score)
Different comorbidities (21)

14. Validation Performance measures Good prediction
Squared Pearson correlation = R2 =
High ↑
Root Mean squared prediction error (RMSPE) =
Low ↓
Low ↓
- or +
Relative BIAS =
Relative mean absolute prediction error (MAPE) =
Low ↓

15. Validation Validation Performance measures calculated on original data
Correcting for optimistic bias
100 bootstrap samples

16. Results coefficients Covariates survivors OLS reg los OLS reg log los
GLM: gaussian
GLM: poisson
GLM: negative binomial
GLM: Gamma
Cox PH
chronic dialysis
-1.04
-0.16
-0.25
-0.26
-0.28
0.31
cva
0.74
0.1
0.13
0.18
0.26
-0.3
diabetes
-0.34
-0.01
-0.07
-0.06
-0.04
0.03
resperatory insufficient
0.38
0.06
0.09
0.15
-0.11
spline Aps (1)
5.55
0.64
1.74
1.65
1.61
-1.52
spline Aps (2)
11.07
1.09
3.16
2.78
2.64
-2.57
spline Aps (3)
15.98
0.99
2.07 2
2.08
-1.79
Covariates non-survivors
OLS reg los
OLS reg log los
GLM: gaussian
GLM: poisson
GLM: negative binomial
GLM: Gamma
Cox PH
chronic dialysis
0.15
0.08
cva
-0.68
-0.18
-0.15
-0.12
0.09
diabetes
0.35
0.03
0.05
0.06
-0.05
resperatory insufficient
-0.51
-0.03
-0.11
-0.1
-0.09
0.07
spline Aps (1)
-5.59
-0.43
-0.94
-0.84
-0.8
0.7
spline Aps (2)
-6.08
-0.73
-1.09
-1.26
-1.53
-1.55
1.54
spline Aps (3)
-6.47
-1.64
-1.76
-1.87
-1.88
1.83

17. Results validation ICU survivors Mean observed > mean expectedR2
RMSPE
Relative BIAS
Relative MAPE
OLS regression (LoS)
0.174
7.448
0.008
0.812
OLS regression (log(LoS))
0.183
7.714
-0.400
0.674
GLM Gaussian
0.197
7.335
0.001
0.771
GLM Poisson
0.194
7.349
0.000
0.769
GLM Negative Binomial
0.186
7.388
0.005
0.773
GLM Gamma
0.184
7.407
Cox PH regression
0.097
9.002
-0.693
0.938
Mean observed > mean expected
Underestimation of mean LoS

18. Results validation ICU non-survivors R2 RMSPE Relative BIASR2
RMSPE
Relative BIAS
Relative MAPE
OLS regression (LoS )
0.107
9.618
0.005
0.891
OLS regression (log(LoS))
10.213
-0.510
0.762
GLM Gaussian
0.134
9.462
-0.009
0.868
GLM Poisson
0.128
9.504
0.000
0.872
GLM Negative Binomial
0.12
9.545
-0.001
GLM Gamma
0.112
9.602
0.877
Cox PH regression
0.075
11.388
-0.808
0.906

19. Conclusion and discussion
Difficult to predict ICU LoS
Influenced by admission and discharge policy
Seasonal pattern for admission and discharge time
Skewed to the right
GLM models shows best performance
Poorest performance found for Cox PH regression
Large relative bias was found for OLS regression of log-transformed LoS
Differences in performance between models not statistically tested

20. Conclusion and discussion
Similar study for CABG patients (Austin et al.), with comparable results
Different patient type
Different distribution of length of stay
Future research
Different models for survivors and non-survivors
combining with mortality in one prediction
Statistical methods to predict ICU LoS
developing a model for benchmarking purposes

21. Thank you for your attention!
Questions?

22. APACHE IV Exclusiecriteria
Age < 16
ICU admission < 4 hours
Hospital admission >365 days
Died during admission
Readmissions
Admissions from CCU/IC other hospital
No diagnose
Burns
Transplantations
Missing hospital discharge