Predicting Readmissions (and other outcomes) Doesn’t Take a PhD John Showalter, MD MSIS Chief Health Information Officer University of Mississippi Medical.

Predicting Readmissions (and other outcomes) Doesn’t Take a PhD John Showalter, MD MSIS Chief Health Information Officer University of Mississippi Medical Center November 12, 2013

Section name here Objectives  Discuss why advanced mathematical modeling is not always superior to straight-forward calculations.  Describe how readily available administrative data can be used to predict risk for readmission at your institution.  Explain certainty factor analysis and how it can be applied to healthcare analytics

Section name here Punch-line  You can predict readmissions: – At the time of admission – With data you already have – Without advanced analytics software

Section name here MYCIN  Computer program that used a rule based system to suggest treatment for serious infections  Developed in the early 1970s  Outperformed specialists in treatment selection  Based on a novel method of handling uncertainty in decision making (Certainty Factors)

Section name here Certainty Factors  Designed to handle dependency between variables  Based on individual estimates of certainty  Scale: -100 (absolute certainty of no event) to 100 (absolute certainty of event)  Calculates the strength of a belief not a probability  Widely used in rule-based computer programs for a short period

Section name here Fall of Certainty Factors  Mathematically proven to be inferior to advanced conditional probabilistic models – Except for simple belief calculations  Development of Belief Networks for both simple and advanced belief calculations  Only allows forward reasoning  Infrequently used and even more infrequently mentioned in the last 20 years

Section name here Why use Certainty Factors in healthcare?  Almost all variables are dependent – Weight effects diabetes risk – Age effects heart attack risk – Treatments effect outcomes  It is designed for rule based logic systems – Almost all/if not all clinical decision support systems are rule-based systems  The math is straight-forward and can be handled in the vast majority of EHRs

Section name here Necessary Modifications to Certainty Factors  Only explore the certainty of the event occurring (i.e. 0 – 100)  Calculate Certainty Factors based on rates since data is readily available  Correlate strength of belief (Certainty Factor)with risk stratification of potential event

Section name here Predicting Readmission Study  Create a method of predicting readmissions at the time of admission  Use readily available administrative data  Compare modified certainty factor analysis to advanced machine learning algorithms  6,448 discharges for the Internal Medicine Service  30 day readmissions

Section name here Predicting Readmission Study  Used four administrative variables – Number of diagnoses bill in 1 year prior to admission – Boolean (Y/N) Hospital admission within 1 year prior to current admission ED visit within 1 year prior to current admission Outpatient clinic visit within 1 year prior to current admission  Compared Several Predictive Model – Certainty Factors – Bayesian Network – 2 Artificial Neural Networks – Support Vector Machine

Section name here Study Results – All Readmissions Certainty Factors Bayesian Network ANN Multilayer Perception ANN Radial Basis Function Number of Discharges Number of Readmissions Rate Number of Discharges Number of Readmissions Rate Number of Discharges Number of Readmissions Rate Number of Discharges Number of Readmissions Rate Low Risk 1,032 (37.6) 108 (26.5%) 10.5 % 1,754 (63.9) 217 (53.3) 12.3% 1,453 (53.0) 173 (42.3) 11.8% 1,720 (62.7) 202 (49.6) 11.7% Moderate Risk 1,441 (52.5) 212 (52.1) 14.7 % 741 (27.0) 115 (28.3) 15.5% 999 (36.4) 140 (34.4) 14.0% 741 (27.0) 115 (28.3) 15.5% High Risk 270 (9.8) 87 (21.4) 32.3 % 248 (9.0) 75 (18.4) 30.2% 291 (10.6) 95 (23.3) 32.6% 282 (10.3) 90 (22.1) 31.9% AUC0.5960.5870.5990.615

Section name here Study Results – Unplanned Readmissions* Certainty Factors Bayesian Network ANN Multilayer Perception ANN Radial Basis Function Number of Discharges Number of Readmissions Rate Number of Discharges Number of Readmissions Rate Number of Discharges Number of Readmissions Rate Number of Discharges Number of Readmissions Rate Low Risk 1,032 (37.6) 56 (18.5) 5.4% 2,335 (85.1) 216 (71.3) 9.3% 2,055 (74.9) 163 (53.8) 7.9% 2,173 (79.2) 183 (60.4) 8.4% Moderate Risk 1,441 (52.5) 165 (54.4) 11.5 % 160 (5.8) 21 (6.9) 13.1% 274 (10.0) 38 (12.5) 13.9% 279 (10.2) 34 (11.2) 12.2% High Risk 270 (9.8) 82 (27.1) 27.1 % 248 (9.0) 66 (21.8) 26.6% 415 (15.1) 102 (33.7) 24.6% 291 (10.6) 86 (28.4) 29.6% AUC0.6480.6200.6470.686 * Defined by readmission to the Internal Medicine Service

Section name here UMMC Preliminary Results – All Readmissions Certainty Factors Number of Discharges Number of Readmissions Rate Low Risk 2,566 (59.7) 84 (21.9) 3.3% Moderate Risk 1,045 (24.4) 135 (35.2) 12.9 % High Risk 675 (15.7) 165 (43.0) 24.4 % AUC0.744

Section name here Create Certainty Factor Model Certainty Factor of Usage (CF U ) Prior IPPrior OPPrior EDReadmission Rate (CF U ) Yes No YesNoYes No Yes NoYesNo Yes No Certainty Factor of Diagnosis (CF D ) Number of Diagnoses in Prior Year Readmission Rate (CF D ) 0 1-10 Greater than 10 General Equation CF T = CF 1 + CF 2 * (1 – CF 1 ) + (CF 3 *(1-(CF 1 + CF 2 * (1 – CF 1 ))))... Study CF R Cut-offs Low Risk 0–0.199 Moderate Risk 0.2–0.352 High Risk 0.353–0.6 Equation for readmssion model CF R = CF U + CF D * (1 – CF U ) Calculate CF R and then select risk level cut-offs

Section name here Certainty Factors – Potential Applications  Risk stratification based on historic data – Incidental finding on chest x-ray  Risk assessment based on current data – Mortality from infection/sepsis  Real-time alerts about changes in risk – Failure to rescue

Section name here Recap  You don’t need “Big Data” to make predictions  You don’t need a PhD to do the math  Timely, actionable knowledge is possible with Certainty Factors

Section name here Questions and Answers  Email: jshowalter@umc.edu

Predicting Readmissions (and other outcomes) Doesn’t Take a PhD John Showalter, MD MSIS Chief Health Information Officer University of Mississippi Medical.

Similar presentations

Presentation on theme: "Predicting Readmissions (and other outcomes) Doesn’t Take a PhD John Showalter, MD MSIS Chief Health Information Officer University of Mississippi Medical."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Predicting Readmissions (and other outcomes) Doesn’t Take a PhD John Showalter, MD MSIS Chief Health Information Officer University of Mississippi Medical.

Similar presentations

Presentation on theme: "Predicting Readmissions (and other outcomes) Doesn’t Take a PhD John Showalter, MD MSIS Chief Health Information Officer University of Mississippi Medical."— Presentation transcript:

Similar presentations

About project

Feedback