1. #HASummit14 Session #23 There’s A 90% Probability That Your Son Is Pregnant: Predicting The Future Of Predictive Analytics In Healthcare Dale Sanders SVP Strategy Health Catalyst

2. #HASummit14 To what degree is your organization using predictive analytics to improve care were reduce cost? a) We are not using any predictive analytics, that I know about b) We are experimenting with predictive analytics in small use cases, but as yet have seen no improvements in care or cost c) We are using predictive analytics in a small number of use cases and the results have been positive d) We are using predictive analytics in a large number of use cases and the results have been positive e) Unsure or not applicable Poll Question #1 2

3. #HASummit14 Acknowledgements Dr. Eric Siegel, Columbia University Ron Gault, Aersospace Corporation 3

4. 4

5. #HASummit14 Key Themes Today 1. Action Matters: Predictive analytics (PA) without actions and interventions are useless 2. Human Unpredictability: Humans behavior, like the weather, is inherently difficult to predict with a computer 3. Socio-Economics: Most of healthcare’s highest risk root causes lie outside the care delivery system’s ability to intervene 4. Missing Data: We are missing key data in healthcare, particularly clinical outcomes data, required for accurate predictive models… so we need to leverage collective wisdom of experts until we close the data gap 5. Wisdom of Crowds: In the pursuit of objectivity of analytics, don’t forget the wisdom of subjective experts sitting right next to you 6. Social Controversy: Even with accurate PA, are we socially prepared to act? Do we want to know? Are we intruding on people’s future? 5

6. #HASummit14 Common Concepts & Provocative Thoughts 6

7. #HASummit14 Man vs. Machine 7 Man + Machine Subjective Objective

8. #HASummit14 Financial Industry Got It, Long Ago “Information about the transactions of money has become almost as important as the money itself.” - Walter Wriston, former chairman and CEO of Citicorp, awardee of Presidential Medal of Freedom, 1989 Could you cut and paste “health” for “money”? What if we gave healthcare away at a discount– or for free-- just so we could collect the data for its analytic value? What if Health Catalyst started a healthcare delivery system so we could collect and control the ecosystem for the downstream value of the data? 8

9. #HASummit14 The Basic Process of Predictive Analytics 9

10. #HASummit14 “Beyond math, there are no facts; only interpretations.” - Friedrich Nietzsche 10

11. Challenge of Predicting Anything Human 11

12. #HASummit14 Sampling Rate vs. Predictability The sampling rate and volume of data in an experiment is directly proportional to the predictability of the next experiment 12

13. #HASummit14 Thank you for the graphs, PreSonus Healthcare and patients are continuous flow, analog process and beings But, if we sample that analog process enough, we can approximately recreate it with digital data 13

14. #HASummit14 We are asking physicians and nurses to act as our “digital samplers”… and that’s not going to work 14

15. #HASummit14 The Human Data Ecosystem 15

16. #HASummit14 We Are Not “Big Data” in Healthcare, Yet 16

17. #HASummit14 Predictive Precision vs. Data Content 17

18. #HASummit14 The Wisdom of Crowds & Suggestive Analytics 18

19. #HASummit14 The Wisdom of Crowds* CriteriaDescription Diversity of opinion Individual members of the group possess personal insights or facts on a topic, even if it’s simply an unusual interpretation of data and facts on that topic IndependenceIndividual members of the group form their own opinions and are not prone to the overt and predictable influence from other members of the group DecentralizationKnowledge on a given topic does not reside in central decision making bodies, and important decisions can be made by members of a local, decentralized crowd who most readily feel the consequences of those decisions AggregationThere are methods and techniques for gathering and aggregating the collective intelligence of the crowd The Criteria For Designing A “Good” Crowd 19 *--James Surowieki

20. #HASummit14 c c Poll Question #2: Guess The Weight Of The Steer Levi Wallace, Guessor Dave Fenn, Owner Charlie Brown, 8-yr old Swiss steer; the Guessee 2014 Southwest Washington State Fair 20

21. #HASummit14 2,767 pounds…! 21

22. #HASummit14 Amazon: Predictive or Suggestive? 22

23. #HASummit14 Poll Question #3 How many physicians were working in Utah in 2010? 2012 Physician Workforce Report from the Utah Medical Education Council 23

24. #HASummit14 5,596 24

25. #HASummit14 Predictive Analytics Outside Healthcare 25

26. #HASummit14 US Strategic Command, underground command center… prior to 9/11 26

27. #HASummit14

28. Reduce variability in decision making & improve outcomes Launch prematurely? Launch too late? Nuclear Operations 28 How And Where Can A Computer Help?

29. #HASummit14 Desired Political-Military Outcomes 1.Retain U.S. society as described in the Constitution 2.Retain the ability to govern & command U.S. forces 3.Minimize loss of U.S. lives 4.Minimize destruction of U.S. infrastructure 5.Achieve all of this as quickly as possible with minimal expenditure of U.S. military resources 29

30. #HASummit14 Odd Parallels “Clinical” observations Satellites and radar indicate an enemy launch Predictive “diagnosis” Are we under attack or not? Decision making timeframe < 4 minutes to first impact when enemy subs launch from the east coast of the US “Treatment” & intervention Launch on warning or not? 30 Healthcare Delivery and Nuclear Delivery

31. 31 Subjective Objective Assessment Plan

32. #HASummit14 NSA, Terrorists and Patients The Odd Parallels of Terrorist Registries and Patient Registries 32

33. #HASummit14 Predicting Terrorist Risk Risk = P(A) × P(S|A) × C Probability of Attack Probability of Success if Attack occurs Consequences of Attack (dollars, lives, national psyche, etc.) What are the costs of intervention and mitigation? Do they significantly outweigh the risk? 33

34. #HASummit14 Nuclear Weapons Risk Scenarios NUCFLASH  Accidental or unauthorized launch that could lead to the outbreak of war Broken Arrow  Accidental or unexpected event, e.g., nuclear detonation or non-nuclear detonation or burning Empty Quiver  Loss, theft, seizure, destruction of nuclear weapon Bent Spear  Damage to a weapon that requires major repair, and has the potential to attract public attention Dull Sword  A nuclear safety deficiency that cannot be resolved by the local unit What are the “adverse events” we were trying to predict and avoid? 34

35. #HASummit14 “Mr. Sanders, while your 9-year tenure as an inmate has been stellar, our analytics models predict that you are 87% likely to become a repeat offender if you are granted parole. Therefore, your parole is denied.” - 2014, 80% of parole boards now use predictive analytics for case management* *--The Economist, “Big data can help states decide whom to release from prison”, Apr 19th 2014 35

36. #HASummit14 Thank you Sonja Star, New York Times “Evidence Based” Sentencing 20 States use predictive analytics risk assessments to inform criminal sentencing 36

37. Recidivism Risk Assessment: Level of Service/Case Management Inventory (LS/CMI)* 37 15 different scales feed the PA algorithm Criminal History Education/Employment Family/Marital Leisure/Recreation Companions Alcohol/Drug Problems Antisocial Patterns Pro-criminal Attitude Orientation Barriers to Release Case Management Plan Progress Record Discharge Summary Specific Risk/Needs Factors Prison Experience - Institutional Factors Special Responsivity Consideration 42.2% of high risk offenders recidivate within 3 years. *--Nov 2012, Hennepin County, MN, Department of Community Corrections and Rehabilitation

38. #HASummit14 “Since the publishing of Lewis' book, there has been an explosion in the use of data analytics to identify patterns of human behavior and experience and bring new insights to fields of nearly every kind.” 38

39. #HASummit14 eHarmony Predictions “Heart” of the system: Compatibility Match Processor (CMP) 320 profiling questions/attributes per user 29 dimensions of compatibility ~75TB 20M users 3B potential matches daily 60M+ queries per day, 250 attributes Thank you, Thod Nugyen, eHarmony CTO 39

40. #HASummit14 Twenty-Nine Dimensions of Compatibility Thank you, Ryan Barker, Principal Software Engineering – Matching, eHarmony 40

41. #HASummit14 41

42. #HASummit14 The Good Judgment Project Funded by Director of National Intelligence, brainchild of Philip Tetlock Can groups of non-experts with access only to open source information, predict world events more effectively than intelligence analysts with access to classified information? What about “internationally recognized” experts? Since 2011: 5,000 forecasters, 1M forecasts, 250 topics  “…from Eurozone exits to Syrian civil war” Non-expert forecasters are 65% better than the experts, 30-60% better than predictive algorithms 42

43. #HASummit14 Predictive Analytics Inside Healthcare 43

44. #HASummit14 True Population Predictive Risk Management Thank you, for the diagram, Robert Wood Johnson Foundation, 2014 Very Little ACO Influence >/=30% Waste* 100% ACO Influence >/=30% Waste* 100% ACO Influence *Congressional Budget Office, IOM, “Best Care at Lower Cost”, 2013 True Population Health Management 44

45. #HASummit14 Socioeconomic Data Matters Not all patients can functionally participate in a protocol At Northwestern (2007-2009), we found that 30% of patients fell into one or more of these categories: Cognitive inability Economic inability Physical inability Geographic inability Religious beliefs Contraindications to the protocol Voluntarily non-compliant 45

46. #HASummit14 The key to predictive analytics in the future of healthcare will be the ability to answer this two part question: What’s the probability of influencing this patient’s behavior towards our desired outcome and how much effort (cost) will be required for that influence? 46

47. #HASummit14 Example Variables: Readmission Drivers Newborn delivery Multiple prior admissions High creatinine High ammonia High HBA1C Low Oxygen Sats Age Admitting physician is pulmonologist or infectious diseases Prior admission for CHF traumatic stupor & coma Prior nutritional disorders Diabetic drugs Thank you, Swati Abbott Which evidence- based Intervention? How much will it cost? How much will it reduce risk? Weighted Predictive Model Risk of Readmission 47

48. #HASummit14 Most Common Causes for Readmission Robert Wood Johnson Foundation, Feb 2013 1. Patients have no family or other caregiver at home 2. Patients did not receive accurate discharge instructions, including medications 3. Patients did not understand discharge instructions 4. Patients discharged too soon 5. Patients referred to outpatient physicians and clinics not affiliated with the hospital 48

49. #HASummit14 What Else Are We Trying to Predict? Common applications being marketed today Identifying preventable re-admissions: COPD, MI/CHF, Pneumonia, et al Sepsis Risk management of decubitus ulcers LOS predictions in hospital and ICU Cost-per-patient per inpatient stay Likelihood of inpatient mortality Likelihood of ICU admission Appropriateness of C-section Emerging: Genomic phenotyping 49

50. #HASummit14 Closing Thoughts & Questions 50 1.Action Matters: What is the return in investment for intervention? Are we prepared to invest more... or say “no”… to patients who score low on predicted engagement? 2.Human Unpredictability: The mathematical models of human behavior are relatively immature. 3.Socio-Economics: Can today’s healthcare ecosystem expand to make a difference? 4.Missing Data: Without patient outcomes, the PA models are open loop. 5.Wisdom of Crowds: Suggestive analytics from “wise crowds” might be easier and more reliable than predictive analytics, until our data content improves 6.Social Controversy: How much do we want to know about the future of our health, especially when the predictive models are uncertain?

51. #HASummit14 Financial Industry Got It, Long Ago “Falling sick is not just an individual’s problem. Nations crumble when their people are not strong. History is full of events riddled with diseases that brought societies to their knees.” - Kofi Annan, former Secretary-General of the United Nations 51

52. #HASummit14 Sometimes, the predictions are wrong Arthur Henning, the Nate Silver of the 1930s-1950s, missed this one… 52

53. #HASummit14 Analytic Insights A Questions & Answers

54. #HASummit14 Session Feedback Survey 54 1.On a scale of 1-5, how satisfied were you overall with this session? 1)Not at all satisfied 2)Somewhat satisfied 3)Moderately satisfied 4)Very satisfied 5)Extremely satisfied 3.On a scale of 1-5, what level of interest would you have for additional, continued learning on this topic (articles, webinars, collaboration, training)? 1)No interest 2)Some interest 3)Moderate interest 4)Very interested 5)Extremely interested 2.What feedback or suggestions do you have?