1. Medically Complex Patients (MCP) Webinar Theories of Change Onil Bhattacharyya, MD, PhD Frigon-Blau Chair in Family Medicine Research, Women’s College Hospital University of Toronto Gary Naglie, MD Hunt Family Chair in Geriatric Medicine, Baycrest Health Sciences University of Toronto October 27, 2014

2. 1. Recognize the maturity and complexity of your models and how this impacts the key goals for your project 2. Introduce the concept of theory of impact as a starting point for thinking about your models 3. Provide a framework to test key aspects of your model and identify opportunities for improvement 2 OBJECTIVES

3. PROJECT MATURITY – SETTING THE RIGHT GOALS Developing new program Major modification to existing project Minor modification to existing project Scale up/replication 3

4. Kaiser Permanente Innovation Consultancy 4 “If things are fundamentally working in your system, then you are operating in the world of improvement. ” “If you fundamentally believe that what you have is broken, then you are operating in the world of innovation.”

5. FeaturesImprovement Typical problem Improve uptake of hand washing GoalHow to deliver service better ParametersClear user and outcome StrategyIterative, tests of smaller breadth Innovation Improve access for frail elderly Develop the appropriate service Tentative solution, user and outcome Iterative, tests of larger breadth IMPROVEMENT VS. INNOVATION 5

6. Developing a Theory of Impact Why might your intervention work?

7. THEORY OF IMPACT 1. Problem – Something important but also manageable 2. Solution (intervention/mechanism) – A target group, a provider and a mechanism to address problem 3. Outcomes – Key outcomes that can be attributed to solution (shorter- and longer term) 7

8. THEORY OF IMPACT - COPD 1. Problem – COPD is a major cause of readmissions and COPD action plans are rarely used 2. Solution (intervention/mechanism) – Nurse case managers can increase self-management and use of action plans 3. Outcomes - Symptom scores, ED visits and hospitalizations, 8

9. THEORY OF IMPACT - ICCT 1. Problem – Growing frail senior population receive fragmented care that is expensive and does not meet their needs 2. Solution (intervention/mechanism) – Integrated primary, community and specialty care outreach service providing intensive case management and coordinated care across transitions 3. Outcomes – Better care experience, smoother transitions, reduced health care utilization 9

10. Original Model: Baycrest Community Outreach Team (COT) 10 Frail Older Adult/Caregiver Referring Community Primary Care Physicians Referring Community Primary Care Physicians Consultation Community Outreach Team

11. New Model: The Integrated Community Care Team (ICCT) Integration at the Point of Care Frail Older Adult/Caregiver Community “Solo” Primary Care Physicians COT + COE + NP Acute Hospital Acute Hospital CCAC Intensive Case Managers + Pharmacist CCAC Intensive Case Managers + Pharmacist Consultation Shared Care Assumed Primary Care

12. Testing Key Hypotheses About Your Program

13. Most programs have many untested elements related May relate to target group, intervention or context Articulating assumptions and testing them will help you find and fix problems early on 13 WHY IS THIS IMPORTANT?

14. KEY CHALLENGES FOR CARE INTEGRATION PROJECTS Difficulty implementing Slow recruitment Poor fit between between intervention and target population not identified or modified Success rate <40% in well- conducted studies Variable results when replicated 14

15. INTEGRATION EXISTS ALONG A CONTINUUM 15 2-Way Communication Joint Care Delivery Joint Management Across Institutions Engagement Communication & Behaviour Culture & Policy Low Medium High Common Strategies Common Challenges

16. Adapted From: Blank, S. Harvard Business Review, May 2013. User Discovery User Validation Recruitment Program Building SearchExecution Change user or intervention 16 FINDING THE RIGHT PATIENTS, BUILDING THE RIGHT PROGRAM

17. TESTING HYPOTHESES AT EACH STEP Feasibility of Implementation Recruitment of patients Recruitment of providers Intervention Outcomes 17

18. FEASIBILITY OF IMPLEMENTATION Hiring staff – clerical, providers, managers Organizational support Data sharing Divergent institutional processes Governance 18

19. ICCT EXAMPLE: IMPLEMENTATION CHALLENGES Hiring staff Data sharing Divergent institutional processes Governance 19

20. PATIENT RECRUITMENT Why would patients join? Which patients will benefit? Are there enough patients? Are referral mechanisms effective? Will patients show up? Are the people who are being referred different than expected? 20

21. PROVIDER RECRUITMENT What is the benefit for providers? Is it easy to join or refer patients? Consider steps in adoption  Enrollment  First use  Regular use 21

22. ICCT EXAMPLE: PATIENTS AND PROVIDERS Are people referred different than those being targeted?  Yes – not all require in-home assessment Is it easy to refer patients?  Not from community docs perspective 22

23. INTERVENTION Is the intervention doable? Are the elements of the intervention working? If reaching the target group, does the intervention meet their needs? Are there any missing components? 23

24. ICCT EXAMPLE: FEATURES OF INTERVENTION Are elements of the intervention working?  Consultation stream and shared care require adjustments in hand-off and communication Are there any missing components?  Over 50% of referrals from Central LHIN 24

25. Do the outcomes match the maturity of the model? Are the outcomes appropriate given the mechanism of the intervention? Are the outcomes appropriate given the target patients? 25 ICCT EXAMPLE: OUTCOMES

26. Do the outcomes match the maturity of the model?  Premature to focus on health care utilization  Focus on patient and care provider experience Are the outcomes appropriate given the target patients?  Add quality end-of-life care 26

27. Use questions to inform yourself about how various parts of your project are working Systematic review of quality improvement studies found that less than 20% documented changes through iteration or used monthly cycles 27 ITERATION IS HARD WORK If you’re worried about your model, you will test it! BMJ Quality and Safety, Taylor MJ et al 2013

28. “The real measure of success is the number of experiments that can be crowded into twenty-four hours” Thomas Edison 28

29. CONCLUSIONS 1. Models are at varying levels of maturity and levels of complexity 2. Start from a high-level theory of impact 3. Drill down and test all of the aspects of your model to identify opportunities for improvement 4. This will help you find the right patients and build the right program 29

30. Questions