1. Dr. Karen Linkins, IBHP, Tides Center Dr. Benjamin Miller, University of Colorado Dr. Lynda Frost, Hogg Foundation for Mental Health Dr. Becky Hayes Boober, Maine Health Access Foundation

2.  1) Become familiar with strategies to evaluate a complex health initiative;  2) Explore strategies for advocating with policy makers;  3) Understand how to use data related to quality health care interventions to create compelling messages;  4) Gain insights on policy development and leveraging; and  5) Share lessons learned and practical tools.

3. Performance Accountability Measures  How much did we do?  How well did we do it?  Is anyone better off?  Friedman, M. (2005). Trying hard is not good enough: How to produce measurable improvements for customers and communities. FPSI Publishing.

4. How much did we do? # Customers served (by customer characteristic) How well did we do it? % Common Measures % Activity-specific Measures Is Anyone Better Off? # Skills/Knowledge #Attitude/Opinion # Behavior # Circumstance #Improved Health Outcomes Is Anyone Better Off? % Skills/Knowledge % Attitude/Opinion % Behavior % Circumstance % Improved Health Outcomes

5.  Who are our “customers”?  How can we measure if our “customers” are better off?  How can we measure if we are delivering services well?  How are we doing on the most important of these measures?  Who are the partners who have a role to play in doing better?  What works to do better, including no-cost and low-cost ideas?  What do we propose to do?

6.  Did we treat you well?  Did we help you with your problem?

7.  Performance Accountability Questions  Population Accountability Questions

8. What are the stories that can influence policy?

9. Benjamin F. Miller, PsyD Director of the Office of Integrated Healthcare Research and Policy Department of Family Medicine University of Colorado Denver School of Medicine

10. The problem(s)  Sometimes in the face of innovation we lose sight of our ultimate goal – to change healthcare.  We focus on the problems rather than recognize what is working.  We focus on meeting immediate needs (e.g. financial) rather than plan for long term success.  We slip into “protective mode” and forget why we started the innovation to begin with.  We stop seeing the other innovators around us and focus on ourselves rather than the larger community or larger field.

11. But first

12. Brilliance

13.  What we do (models)  What data we collect (clinical)  What we call ourselves (integrated)  What we need for sustainability (money)  Who we talk to (ourselves)  What we want (change)

14.  Measuring integrated mental health (what is that exactly?) ◦ There is no gold standard “tool” ◦ Consistency across sites (e.g., documenting mental health diagnosis)  The evidence is lacking and the field is in need of knowledge around the “elements”  HUGE scope  Financial sustainability (or the business case)

15. Mental and Physical Health Multimorbidity Coordination of mental and physical health treatment plans Severe Mental Illness and/or Substance Abuse Full coordination with specialty care Medical issues with psychosocial barriers to care Medical issues with psychosocial barriers to care Psychosocial Support Services Psychosocial Support Services Medical issues requiring behavioral or psychological intervention Behavior Change Education & Evidence-Based Treatments Behavior Change Education & Evidence-Based Treatments Mental Health and Substance Use Presentations Mental health treatment plan Example Targeted Service Response Mental Health Presentation

16.  Two pots of money  Workarounds are often viewed as the solution  We don’t know what we don’t know (but we think we know what we don’t know)  Turf wars and bad feelings

18. Clinical data Language Financial data What we measure How we track and measure what we do Better community connections and state to state connections (and collaborations) Shared and consistent evaluation plans for integration projects

19. Case study

20. Miller, B. F., B. Teevan, et al. (2011). "The importance of time in treating mental health in primary care." Families, systems & health : The journal of collaborative family healthcare 29(2): 144-145.

21.  Time spent with patient  Time spent with other providers  Assigning monetary amounts to time (and or patient volume)  Assessing changes in time and volume  Assessing value and outcomes  Learning about what patients use more time and benefit from integrated initiatives

22. Case study

23. Phillps, R. L., B. F. Miller, et al. (2011). "Better Integration of Mental Health Care Improves Depression Screening and Treatment in Primary Care." American Family Physician 84 (9): 980.

24.  Number of patients identified  Number of patients treated  Number of patients who improve from treatment  Comparing rates of identification to rates of diagnosis (accuracy)  Using screening tools repeatedly for treatment tracking

25. In summary

26.  Be heard  Know what policy solutions can help lead to sustainability (including financial)  Begin to collect some of the same data  Make sure our data are put into the medical record in such a way it can be extracted  Have an entity that can pull it all together  Be compelling, be accurate, be timely

27. Thank you

28. Karen W. Linkins, PhD Project Director Integrated Behavioral Health Project Tides Center

29. “Change is disturbing when it is done to us, but exhilarating when it is done by us” (Elizabeth Moss Kanter, Professor, Harvard Business School)  Many different definitions of systems change exist, but they share common elements: policies and practices, resources, relationships, power and decision-making, values, attitudes, skills, governance, and supportive policies and reforms.  Systems change is dynamic, developmental, non- linear, and complex.  The target of change is the system, not the individual.

30.  System change is defined as: changes in organizational culture, policies and procedures within and across organizations that enhance or streamline access, and reduce or eliminate barriers to needed services by target populations.

31.  Changes that endure beyond the funded project that lead to any or all of the following: ◦ Increased Access ◦ Improved Quality ◦ Enhanced Efficiency ◦ Increased Consumer Empowerment

32. Systems ChangeExample Increased AccessChanges in clinic operational policies (e.g., electronic open scheduling and wait time monitoring, expanding specialty staffing (telepsychiatry)) Improved QualityImprove provider capacity to meet patient needs by learning new skills and knowledge through distance learning Enhanced EfficiencyData sharing across PC and BH providers to increase identification and care coordination Increased Consumer Empowerment Access to personal health record; use of technology to facilitate client support groups

33.  Stakeholder interests  Initiative goals, including desired outcomes and impacts  How findings will be used, e.g.: ◦ Educate policy makers ◦ Disseminate best practices ◦ Change local systems and policies ◦ Support sustainability plans and garner new funding sources  Available resources for the evaluation

34.  Providers: Individual patient outcomes, panel management  Clinics/Clinic Systems: Population health management, administrative metrics (e.g., cycle times, provider productivity, patient and provider satisfaction), billing, culture change  Policy Makers: Cost and other administrative metrics  Community: Prevention, community health and wellness, healthy behaviors, consumer engagement  Foundations: Alignment with strategic priorities, return on investment, grantee accountability

35.  Step 1: Engage stakeholders  Step 2: Describe the program  Step 3: Focus the evaluation design  Step 4: Gather credible evidence  Step 5: Justify conclusions  Step 6: Ensure use and share lessons learned

36.  What will be evaluated? (program, context)  What aspects of the program will be considered in assessing program performance?  What standards (i.e., type or level of performance) must be reached for the program to be considered successful?  What evidence will be used to indicate how the program has performed?  How will the lessons learned be used to improve public health effectiveness?

37.  Design types: experimental, quasi- experimental, and observational designs.  No design is better or best in all circumstances.  Design and methods should be matched to the interests of targeted stakeholders (e.g., foundation, grantees, policymakers).

38.  Design drives what counts as evidence, how data are gathered, what claims can be made, who needs to be involved, and what data management systems are needed.  Mixed method designs are most effective because each method has biases and limitations.  During the course of an evaluation, methods might need to be revised or modified.

39.  Complex initiatives require significant investments of time, resources and energy to create common ground for change.  Programs often become so focused on immediate implementation issues (client “fixes”), the long-term vision for systems change becomes lost or deferred.  Balancing the funder’s need for accountability/rigor in reporting with developing and maintaining authentic relationships with grantees.

40.  Data collection must be relevant.  Data should not be collected unless they are shared and fed back to those responsible for collection.  Evaluation should be clearly connected to longer term outcomes. Failure to do so limits buy-in, understanding, and a greater sense of accountabilitytothe process.

41. Initiative Goals: Create a more responsive and integrated system of care to increase access and reduce costs for individuals with co-morbid conditions (MH & chronic conditions)  Patient focused ◦ Address patients’ needs, improve health outcomes ◦ Reduce reliance on ED resources for care that is more effectively provided in less costly, community-based settings  System Focused ◦ Reduce ED volume and diversion time, and avoidable inpatient use ◦ Encourage financing and policies that promote coordinated, cross system, multidisciplinary care and integration of services

42. Foundations Project Officers/Program Staff Policy Staff Evaluation Staff Program OfficeEvaluation Team Grantees & Collaboratives Community-Based Organizations Hospitals Public Health, Housing/Homeless Programs, Mental Health, Substance Abuse, MediCal, Criminal Justice Oversight Group

43.  Participatory approach  Three evaluation phases of the evaluation ◦ Planning ◦ Implementation Process ◦ Outcomes and Promising Practices (“What Works”)  Multi-level, pre-post design

44. Enrolled TP Clients Outcomes Service utilization Costs Organizations Policies and practices Data systems MOUs Changes in services County System Data systems Financing Collaborations New services Restructuring State Level Laws and regulations Budget and financing Implementation Grants (e.g., Intensive Case Management) Structure Intensity Other Activities Meetings/Convening s Other activities Broader FUI Initiative Policy papers Other activities Intermediate Outcomes/Changes Interventions Long Range Impacts Service Delivery Change Client-based: Compare enrolled clients & TP at beginning and end of grant period (utilization and cost) System-based: MIS analysis of changes in the patterns of service utilization and costs system wide Planning Grants Broad Systems Change County State Frequent Users Initiative TP = Target Population

45. Measures  Cost and utilization (ED, inpatient and other systems as available)  Clinical measures of health and functioning  Stability (e.g., income and insurance enrollment)  Service intensity (frequency and duration)  Strength of partnerships and collaborations  Policy and systems change (evidence of improved coordination, streamlined access, permanent policy changes to address/eliminate barriers)

46.  Participatory orientation ◦ Balancing research rigor with “what’s reasonable and feasible” – selecting outcome measures and data collection strategies that matched capacity and didn’t over burden staff ◦ Developing and maintaining meaningful stakeholder participation (on-going communication) ◦ Establishing and maintaining trust of programs to ensure buy-in and data integrity  Defining/operationalizing multi-level outcomes  Ensuring evaluation findings aligned with and relevant to information needs of various stakeholders – at the “right time”

47.  Client centered interventions: challenge of programs/ models balancing individual client “fixes” vs. permanent programmatic and systems change  Data accuracy and consistency  Data availability and linkage capability  Mis-match of Foundation and Grantee Goals -- Foundations wanted systems and policy change, but funded local interventions

48.  Findings were compelling and rigorous enough to use for policy development (Medicaid Waiver and other legislation).  The combination of quantitative and cost data, as well as qualitative process and outcome data created a strong and policy relevant story of sustainable systems change.

49. Lynda Frost Director of Planning and Programs Hogg Foundation for Mental Health

50.  20 years of research on collaborative care model framed grant program on integrated healthcare  Large conference highlighted research and grantees’ work  Grantees engaged in advocacy around reimbursement, other issues  Evaluation of grant program gathered state- specific outcome data and identified barriers to implementation

51.  Foundation convened key stakeholders to identify barriers to implementation  One stakeholder lobbied for creation of “Integration of Health and Behavioral Health Workgroup”  Legislation mandated broad group of appointed workgroup members  Resulting report described “best practices” and recommended next steps

52.  Foundation signed agreement with DHHS Office of Minority Health to examine integrated healthcare as a means of eliminating health disparities in racial and ethnic minority populations and persons with limited English proficiency  Developed consensus report drawing on practice-based evidence  Held large conference to share results; OMH will release report with other national reports

53. Becky Hayes Boober Program Officer Maine Health Access Foundation

54.  Maine Health Access Foundation—2 stories ◦ Middle of the night sentence embedded in budget shifting hospital-based outpatient BH care from Section 45 to Section 65.  “Medical Care - Payments to Providers 0147 Initiative: Reduces funding from reducing reimbursement for outpatient substance abuse and mental health services to MaineCare Section 65 rates effective July 1, 2012.”

55.  Know what is happening (rule making draft)  Take action (Work with DHHS to slow process)  Explore alternatives  Partner (Maine Hospital Association and legislators)  Monitor

56.  IFS Committee Invitation ◦ Budget shortages ◦ Messaging is important (Endowment is 1/10 of 1% of what is spent annually in Maine on health care costs)

57.  Build relationships, partnerships.  Be proactive.  Tell a compelling story. ◦ Human element (sans drama) ◦ Data ◦ Cost effectiveness ◦ Resulting outcomes  Embed into other key endeavors.  Identify key leverage points (employers)

58. Maine’s Medical Home Movement 540 Maine Primary Care Practices 26 Maine PCMH Pilot Practices 20 Pilot Phase 2 Practices 14 FQHCs CMS APC Demo 82 NCQA PCMH Recognized Practices 100 MaineCare Health Home Practices Payers: Medicare Medicaid Commercials (Anthem, Aetna, HPHC) Payer: Medicare Payer: Medicaid

59. BH HIT Support and grant ACA Embedding Integrated Care into Maine’s Transforming Health System Reforms 540 Maine Primary Care Practices; 53 Community Behavioral Health Agencies; 30 SA Agencies 26 Maine PCMH Pilot Practices + 20 new 14 FQHCs CMS APC Demo (Medicare) 82 NCQA PCMH Recognized Practices ~100 MaineCare HH Practices?? Beacon Payment reform grants; ACOs Community Care Teams DHHS Value- based contracting SAMHSA Health Home ACOs: Pioneer and Employer-Based Section 1703 FQHC expansion AHRQ Academy MeHAF IC grants, TA

60. The Kid’s good. The New Yorker. March 21, 2011

61. What are your experiences?

62.  In your small group, select a policy change you would like to see happen. Develop a messaging plan.

63.  What compelling human interest stories will build the case?  What data do you have that will help build a compelling story? What data do you still need? How will you get it? Present it?  How will you involve patients/families?  Who are potential partners (current and needed)?

64.  Share 1 key idea about messaging.  Share 1 key strategy for influencing policy, using data/stories.

65.  Benjamin.miller@ucdenver.edu @miller7 occupyhealthcare.net  Lynda.frost@austin.utexas.edu  karen@desertvistaconsulting.com  bhboober@mehaf.org