1. A Framework for Evaluating Coalitions Engaged in Collaboration ADRC National Meeting October 2, 2008 Glenn M. Landers

2. Georgia Health Policy Center Established in 1995 Andrew Young School of Policy Studies, Georgia State University Long relationship with Georgia Department of Human Resources

3. Background Began evaluation activities in March 2007 –Process evaluation of all sites Focus on Advisory Boards –Assessing potential for outcome evaluation Evaluation of data integrity –Logic modeling with expansion sites –Evaluation of ADRC training workshops

5. Evaluation Background Frances D. Butterfoss – Center for Pediatric Research Center for the Advancement of Collaborative Strategies in Health – partnership self-assessment tool Community Health System Development Technical Assistance Program

6. Why Evaluate Coalitions Engaged in Collaboration? Builds capacity within coalition & community Determines whether objectives are achieved Improves coalition interventions Provides accountability to community, funding agencies & stakeholders Increases community awareness & support *F.D. Butterfoss, 2004

7. Why Evaluate Coalitions Engaged in Collaboration? National expectations for ADRC –Seamless system for consumers –Integrated access – streamlined eligibility –Meaningful involvement of consumers and other stakeholders –Partnership between Aging Network and Medicaid –Sustainability

8. BenchmarksKey Elements of Success BeginningDevelopingHigh-Functioning Clear Vision and Intent Mission/vision statement is written and agreed upon by ADRC members No mission statement.Mission statement exists. Mission has been confirmed or revised within 3 years Mission is understood and clearly communicated by ADRC members Partners do not understand and communicate mission. Mission is understood and communicated by members Partners advocate for the ADRC mission. Memoranda of Understanding (or other formal agreements) exist between partner agencies ADRC has no mechanism for ensuring participation from members. ADRC is in process of developing MOUs. ADRC has signed MOUs with members. ADRC activities represent stated goals and missionActivities are outlined. Activities are aligned with the goals mission. The mission drives ADRC activities Culture of Caring ADRC activities address access to services, increase efficiency and improve care of vulnerable populations ADRC addresses health status or access or efficiency of care delivery to vulnerable populations. ADRC addresses more than one dimension. ADRC addresses all dimensions. ADRC delivers integrated services in collaboration with partnersIntegration is planned.Some services are integrated.All services are integrated. Communication and Campaigning Community awareness of existence of ADRC The community is not aware the ADRC exists. The community may know the ADRC exists but may not know what it does. The ADRC is widely known throughout the community. ADRC engages in patient/client outreach and education activities Education activities are planned. Some education activities have taken place. Education activities are extensive. ADRC has regular mechanism for seeking out and using community input The ADRC does not seek community input. The ADRC seeks community input. The ADRC seeks community input and has a mechanism to respond to it. The ADRC engages in advocacy efforts with an eye towards sustaining activities and engaging additional external partners The ADRC‘s only source of funding is from a grant. The ADRC has concrete plans to seek other funds. The ADRC engages its partners in advocacy efforts, enabling funding from diverse sources. Infrastructure to Support The ADRC has sufficiently trained staff to realize key program activities The ADRC has at least one sufficiently trained fulltime staff member. The ADRC has job descriptions for all staff, a staffing structure, and a training plan. ADRC has sufficient number of trained staff. Existence of an effective information management system to track eligibility, enrollment and referrals at a minimum Partners have proprietary systems. ADRC has plans to integrate MIS systems. MIS systems are integrated across partners. Sustainability Based on Demonstrated Value ADRC is developing a sustainability plan Grants are the soul source of revenue. ADRC is developing a sustainability plan. Coalition has a current sustainability plan ADRC is collecting data that are specific, measurable and demonstrate key intermediate and impact outcomes ADRC has identified at least process measures. ADRC is measuring process and is identifying outcomes measures. ADRC measures both process and outcomes.

9. Coalition Benchmarks Clear vision and intent Culture of caring Communication and campaigning Infrastructure to support mission Sustainability based on demonstrated value Technical assistance

10. Benchmark: Clear Vision and Intent Mission/vision statement is written and agreed upon by ADRC members Mission is understood and clearly communicated by ADRC members Memoranda of Understanding (or other formal agreements) exist between partner agencies ADRC activities represent stated goals and mission

11. Benchmark: Clear Vision and Intent Beginning –No mission statement –Partners do not understand and communicate mission –ADRC has no mechanism for ensuring participation from members –Activities are outlined

12. Benchmark: Clear Vision and Intent Developing –Mission statement exists –Mission is understood and communicated by members –ADRC is in process of developing MOUs –Activities are aligned with the goals mission

13. Benchmark: Clear Vision and Intent High-functioning –Mission has been confirmed or revised within 3 years –Partners advocate for the ADRC mission –ADRC has signed MOUs with members –The mission drives ADRC activities

14. Benchmark: Culture of Caring ADRC activities address access to services, increase efficiency and improve care of vulnerable populations ADRC delivers integrated services in collaboration with partners

15. Benchmark: Communication and Campaigning Community awareness of existence of ADRC ADRC engages in patient/client outreach and education activities ADRC has regular mechanism for seeking out and using community input The ADRC engages in advocacy efforts with an eye towards sustaining activities and engaging additional external partners

16. Benchmark: Infrastructure to Support Mission The ADRC has sufficiently trained staff to realize key program activities Existence of an effective information management system to track eligibility, enrollment and referrals at a minimum

17. Benchmark: Sustainability Based on Demonstrated Value ADRC is developing a sustainability plan ADRC is collecting data that are specific, measurable and demonstrate key intermediate and impact outcomes

18. Collaboration: Lessons from Georgia

19. Lessons from Georgia All sites were ranked “beginning” on most dimensions. Where sites were ranked “developing”: –ADRC activities represent stated mission and goals –ADRC activities address access to services, increase efficiency and improve care of vulnerable populations

20. Lessons from Georgia Where sites were ranked “developing”: –Community awareness of existence of ADRC –ADRC coalition is collecting data that are specific, measurable and demonstrate key intermediate and impact outcomes.

21. Lessons from Georgia Collaboration challenges: –Delays in hiring new staff at MHDDAD –Concern about an “aging slant” –Differences in organizational culture –Differences in vocabulary –IT challenges; lack of integration

22. More information www.georgiaADRC.com glanders@gsu.edu www.gsu.edu/ghpc