1. Building Professional Networks to Support Implementation of Evidence-Based Mental Health Services Funding: NIMH (R25 MH080916-01A2, T32 MH019117; F31 MH098478), VA (QUERI) Alicia Bunger Ohio State University Byron Powell Washington University in St. Louis Rochelle Hanson Medical University of South Carolina Nathan Doogan Ohio State University Yiwen Cao Ohio State University Jerry Dunn University of Missouri-St. Louis

2. Purpose Examine change in professional advice- seeking patterns among mental health clinicians participating in a learning collaborative for implementation.

3. Learning Collaborative Models IHI’s Breakthrough Series Learning Collaborative Quality Improvement Teams from multiple agencies Emphasizes shared learning Stimulating interactions Within & Across organizational teams Are they effective? How? Mixed evidence (Schouten et al, 2008) “Black Box” (Mittman, 2004) Expert Panel Commitment Preparatory Work In-Person Learning Sessions (3) Plan-Do-Study-Act (PDSA) cycles Active Learning Team Calls Web Support Quality Improvement Techniques Supports (IHI, 2003; Nadeem, et al, 2013)

4. Social Networks and Implementation LCs may support implementation by building social networks within and across participating agency teams. Networks are conduits for technical information and social support

5. 3 Ways LCs May Build Networks: Content Experts Peers at Home Agency LC Peers at other agencies Clinician Intra- Organizational Support Technical info – knowledge/skill Inter- organizational support, New ideas, Referrals Opportunities for Interaction Learning Sessions Consultation Calls Learning Sessions PDSAs Learning Sessions Group Calls

6. Do learning collaboratives “rewire” social networks in a way that supports implementation? AIMS: 1. Assess change in the composition of clinicians’ professional advice networks over the duration of a learning collaborative. 2. Examine how changes in clinician advice seeking patterns alter the structure of the regional network.

7. Study Setting $2 million regional initiative to implement TF-CBT funded through a county-based tax levy 32 Children’s behavioral health agencies Community-based trainers, certified by NCTSN as TF-CBT therapists Expert Panel Commitment Preparatory Work In-Person Learning Sessions (3) Plan-Do-Study-Act (PDSA) cycles Active Learning Team Calls Web Support Quality Improvement Techniques Supports

8. Study Setting $2 million regional initiative to implement TF-CBT funded through a county-based tax levy 32 Children’s behavioral health agencies Community-based trainers, certified by NCTSN as TF-CBT therapists Enhanced Learning Collaborative Model Expert Panel Commitment Preparatory Work In-Person Learning Sessions (3) Plan-Do-Study-Act (PDSA) cycles Active Learning Team Calls Web Support Quality Improvement Techniques Supports Coaching Calls On-Site Visits Local Trainers Rostering Enhanced Features

9. Method Sample 132 participants from 32 agencies (with pre & post data) 90% of Learning Collaborative completers Data Collection Surveys administered in-person during 1 st & 3 rd learning sessions (est. 10 months apart) DisciplineRoleExperience Social Work53%Sr. Leader10%GT 5 yrs in field65% Counseling28%Supervisor22%LT 1 yr in job43% Psychology12%Clinician68%

10. Measures Rank Column A – Names Column B - Communication Who do you turn to for professional advice about youth with trauma histories? Please list their name and organization in order you would contact them. In the past 6 months, how frequently have you communicated or been in contact with this person via in-person contact, telephone, or email? (Circle the most accurate number from the answer scale below for each person.) Not Once 1-2 times About once/ month About every 2 weeks About once/ week About daily Many times daily 1. Name: Organization: 1234567 …… Nominate up to 5 sources of professional advice 422 Unique individuals nominated across both waves of data collection

11. Analysis 1. Compare Composition of Professional Advice Networks Clinician Ego-Network at LS1 and LS3 Calculate and compare Exposure (% of Ego-net) using paired samples t-test in Stata 13 (Valente, 2010) 2. Compare Network Structure Visualize Network Descriptives (R - sna, igraph) Content Experts LC Peers at other agencies Private Practice Other Peers at Home Agency

12. Ego-Net: Size of Professional Advice Networks *t(131)=2.06, p<.05

13. Ego-Net: Composition of Professional Advice Networks

14. **Experts: T(131)=6.60, p<.001 **Private Practice: T(131)=-3.24, p<.001 **Other: T(131)=-3.41, p<.001

15. Whole Network Structure – LS 1 Node = Person N=422 Diamond = Faculty Expert N=5 Line = Nomination/Tie N=2487 Isolate = Person w/no ties N=74 Components

16. Compare Network Structure Learning Session 1 Learning Session 3

17. Compare Network Structure Learning Session 1Learning Session 3 N 422 Isolates 74177 Density 0.0140.013 Centralization (in-degree) 0.097.182 Clustering (weighted) 0.293.356

18. Compare Network Structure Learning Session 1Learning Session 3 N 422 Isolates 74177 Density 0.0140.013 Centralization (in-degree) 0.097.182 Clustering (weighted) 0.293.356 Reciprocity 0.1640.227 Weighted Reciprocity 0.1880.246

19. Compare Network Structure Learning Session 1Learning Session 3 N 422 Isolates 74177 Density 0.0140.013 Centralization (in-degree) 0.097.182 Clustering (weighted) 0.293.356 Reciprocity 0.1640.227 Weighted Reciprocity 0.1880.246 Agency Homophily 89.0877.72

20. Limitations Generalizeability 1 region No comparison/control Was the LC responsible for making net change? What elements of the LC?? Measurement validity Self-report measures Drop-Out/Missing data Some participated in only one wave of data collection Drop-Out Opt-Out Snow-Out (winter weather during one LS)

21. Summary of Findings Clinician-Level Clinicians rely on colleagues at their home agency Exposure to faculty experts increased Slight reduction in exposure to external sources of advice (perhaps because of coaching+consultation) Whole Network Centralization around Faculty Experts Reciprocity

22. Implications For Learning Collaborative Organizers Provide additional opportunities for participants to network across organizational boundaries. For Policy Makers and Administrators Benefits of local experts/knowledge leaders for scale-up initiatives. Potential for sustainment? Integration of local service delivery system (in terms of advice sharing) Small changes at the individual clinician-level can translate to big changes at the systems-level.

23. Future Research Questions: Why do professional advice ties change? LC Components: LS? Coaching? LS + Coaching? Network dynamics? Readiness for implementation? Supportive climate? Do professional advice networks have a role in implementation success? What is the relationship between ego-net composition, position in the network, etc. with implementation fidelity? Treatment outcomes? Why do some clinicians/organizations remain disconnected? Initial Readiness? Innovation-values fit?

24. Contact information Alicia Bunger Bunger.5@osu.edu

25. References Aarons, GA, Hurlburt, M, & Horwitz, SM. (2011). Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Administration and policy in mental health, 38(1), 4–23. Damschroder, LJ, Aron, DC, Keith, RE., …(2009). Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation science, 4, 50. IHI (2003). The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. Cambridge, MA. Retrieved from http://www.ihi.org/IHI/Results/WhitePapers/ Mittman, BS. (2004). Creating the Evidence Base for Quality Improvement Collaboratives. Ann Intern Med, 140(11), 897–901. Nadeem, E, Olin, SS, Hill, LC, Hoagwood, KE, & Horwitz, SM. (2013). Understanding the components of quality improvement collaboratives: a systematic literature review. The Milbank quarterly, 91(2), 354–94. Powell, BJ, McMillen, JC, Proctor, EK … (2011). A Compilation of Strategies for Implementing Clinical Innovations in Health and Mental Health. Medical care research and review, 69(2), 123–157. Schouten, LMT, Hulscher, MEJ, van Everdingen, JJE, …(2008). Evidence for the impact of quality improvement collaboratives: systematic review. BMJ (Clinical research ed.), 336(7659), 1491–4. Valente, TW (2010). Social networks and health: models, methods, and applications. Oxford University Press.

26. Acknowledgements Missouri Academy of Child Trauma Studies (MoACTS) at the Child Advocacy Center of Greater St. Louis (UMSL). NIMH Postdoctoral Traineeship (T32 MH019117) sponsored by UNC-CH & Duke (Bunger) Predoctoral traineeship (F31 MH098478) (Powell) NIMH/VA Implementation Research Institute (R25 MH080916-01A2) (WUSTL) (Bunger & Hanson) Doris Duke Charitable Foundation Fellowship for the Promotion of Child Well-Being (Powell)