1. The Implementation Process: Perspectives from Frontline Providers and Managers JoAnn E. Kirchner, MD Louise E. Parker, PhD Laura Bonner, PhD Elizabeth M. Yano, PhD, MSPH Mona Ritchie, MSW

2. Agenda  Formative Evaluation –TIDES  Methods  Findings

3. Formative Evaluation  Types of formative evaluation –Developmental evaluation –Implementation-focused evaluation –Progress-focused evaluation –Interpretive evaluation  The “process” as a product

4. Formative Evaluation: TIDES  Developmental evaluation –Pre-implementation stakeholder interviews with MH and PC leadership  Implementation-focused evaluation – Mid-implementation qualitative interviews – Early feedback of initial findings to implementation team  Progress-focused evaluation –Program penetration and utilization  Interpretive evaluation –Site case studies –Cost analysis –Qualitative interviews

5. Methods  Conducted 106 qualitative semi-structured interviews during field visits with – Managers across four participating medical centers and three participating regional networks – Providers, administrators, and consumers across five participating facilities  Analysis – Data management package – Coded by two-investigator teams – Independent coding with resolution of conflicts

6. Methods  Here we report findings from – 19 medical center and regional network managers – 49 Frontline providers and administrators  Concerning their perspectives on the QI program implementation process

7. Results

8. Local Site Resources Implementation process Local Frontline Staff: Attitudes Behaviors Penetration and Sustainability Local Outcomes Spread

9. Importance of site characteristics  Attitudes/beliefs/experiences – Perceived need for the intervention – Competing needs – Staff open to innovation – MH/PC relationship Expected would affect implementation success but did not

10. Importance of site characteristics  Resources – Perceived time to Utilize the program  Marketing opportunity Participate in implementation activities – Organizational structure – Staffing – Prior QI experience – Tools (e.g., access to informatics support)

11. Who to Involve  All Informants –Medical Center Managers –Frontline Clinical Managers –Frontline Primary Care Providers (MD, PA, APN)  Frontline Providers –RNs, LPNs –Mental Health Providers

12. Key Roles in Implementation  Leadership leadership support is obviously also critical. Unless the leaders are able to translate the importance of the initiative, it's probably not going to take place for a variety of reasons. One of which is that there are a thousand things that need to be done, priorities need to be set and it’s leaderships responsibility to make sure that those priorities are delineated and made clear to everybody, what is it that we’re going to do and what is it we're not going to do - Chief of Staff –Importance of leaders valuing the program I think it’s [leadership support] important because if the leadership shows that it’s important enough for them to spend their time, then it implies that it’s important enough for us to spend our time - MH Provider, Physician

13. Key Roles in Implementation initiating the program “cheerleading” Managers just have to go back out and continue doing cheerleading sessions. A little more cheerleading. You go back, and you... when you see that that’s where you are, you have to just go back and reemphasize. Okay. We’ve gotten to here, we’ve got to be over here or...you guys are doing a great job here, but we could do a little more here, and it’s that cheerleading. You know, 90 percent I think of what managers do are cheerleading people on - Network Manager resources I feel that anything that you have in place, if the head of anything...is not going to support you, give you what you need, be there for you in every sense of the word, it’s not worth having. I think it is doomed to fail if you don’t have that in place - Network Manager importance of management at multiple levels

14. Key Roles in Implementation  But in absence of clinic perceived need for the program, leadership will have an uphill battle

15. Key Roles in Implementation  Champion – Who should be a champion? Does that person have credibility with the other staff, you know, and that’s not measured in what letters you have behind your name. That’s do people listen to you when you speak. I don’t know how you put a degree on that, but that’s what you need - Primary Care, RN  Colleague who has experienced the program  High status peers  Someone interested in quality improvement and in improving mental health care in primary care setting  Credible  Able to communicate well to groups  Self-driven  Good rapport

16. Key Roles in Implementation  Role/activities of the champion He’s [the local champion] been the person who’s at our meetings reminding us about the program…And I think the more you do that, the more you get people using it - Primary Care, RN –Serve as the local expert and resource person –Encourage people to attend related meetings –Remind people about the program –Provide ongoing marketing  Needs of the champion –Time

17. Key Roles in Implementation  Leadership as a champion –Roles of champion, leadership, and opinion leaders are not distinct in the views of clinical staff and managers –Thus, implementation researchers may be creating artificial distinctions

18. Site Readiness Low hanging fruit What about the fruit higher in the tree?

19. Informing Readiness  Message will depend on how ready sites are for a particular intervention (i.e., perceived need)  Regardless of readiness, different audiences (i.e., managers versus frontline) have different needs to promote readiness  Importance of ongoing marketing for all Sometimes in the VA, you’ve got to tell me three times to do it because if you tell me three times, it was really important. If you tell me once and I never hear from you again, then it probably wasn’t important and was a passing fad and I’ll kind of wait – Chief of Staff

21. PLAN DO STUDY ACT 1 st Generation Sites 2 nd Generation Sites 1 st Generation Sites sustainability penetration

22. Ongoing Formative Evaluation  Developmental evaluation –Site level needs assessment tool  Implementation-focused evaluation –Site level fidelity monitoring tool  Progress-focused evaluation –Program penetration and utilization through National Depression Monitor  Interpretive evaluation –Formative evaluation of regional dissemination of TIDES –Assessment of the sustainability effort