1. Applying Multiple Frameworks and Theories in Implementation Research (Part 2) Jeffrey Smith Implementation Research Coordinator Mental Health QUERI

2. Outline Review major points on multiple theory approach from April 14 cyberseminar Review major points on multiple theory approach from April 14 cyberseminar Provide example of application of multiple theory approach in MH QUERI project Provide example of application of multiple theory approach in MH QUERI project

3. Mental Health QUERI Approach to Implementation Design interventions based on theory, lit review and results from formative evaluation Design interventions based on theory, lit review and results from formative evaluation Conduct formative evaluation Conduct formative evaluation –engage with stakeholders –identify determinants of current practice –assess barriers and facilitators to implementation organizational-level organizational-level team / clinic-level team / clinic-level individual provider-level individual provider-level patient-level patient-level –tailor intervention design and implementation to local context

4. Mental Health QUERI Approach to Implementation (cont) Use external facilitation techniques (PARiHS Framework) Use external facilitation techniques (PARiHS Framework) –engage with stakeholders to problem-solve and identify new strategies or tools for overcoming barriers when initial success is sub-optimal Conduct summative (or impact) evaluation Conduct summative (or impact) evaluation –assess intervention effects on targeted clinical processes and/or patient outcomes

5. Multiple Theory Approach Application of multiple frameworks/theories in guiding intervention design and implementation can be successful in implementing EBPs Application of multiple frameworks/theories in guiding intervention design and implementation can be successful in implementing EBPs – accommodates tailoring to setting when combined with formative evaluation acknowledges there are generally multi-level determinants to complex, clinical QI issues acknowledges there are generally multi-level determinants to complex, clinical QI issues – organizational-level – team-level – interpersonal-level – individual-level

6. Antipsychotic Treatment Improvement Program (ATIP) Goals Goals –Improve adherence to CPGs for antipsychotic treatment of psychosis –Reduce use of very high doses of antipsychotic medications –Encourage use of newer “atypical” antipsychotic medications for non-responders to conventional antipsychotic medications

7. ATIP Application of Frameworks, Theory and Planning Models Diffusion of Innovation (Rogers), Social Cognitive Theory (Bandura), Social Influence Theory (Mittman) Diffusion of Innovation (Rogers), Social Cognitive Theory (Bandura), Social Influence Theory (Mittman) – –Utilize influential local clinician leaders (opinion leaders) to inform other clinical staff about evidence-based antipsychotic medication management, model-targeted prescribing behaviors, and motivate practice change Complexity Theory – –Initial conditions in HCOs are important in intervention planning, but HCOs are adaptive and change over time – –Need mechanism for assisting clinical partners in modifying or adapting initial intervention strategy based on changing circumstances, initial success

8. ATIP Application of Frameworks, Theory and Planning Models (cont) Promoting Action Research Implementation in Health Services (PARiHS) Promoting Action Research Implementation in Health Services (PARiHS) – –Use external facilitation techniques to enable and assist clinical opinion leader with problem-solving; address challenges to intervention implementation PRECEDE Planning Model (Predisposing, Reinforcing, and Enabling Constructs in Education Diagnosis and Evaluation) – –Address predisposing factors (eg, knowledge, attitudes) that influence EBP adoption – –Enable providers to follow guideline recommendations at the point of care – –Apply social incentives through performance audit/feedback to reinforce providers’ implementation of EBPs

9. Facilitation is a process of interactive problem-solving and support to meet specific implementation goals, which occurs in the context of a recognized need for improvement and a supportive interpersonal relationship. “Refined” Description of Facilitation Based on VA QUERI Experience * * Stetler CB, et al. Implementation Science 2006; 1:23.

10. Operationalization of External Facilitation in MH QUERI (based on PARiHS) Facilitator Facilitator –Maintains regular contact w/ local QI leader / team Email communication Email communication Telephone Telephone Participate in QI Team meetings (as feasible) Participate in QI Team meetings (as feasible) –Assist in goal-setting; monitor implementation of project tools/strategies; performance on clinical QI goals –Identify and problem-solve barriers to implementation –Assist in adapting tools/strategies as needed or suggested by site partners to meet project goals

11. ATIP Intervention Components Intervention Component Supporting Theory or Planning Model Clinical Opinion Leader Diffusion of Innovation, Social Cognitive Theory, Social Influence Theory External Facilitation PARiHS, Complexity Theory Psychosis Guidelines Help File PRECEDE (predisposing) Pocket Card on Antipsychotic Treatment PRECEDE (enabling) Pharmacy Order-Entry Reminder on Antipsychotic Dose Recommendations PRECEDE (enabling) Clinical Reminder on Metabolic Side Effects Associated with Olanzapine PRECEDE (enabling) Audit/Feedback Performance Reports on Antipsychotic Dosing / Side Effect Monitoring PRECEDE (reinforcing)

12. Summative Evaluation ATIP intervention improved antipsychotic medication management in concordance with CPGs (results varied by site) Reduced pharmacy costs for antipsychotics (results varied by site) Participating clinicians reported positive experiences with ATIP educational materials and clinical support tools

13. An Approach to Using Theory for Implementation Planning Select theory of planned behavior change Assess fit with initial theory Identify potential strategies for achieving change Select interventions that fit with planned strategies (based on theory) Identify intervention tools that fit both strategy and theory Launch intervention using identified tools and strategies Evaluate effectiveness of intervention, strategies, tools From: Sales A, Smith JL, Curran G, Kochevar L. Models, strategies and tools: The role of theory in implementing evidence-based findings into health care practice. Journal of General Internal Medicine 2006; 21:S43-49.

14. Formative Evaluation: Barriers and Lessons Learned Physicians do not always agree on who is an opinion leader; some sites may have no opinion leader Physicians do not always agree on who is an opinion leader; some sites may have no opinion leader Some sites had poorly developed formal and informal social networks among physicians Some sites had poorly developed formal and informal social networks among physicians A focus on physicians only as agents of change A focus on physicians only as agents of change How much directive should be given to the opinion leaders on how to influence attitudes and behaviors? How much directive should be given to the opinion leaders on how to influence attitudes and behaviors? Audit/feedback reports need to be more actionable Audit/feedback reports need to be more actionable From: Curran GM, Thrush CR, Smith JL, et al. Jt Commission J Qual Safety 2005; 31(12):700-707.

15. Multiple Theory Approach Strengths – useful in designing multifaceted interventions to influence multi-level determinants of care (flexible) – allows integration of theory, knowledge, methods from multiple disciplines (multidisciplinary) Limitation – Can be unwieldy… need to provide rationale for applying multiple theory approach, and rationale for selecting the specific frameworks / theories applied Key Guidance on Evaluation – combine with rigorous formative evaluation – conduct summative (impact) evaluation to assess intervention effectiveness on key study outcomes – confirm, refute or propose refinements to selected theory (or applied strategy based on selected theory) based on study findings

16. QUESTIONS? Contact: Jeff Smith VA Mental Health QUERI E-mail: Jeffrey.Smith6@va.gov

18. Developmental Identify determinants of current practice Identify potential barriers / facilitators Assess feasibility of proposed intervention Integrate findings into intervention design and refinement prior to implementation Implementation-Focused Assess discrepancies between implementation plan and execution, exploring issues of fidelity, intensity, exposure Understand and document nature and implications of local adaptation Progress-Focused Monitor impacts and indicators of progress toward project goals Use data to inform need for modifying or refining original strategy Provide positive reinforcement to high performers; negative reinforcement to low performers Interpretive Assess intervention usefulness/value from stakeholder perspective Elicit stakeholder recommendations for further intervention refinements Assess satisfaction with intervention and implementation process Identify additional barriers / facilitators Stages of Formative Evaluation Post- Implementation Pre- Implementation Implementation