1. Implementation Science: Research Priorities and Needs for the Field Ian D Graham PhD Vice President Canadian Institutes of Health Research Chicago, July 28 th 2009 Cross QUERI Working Group on Implementation Science

2. Working in a living Lab You have a unique opportunity to not only improve health and quality of life of your clients but also to further implementation (KT) science This can be done through 3 general strategies: –Move forward through retrospection and reflection –Make your tacit knowledge explicit through rigorous and “infectious” reflection and documentation –Use conceptual frameworks

3. Why use a conceptual framework? “It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.” Albert Einstein

4. Why use a conceptual framework? as with other forms of science, the success of a field is dependent on its theoretical foundation getting research into practice requires a systematic effort as the process is complex and occurs in the face of competing organizational and practice priorities hence researchers need to analyze and synthesize both the empirical and the theoretical underpinnings to be able to look inside the implementation “black box”

5. Why use a conceptual framework? theory can be the driver of comparable and consistent measurement/evaluation of implementation objectives important for researchers to evaluate the utility of their chosen theory including its internal and external validity – which components are critical, how should they be weighted The existing body of evidence about the effectiveness of different interventions suggests their impact is variable and effect sizes are generally moderate – thus evidence alone cannot guide implementation efforts

6. We undertook a critical analysis of conceptual models of implementation as a means of understanding the theoretical underpinnings of implementation or what we at CIHR call knowledge translation. They have the basic purpose of focusing, ruling some things in as relevant and ruling others out due to their lesser importance. Change theories/models fall into two basic kinds: Classical - models that describe change, but were not specifically designed to cause change (e.g.PARIHS Promoting Action on Research Implementation in Health Services; Diffusion of Innovations ) Planned – models that are intended to be used to guide or cause change (QUERI framework is a quality improvement PAT) …about conceptual frameworks

7. about conceptual frameworks Many of the PATs propose similar steps – can see that the QUERI framework addresses the action categories – the implementation “steps” The PATs are varied in their consideration of implemenation “factors” (covered in so called “classical” theories (of which there are many)) Besides these macro and meso-level theories,there are also many theories dealing with behaviour at the individual as well as organizational level (see chapter by Wensing et al. in “From Evidence to Practice” in particular, and the entire book, in general)

8. Monitor Knowledge Use Sustain Knowledge Use Evaluate Outcomes Adapt Knowledge to Local Context to Local Context Assess Barriers/facilitators to to Knowledge Use Knowledge Use Select, Tailor, Select, Tailor, Implement Interventions Identify Problem Identify Problem Identify, Review, Identify, Review, Select Knowledge Select Knowledge Products/Tools Synthesis Knowledge Inquiry Tailoring Knowledge KNOWLEDGE CREATION from: Graham et al: Lost in Knowledge Translation: Time for a Map? http://www.jcehp.com/vol26/2601graham2006.pdf

9. Monitor Knowledge Use Sustain Knowledge Use Evaluate Outcomes Adapt Knowledge to Local Context Assess Barriers/facilitators to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Products/Tools Synthesis Knowledge Inquiry Tailoring Knowledge KNOWLEDGE CREATION http://www.jcehp.com/ vol26/2601graham2006.pdf from: Graham et al: Lost in Knowledge Translation: Time for a Map?

10. Knowledge Inquiry studies using all types of study designs primary research needs to be targeted to fill the known gaps in our knowledge base primary research needs to be solutions-based Implementation Science: –need research on effectiveness/impact of interventions and develop common methods of evaluation QUERI 4 Phase Pipeline: –Phase 1: single site pilot; –Phase 2: small scale/multi site implementation trial; –Phase 3: large scale/multi-region implementation trial; –Phase 4: system wide roll out Products/ Tools Synthesis Knowledge Inquiry Tailoring Knowledge

11. Synthesis need for synthesis to determine what we already know (or should know if we were to summarize the existing knowledge) can take the form of systematic reviews, meta analyses, realist reviews, etc. need to determine where there is a strong evidence base and move that evidence into action Products/ Tools Synthesis Knowledge Inquiry Tailoring Knowledge

12. Knowledge Products/Tools Algorithms Decision rules Decision aids Guidelines Measurement tools Products/ Tools Synthesis Knowledge Inquiry Tailoring Knowledge

13. Monitor Knowledge Use Sustain Knowledge Use Evaluate Outcomes Adapt Knowledge to Local Context to Local Context Assess Barriers/facilitators to to Knowledge Use Knowledge Use Select, Tailor, Select, Tailor, Implement Interventions Identify Problem Identify Problem Identify, Review, Identify, Review, Select Knowledge Select Knowledge Products/Tools Synthesis Knowledge Inquiry Tailoring Knowledge KNOWLEDGE CREATION Action Cycle (Application)

14. EITHER Start with problem/issue concern and look for research to solve the problem OR Become aware of the research/guideline and assess whether current practice is in keeping with it Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION Monitor Knowledge Use Sustain Knowledge Use Evaluate Outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge

15. To apply knowledge, one needs to contextualize or adapt it for local use –e.g. ADAPTE Collaboration (for guideline adaptation) How might the implementation plan need to be adapted for local use? Adapt Knowledge to Local Context Sustain Knowledge Use Evaluate outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION Monitor Knowledge Use

16. Assess barriers and supports related to: –the knowledge (e.g. guideline) –the potential adopters –the practice setting –the implementation plan What might be the individual, team, and organizational barriers to uptake? Can the development process anticipate the barriers to use and address any of them? Assess Barriers/facilitators to Knowledge Use Monitor Knowledge Use Sustain Knowledge Use Evaluate outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION

17. Based on the barriers and supports identified, select or tailor implementation interventions e.g. Cochrane’s Effective Practice and Organization of Care review group source of synthesized info on effectiveness of KT interventions http://www.epoc.cochrane.org/en/index.html Select, Tailor, Implement Interventions Sustain Knowledge Use Evaluate outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION Monitor Knowledge Use

18. Assess the use or uptake of the knoweldge –Conceptual use (knowledge, attitudes, intentions) –Instrumental Use (behavioural), –symbolic knowledge use (to persuade) What “dose “ of the intervention is being delivered? Monitor Knowledge Use Monitor Knowledge Use Sustain Knowledge Use Evaluate outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION

19. Determine impact of the intervention –Client health and other outcomes (eg. symptoms, morbidity, HRQoL, mortality, satisfaction) –Provider outcomes (eg. satisfaction) –System/organization outcomes (eg. Wait times, expenditures, readmissions) –Unintended impacts Evaluate Outcomes Monitor Knowledge Use Sustain Knowledge Use Evaluate Outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION

20. What is needed to sustain ongoing use? What ongoing monitoring of knowledge use and impacts needed? How long are the skills/knowledge maintained? Sustain Knowledge Use Monitor Knowledge Use Sustain Knowledge Use Evaluate Outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION

21. CIHR:Two kinds of KT End of grant KT Integrated KT  The researcher develops and implements a plan for making knowledge users aware of the knowledge generated through a research project  Research approaches that engage potential knowledge-users as partners in the research process.  requires a collaborative or participatory approach to research that is action oriented and is solutions and impact focused (Mode 2).

22. What is integrated KT? a way of doing research collaborative, participatory, action-oriented, community based research, co-production of knowledge, mode 2 knowledge production involves engaging and integrating knowledge users into the research process Knowledge users can be: –Policy- and decision-makers from the community to the federal level, researchers, the public, industry, clinicians, the media –Investigators from different disciplines, teams, countries

23. In addition, knowledge users and researchers (knowledge creators) can work together to: shape the research questions decide on the methodology interpret the study findings and craft messaging around them help with data collection and tools development move the research results into practice widespread dissemination and application What is integrated KT?

24. IKT and QUERI QUERI’s action-oriented approach that meaningfully engages clinicians, managers, patients/clients, and researchers in research-driven initiatives to improve quality = IKT (Mode 2 knowledge production) IS research agenda: –What has been learned so far about IKT at QUERI? –What’s involved in doing QI this way? –How is it working (how effective is it)? –How can it work better? –Strategies and tools to engage knowledge users –Guidelines/criteria for good Mode 2 knowledge production (IKT)

25. Comparing the knowledge to action cycle with the QUERI model

26. Step 1: Select conditions per patient populations associated with high risk of disease and/or disability and/or burden of illness for veterans 1A. Identify and prioritize (via a formal ranking procedure) 1B. Identify high-priority clinical practices and outcomes within a selected condition Step 3: Measure and diagnose quality and performance gaps 3A. Measure existing practice patterns and outcomes across VA and identify variations from evidence-based practices ("quality/performance gaps") 3B. Identify determinants of current practices 3C. Diagnose quality/performance gaps Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION Monitor Knowledge Use Sustain Knowledge Use Evaluate Outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge

27. ------------------------------------------------------------------------------------------------- Step 2: Identify evidence-based guidelines, recommendations and best practices 2A. Identify evidence-based clinical practice guidelines 2B. Identify evidence-based clinical recommendations 2C. Identify evidence-based clinical practices Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION Monitor Knowledge Use Sustain Knowledge Use Evaluate Outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge

28. Adapt Knowledge to Local Context Sustain Knowledge Use Evaluate outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION Monitor Knowledge Use Step 4: Implement improvement programs 4A. Identify improvement/implementation strategies, programs and program components or tools 4B. Develop or adapt improvement/implementation strategies, programs and program components or tools 4C. Implement improvement/implementation strategies/programs to address quality gaps

29. Adapt Knowledge to Local Context Sustain Knowledge Use Evaluate outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION Monitor Knowledge Use Step 4: Implement improvement programs 4B. Develop or adapt improvement/implementation strategies, programs and program components or tools IS Research Agenda:IS Research Agenda: Most efficient and useful methods for adapting guidelines/research to settingsMost efficient and useful methods for adapting guidelines/research to settings How to develop local capacity for adaptation and sustainable process for adaptationHow to develop local capacity for adaptation and sustainable process for adaptation

30. Step 3: Measure and diagnose quality and performance gaps 3A. Measure existing practice patterns and outcomes across VA and identify variations from evidence-based practices ("quality/performance gaps") 3B. Identify determinants of current practices 3C. Diagnose quality/performance gaps 3D. Identify barriers and facilitators to improvement Assess Barriers/facilitators to Knowledge Use Monitor Knowledge Use Sustain Knowledge Use Evaluate outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION

31. Step 3: Measure and diagnose quality and performance gaps 3D. Identify barriers and facilitators to improvement IS research agenda: –Determinants of research use (role of unit/org culture, leadership) –Taxonomy for barriers and facilitators (individual, team, organizational) –Validated measures to identify barriers and facilitators –Most efficient methods of identifying barriers and facilitators (e.g. surveys vs interviews vs key informants) Assess Barriers/facilitators to Knowledge Use Monitor Knowledge Use Sustain Knowledge Use Evaluate outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION

32. Step 4: Implement improvement programs 4A. Identify improvement/implementation strategies, programs and program components or tools 4B. Develop or adapt improvement/implementation strategies, programs and program components or tools 4C. Implement improvement/implementation strategies/programs to address quality gaps Select, Tailor, Implement Interventions Sustain Knowledge Use Evaluate outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION Monitor Knowledge Use

33. Step 4: Implement improvement programs 4A. Identify improvement/implementation strategies, programs and program components or tools 4C. Implement improvement/implementation strategies/programs to address quality gaps IS research agenda: –Effectiveness of implementation strategies (what works for whom, under what conditions, with what evidence (guidelines)?) Methods to determine dose of strategies –Strategies directed at individuals (e.g. psychological) vs social setting/organization (e.g. sociological, organizational) (health system interventions key) Select, Tailor, Implement Interventions Sustain Knowledge Use Evaluate outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION Monitor Knowledge Use

34. Step 5/6: Evaluate improvement programs 5. Assess improvement program feasibility, implementation and impacts on patient, family and healthcare system processes and outcomes 6. Assess improvement program impacts on health related quality of life (HRQOL) Monitor Knowledge Use Monitor Knowledge Use Sustain Knowledge Use Evaluate outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION

35. Step 5/6: Evaluate improvement programs 5. Assess improvement program feasibility, implementation and impacts on patient, family and healthcare system processes and outcomes 6. Assess improvement program impacts on health related quality of life (HRQOL) Evaluate Outcomes Monitor Knowledge Use Sustain Knowledge Use Evaluate Outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION

36. Step 5/6: Evaluate improvement programs 5. Assess improvement program feasibility, implementation and impacts on patient, family and healthcare system processes and outcomes 6. Assess improvement program impacts on health related quality of life (HRQOL) IS research agenda: –Validated measures of knowledge use and impact Evaluate Outcomes Monitor Knowledge Use Sustain Knowledge Use Evaluate Outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION

37. the sound of one hand clapping IS research agenda: –Determinants of sustained knowledge use (organizational, team, individual) –Methods to measure ongoing use and impact –Sustainability of implementation strategies overtime and their impact –Creating sustainable research use organizations and cultures Sustain Knowledge Use Monitor Knowledge Use Sustain Knowledge Use Evaluate Outcomes Adapt Knowledge to Local Context Assess Barriers to Knowledge Use Select, Tailor, Implement Interventions Identify Problem Identify, Review, Select Knowledge Tailoring Knowledge Products/ Tools Synthesis Knowledge Inquiry KNOWLEDGE CREATION

38. Finding your way through the implementation maze Welcome to QUERI! This way for useful advice

39. How to move the implementation science research agenda forward You have the QUERI Framework (PAT) and QUERI Pipeline –Review and refine the frameworks –Consciously categorize all activities by these frameworks to allow better understanding of projects You have selected a classical theory to define how you examine/approach the factors (PARIHS, for example) –Theory testing

40. How to move the implementation science research agenda forward Theories range from the simple to the complex Not practical nor feasible to control and/or measure every possible variable nor to test every construct or hypothesis Need to do enough ground work to understand the implementation specific context and to get a sense of what the key change objectives are important to the knowledge-users in that context Need to balance need for context-specific tailoring with need for replicability and commonality of measurement tools

41. How to move the implementation science research agenda forward View all QI efforts as opportunities to advance implementation science –Be reflective: document processes and experiences (the science and the art) –What are you learning? –Turn tacit knowledge about implementation and implementation research into explicit knowledge- i.e. how to conduct implementation research but also how to implement

42. How to move the implementation science research agenda forward Develop and test measures (barriers assessment, knowledge use, impact, sustainability) Encourage use of common measures (or batteries of measures) to allow comparison across studies

43. How to choose the appropriate intervention Link specific interventions to the barriers –This is an art, not a science –If barrier is knowledge or confidence, then self efficacy theory can be used, if it is lack of awareness, then the transtheoretical model could be used etc. Test methods of tailoring strategies to barriers –involving a large number of potential knowledge- users in this process could contribute to a successful intervention (but this is an empirical question )

44. How to choose the appropriate intervention Decide if you need a single or multiple/complex interventions With multiple interventions – are you targeting the same or different barriers? Are the interventions targeted to different levels: individual, team, organization?

45. How to move the implementation science research agenda forward There are also many theories dealing with behaviour at the individual as well as organizational level (see chapter by Wensing et al. in “From Evidence to Practice” in particular, and the entire book, in general)

46. Whenever you are combining the QUERI framework with other theories, consider documenting your experiences so as to advance understanding of how useful the model is and to provide information to others who are attempting a similar project –Have you been able to derive and test hypotheses and propositions based on the theory? –Does it explain successes and failures? –Does it suggest or provide ways to measure/evaluate the hypotheses? –Does it adequately capture the critical components (the active ingredients) of the implementation process in your project?

47. Back to the idea of working in a living Lab You have a unique opportunity to not only improve health and quality of life of your clients but also to further implementation (KT) science This can be done through 3 general strategies: –Move forward through retrospection and reflection –Make your tacit knowledge explicit through rigorous and “infectious” reflection and documentation –Use conceptual frameworks

48. working in a living Lab You have seen the handout with implementation science goals listed in recent publications Think about how your work can push the implementation science (KT) agenda Relationships seem to be a key driver of implementation efforts – can this be studied and documented as well practiced? Context is seen as important: what are the key features to either sink an implementation effort or successfully launch one?

49. working in a living Lab We need to be able to monitor knowledge use, evaluate outcomes and sustain change To do this effectively, transparently and in a manner that is reproducible, we need appropriate measurement tools It is not helpful if everyone makes up their own every time they need one Identify (adapt or develop) and use psychometrically validated tools

50. working in a living Lab Sustainability is a key issue that needs basic research (how is it defined, how often and under what circumstances does it happen) as well as enabling research (what can be done to ensure sustainability – in a variety of contexts and settings) Think back to past implementation successes and failures: how can theory be harnessed to differentiate them? As implementation scientists, we need to work reflexively: using what we know about knowledge translation to inform and guide our own work

51. working in a living Lab Capacity development –Implementation researchers –Among service providers to effectively facilitate implementation Ensuring relevance with service providers- making QUERI indispensible

52. working in a living Lab QUERI is at the cutting edge of Implementation Science What does QUERI need to do to stay at the cutting edge in the future?

53. Knowledge, if it does not determine action, is dead to us Plotinus (Roman philosopher 205AD-270AD)

54. ian.graham@cihr-irsc.gc.ca Thank you for your attention Merci For more information, visit our web page: http://www.cihr-irsc.gc.ca/e/29418.html http://www.cihr-irsc.gc.ca/f/29418.html