Design and Critique of Grants for Implementation Research

Opportunity PAR-13-055: Dissemination and Implementation Research in Health Study Section: DIRH

Implementation research: a field of health services research that addresses the gaps between what we know and what we are doing.

I have been impressed with the urgency of doing I have been impressed with the urgency of doing. Knowing is not enough; we must apply. Being willing is not enough; we must do. Leonardo da Vinci

The implementation gap What we know What we are doing CRC screening reduces CRC morbidity and mortality HPV vaccination reduces the rate of cervical cancer The Diabetes Prevention Program can reduce the risk of diabetes by 40-50% High-risk women should be screened for BRCA mutations ~50% of patients > 50 are UTD with CRC screening Kansas ranks 51st out of 50 states in HPV vaccination There is no standardized application of the DPP at KUMC Many high risk women remain unscreened

Purpose of Research Identify INNOVATIVE approaches To Answer SIGNIFICANT questions What? Why? How?

What constitutes a good research idea? Feasible2 Subjects readily available Technical expertise Affordable (time/money) – Fundable Manageable in scope Interesting1 Investigator Novel1 New, extends, refutes, or confirms prior findings Ethical2 Relevant1 Scientific knowledge Patient care Public policy Future research 1Is it exciting? 2Can you do it?

NIH Review Criteria Overall Impact 1Is it exciting? 2Can you do it? Significance1 Investigator(s)2 Innovation1 Approach2 Environment2 1Is it exciting? 2Can you do it?

Characteristics of reviewers Implementation methodologists Clinicians Physicians, PT, OT, social workers, psychologists, nutritionists Community engagement 2 of 3 reviewers will have strong opinions about your problem and how it should be addressed

What makes an implementation research project significant? The problem? OR The solution (innovation)? The implementation methodology? What is the ‘implementation problem?

Significance: Common failings Only talk about the significance of the problem One-off innovations Don’t address evidence base of how the innovation may have been used elsewhere e.g. patient navigators for cervical cancer screening Separate ‘scientific premise’ section?

Addressing the scientific premise What is the evidence base for addressing the problem? How does your ‘innovation’ build on this evidence base? What is known about the problem of implementing the innovation? How does your ‘implementation strategy’ build on what is known?

Innovation This is the first project to… Problem Innovation Address this problem Innovation Test this innovation Apply this innovation to this problem Implementation Use this implementation strategy Apply this strategy to this problem/setting Analysis Apply a new theoretical framework to the analysis

Innovation – Common problems Simply repeat the significance setting Don’t frame issues as innovative Too many randomized clinical trials* *personal opinion

Step-wedged design

Investigators Setting Problem Implementation process Primary care physicians if working in primary care Problem e.g oncologists to work with survivorship care plans Implementation process Experience with implementation resarch Measurement and analysis Statistics Qualitative methods

Investigators – Common problems Lack of experience (they can’t do it) Poor approach They don’t understand: The setting Implementation research Analysis

Approach Preliminary studies (can they do the work) Theoretical framework (intervention/innovation) Big picture overview Lay out the workflow Measures relate to theoretical framework Framework for implementation assessment

Approach – Common failings Don’t understand the setting – have not thought through the workflow or how the innovation will work Not implementation research (all work done centrally with passive target institution Theoretical framework presented but then not incorporated into approach/evaluation Fail to link the innovation/implementation strategies to previously identified barriers Dump of measures Not paying attention to who is asked to be doing what Fidelity assessments missing (adoption)

Approach – Common failings Inadequate data from pilot or pilot data not published Data from prior studies presented inaccurately Implementation models misapplied/misinterpreted Poorly specified cost analyses Measures not clearly justified Confusing

Theory/Evidence-based Intervention Design and Implementation

Intervention Mapping – 6 steps

www.cfirguide.org

RE-AIM Evaluation methods Reach: number, proportion, and representativeness of participants Effectiveness: impact of intervention on outcomes (positive and adverse) Adoption: number, proportion, and representativeness of settings willing to adopt then intervention Implementation: fidelity of implementation to the effective protocol; consistency of delivery as intended Maintenance: extent to which the intervention, program, or policy becomes institutionalized

Environment Experience in the setting Able to recruit/track participants Data readily available

Impact Advance management of the problem Advance the field of implementation research

Proctor EK Implementation Science 2012; 7:96

Questions?