Bridging the Research to Practice Gap: Perspectives from the Practice Side Ronnie Detrich Wing Institute

Questions What do you think when you hear the phrase “research to practice” gap? How often do you interact with researcher/practitioner in collaborative relationship? As practitioner what keeps you from being more research-based? As practitioner how often do you give feedback to researchers about practices?

Is There a Research to Practice Gap? Concern is that research based practices are not showing up in practice

Scope of the Problem 550 named interventions for children and adolescents Behavioral Cognitive- behavioral Empirically evaluated? Evidence-based interventions are less likely to be used than interventions for which there is no evidence or there is evidence about lack of impact. Kazdin (2000)

Are We Training Educators to be Evidence- based? Survey of School Psychology Directors of Training 29% Evidence-based interventions Knowledge (Shernoff, Kratochwill, & Stoiber, 2003) training 41% programs

Changing the Conversation

Why Not a Practice to Research Gap Out of necessity, practitioners are innovating in almost every case/project. These innovations rarely are communicated to researchers.  Creating a Practice to Research Gap

Why is There a Gap? Technical challenges getting research to practitioners. Cultural differences between research and practice.

Technical Challenges Dissemination problem.  Researchers primary form of dissemination is professional journals and conferences.  Research published in diverse journals.  Journals are mechanisms for researchers talking to each other.  Practitioners don’t read professional journals often enough to keep informed. Data suggest practitioners read very infrequently. Often a problem of access Certainly a problem of time.

Cultural Differences: Assumptions, Values, Goals Researchers Knowledge creation Identify relations among variables. Research will lead to social benefit someday. Place high value on “formal” knowledge. High value on rigor Practitioners “Doing good’ helping others. Practice results in benefit now. “Craft” skills highly valued. High value on relevance

Clash of Cultures for Researchers and Practitioners Researchers Publication Promotion and tenure Funding cycles Practitioners Legal Regulatory Policy Budget

Influences on Researchers Publication  Original research  Experimentally well controlled Promotion and Tenure  High level of productivity: publication rates is one of the primary measures. Funding Cycles  Typically three year cycles.

Influences on Researchers Greater emphasis on efficacy than effectiveness.  Journals and promotion and tenure rules place greater value on original research. Effectiveness research is essentially replication research.  Short funding cycles encourage efficacy research.  Typically efficacy requires less time and resources than effectiveness research, resulting in higher rates of published research. Questions related to generality (external validity, relevance) are slow to be answered.

Dissemination of Research and the Practitioner The sources of influence associated with promotion and tenure encourage dissemination through professional journals. Studies published across a large number of journals make it difficult for practitioners to have breadth of knowledge in a particular area. There are no direct requirements on researchers to make results available to practitioners.  Interaction largely between researchers rather than researchers and practitioners.

Dissemination of Research and the Practitioner What Works Clearinghouse and Campbell Collaboration are recent attempts to make evidence available to practitioners in usable form. Current limitations with clearinghouse approach  Standards for quantity and quality of research. It is necessary for practitioners to have data now or will use other criteria to determine interventions. There is not always a large body of evidence. High quality data not always available. Must address issue between most rigorous evidence and best available evidence.

Relevant Rigor Researcher Concerns Internal Validity Practitioner Concerns External Validity

Low Relevant High Low Practice Based Evidence Pseudo-Science High Effectiveness Research Efficacy Research Rigor

Low Relevant High Low Practice Based Evidence Pseudo-Science High Effectiveness Research Efficacy Research Rigor

Efficacy Research (What Works?) Largely concerned with integrity of independent variable (internal validity).  Research is often conducted in analog settings where all relevant variables can be controlled.  Studies conducted by well trained graduate students and research assistants.  Very close oversight to assure integrity.  Funded by research grants.

Effectiveness Research (When Does it Work) Concerned with “robustness” of an intervention when implemented in typical practice settings by usual care staff.  Answers questions related to external validity or generalizability of effects.  Typically smaller magnitude of effect. Likely a function of poor treatment integrity.

Impact of Efficacy Research on Practitioners Often seen as:  Irrelevant because of the analog nature of the work (“not the real world”).  Impractical because of the level of training required.  Impossible because of the resources required. Researchers are engaged in behavior that is not always highly valued by practitioners.

Influences on Practitioners Practitioners responsible to provide educational services for all children (ESEA/IDEIA). Eligibility categories for special education are very broad and are not diagnostic categories.  Researchers often narrow characteristics of students for the purpose of research. As an example, Learning Disabilities includes a very broad range of characteristics and is not particularly meaningful when developing an intervention for a specific student.  Students in special education programs often have co-morbid conditions which may limit effects of a particular intervention. Data are not readily available on effective interventions for students with co-morbid conditions because researchers exclude these students from studies.

Influences on Practitioners An insufficient number of practitioners have necessary training to implement evidence-based interventions.  Failure to have well trained, qualified staff will result in something other than evidence-based interventions being implemented. Even if well trained, necessary resources may not be in place to support evidence-based interventions. An insufficient number of decision makers have necessary skills to evaluate research and translate research to practice.

Influences on Practitioners Empirically-supported interventions may be more costly than categorical, “generic” services, i.e., early intensive behavioral interventions for children with autism, resulting in decisions to provide more costly services only following litigation. Evidence-based practice may be seen as a fad and discounted without examination.

Direction of Influence Research Practice Research Practice Unidirectional Feedback Loop Bidirectional

Closing the Gap Requiring practitioners to be researchers will not be effective, Practitioners should be consumers of research but not primary source journals.  What Works Clearinghouse and Best Evidence Encyclopedia.  Will be useful only to the extent it solves a problem for the practitioner.

Closing the Gap Create occasions for researchers and practitioners to interact.  Communities of Practice. Create context for learning difference in values and assumptions. Context for collaboration once common ground is found.

Closing the Gap Universities teach interventions that have empirical support.

Do We Have the Necessary Conditions for Closing the Gap? Not yet. Shared perspective between researchers and practitioners not readily apparent. Very little research on bridging research and practice (Schoenwald & Hoagwood, 2001; Ringeisen, Henderson, & Hoagwood, 2003).

Where to Start? Addressing motivation of practitioners to implement empirically-supported interventions.  No Child Left Behind Programmatic research related to“Goodness of fit” between an intervention and the setting in which it is to occur (one size does not fit all). Identify practices that promote sustainability. Establish empirically supported interventions for assuring treatment integrity and necessary levels of integrity to have an effect. Conditions to support effectiveness research (funding, publication guidelines, etc.)

Where to Start? Pre-service training emphasizing evidence-based interventions. Multiple methods and multiple groups evaluating research with transparent standards. Distinguish between best evidence and best available evidence.  Establish a continuum of rigor that reflects current contingencies influencing practitioners. Increase interaction between researchers and practitioners.

Summary Researchers and Practitioners responding to different assumptions, values, goals, and pressures. Research will not be important to practitioners until it solves problems that are important to them. Gap will not be bridged until direct contact with variables that influence each groups behavior. Continuous feedback loop between researchers and practitioners increases chance of success.

Thank you Copies available at winginstitute.org

Influences on Researchers and Practitioners Researchers  Publication  Promotion and tenure  Funding cycles Practitioners  Legal  Regulatory  Policy  Budget

Low Relevant High Low Practice Based Evidence Pseudo-Science High Effectiveness Research Efficacy Research Rigor

Low Relevant High Low Practice Based Evidence Pseudo-Science High Effectiveness Research Efficacy Research Rigor