1. Michelle A. Duda, Ph.D., BCBA, Dean L. Fixsen, Ph.D. & Karen A. Blase Ph.D., Melissa Van Dyke, LCSW Frank Porter Graham Child Development Institute University of North Carolina – Chapel Hill Working Smarter: The implementation journey to improve student outcomes

2. Innovation Fluency Part 1

3. A Problem Too often, the choice of an intervention to implement has had little to do with the quality or weight of the evidence regarding practices Decisions are made for a variety of reasons not necessarily related to the data about effectiveness (e.g. philosophy, values, comfort, acceptance, availability, finances)

4. Follow Through Programs Figure 1: This figure shows the average effects of nine Follow Through models on measures of basic skills (word knowledge, spelling, language, and math computation), cognitive-conceptual skills (reading comprehension, math concepts, and math problem solving) and self-concept. This figure is adapted from Engelmann, S. and Carnine, D. (1982), Theory of Instruction: Principles and applications. New York: Irvington Press.

5. Innovation Fluency Definition: Innovation Fluency refers to the degree to which the team knows the innovation with respect to: 1.Evidence 2.Program and Practice Features 3.Intervention Readiness for Replication 4.Implementation Capacity Requirements Because all of the above impact the success of implementation efforts

6. Intervention Science Research has identified programs and practices that can help students, communities, and education systems What Works Clearinghouse (http://ies.ed.gov/ncee/wwc/)http://ies.ed.gov/ncee/wwc/ NREPP (http://www.nrepp.samhsa.gov/)http://www.nrepp.samhsa.gov/ Colorado Blueprints (http://www.colorado.edu/cspv/index.html)http://www.colorado.edu/cspv/index.html Education research journals/ American Education Research Association (http://www.aera.net/)/ J. of Evidence-Based Practices in Schoolshttp://www.aera.net/)/

7. 1. Research View of Evidence Descriptions of practices Observational & survey information Correlational studies Quasi-experimental studies Randomized clinical trials (programs) o in situations of convenience o in clinical context Single-subject experiments (practices, components, programs, systems)

8. A Different View of Evidence From an implementation perspective, what do we need to know about evidence?

9. 1. Evidence Related to the Intervention o Theoretical Foundations – Intervention  What principles make “it” work? o Empirical Support – Intervention  How much research has been done?  How much of a difference could it make (e.g. effect size)? o Fidelity Data (e.g. predictive and discriminative) o Practical to Collect and Analyze o Cost-effectiveness Data o Populations (e.g. culture, race, language, ethnicity) o Expected Student Outcomes – Intervention  What outcomes have been achieved in the “real world”?

10. 2. Program and Practice Features o Start where it matters: At the interface between Teachers and Students o Build on those Program and Practice features that interface as the foundation, then…. o Identify what needs to happen at the classroom level so the teacher can interact with students as intended o Identify what needs to happen at the school level to support what happens at the classroom level o Identify what needs to happen at the Division level to support school and classroom implementation o ….and the State?

11. A Different View of Program and Practice Features From an implementation perspective, what do we need to know about P & P Features?

12. 2. Program and Practice Features 1.What you can observe happening at classroom, building, and within the division level ? 2. Fit with current priorities and issues 3. Resource Requirements

13. Program and Practice Features through an Implementation Lens o Fit with:  Current District and School Initiatives & Priorities  AYP Priorities  RtI Processes  Current Organizational Structures at School and District Level  Parent and Community Values – Social Validity  Current skill set of staff expected to implement the innovation o Resource Requirements  Materials  Staffing (recruitment, reallocation)  Training costs  Coaching and Supervision  IT and Data Requirements  Administrative time and supports Do you Recognize The Implementation Drivers?

14. Innovation Fluency The “it” must be operationalized whether it is:  An Evidence-Based Practice or Program  A Best Practice Initiative  A Systems Change Initiative Operationalize  Part of Speech: verb Definition: to define a concept or variable so that it can be measured or expressed quantitatively  Webster's New Millennium™ Dictionary of English, Preview Edition (v 0.9.7) Copyright © 2003-2008 Lexico Publishing Group, LLC

15. Practice Profile Increased Innovation Fluency Through the Development and Use of Practice Profiles Identifies critical components For each critical component: o Identifies gold standard o Identifies acceptable variations in practice o Identifies ineffective practices and undesirable practices o Heartland Area Education Agency 11, Iowa

16. Practice Profiles Each critical component is a heading Each level of implementation becomes a dimension on the rubric associated with that critical component. Critical Component Ideal Implementation Acceptable Variation Unacceptable Variation Critical Component 1: Description Description of implementer behavior Drastic Mutation Adapted from work by Heartland Area Education Agency 11, Iowa

17. ACTIVITY ONE Critical Component Ideal Implementation Acceptable Variation Unacceptable Variation (Drastic Mutation) ESD Team Membership Team Has broad representation Membership includes invested individuals with knowledge about behavior and ESD implementation. Team Incl. Behavior Specialist, Principal, 3 lead teachers, parent, school psych Membership includes invested individuals with knowledge about behavior and ESD implementation. Team Incl. Behavior Specialist, Principal, 2 lead teachers and a parent Membership includes individuals with minimal knowledge about behavior and ESD implementation. Team incl. Admin Assistant and 2 teachers Membership includes individuals with common planning periods. Team includes 3 teachers, but team members change each semester Part ONE: Practice using the practice profile format. EXAMPLE: School Level

18. ACTIVITY ONE Critical Component Ideal Implementation Acceptable Variation Unacceptable Variation (Drastic Mutation) Link school teams to supporting resources Part ONE: Practice using the practice profile format. EXAMPLE: Coaching/District Level

19. ACTIVITY ONE Part Two: Reflection Questions How does a schools' implementation of a core feature support the division goals/outcomes? How does the division support the school in implementing these core features?

20. 3. Intervention Readiness for Replication o What are the Qualified “purveyor” and researcher connections? o Are the Content ‘experts’ and qualified TA available? o Have Replication Success (e.g. % of attempted replications still implementing) been demonstrated? o Are the Core Intervention Components (program features) well operationalized so that Practice Profiles can be developed if necessary? o Have the Implementation Drivers been:  Well operationalized (Selection, Training, Coaching, Data processes, etc)?  Demonstrated to be effective (e.g. change & sustain new teacher behavior)?  Practical and affordable (e.g. Drivers have been utilized in typical settings)?

21. 4. Implementation Capacity Requirements o Sufficient Staff who meet minimum requirements o Region/Division prepared to define practice/program features and elements if necessary (e.g. Practice Profiles) o Region/Division able to sustain Implementation Drivers  Fiscally  Structurally  Functionally o Buy-in Process Operationalized  School  District  Parents

22. Innovation Fluency Implementation Teams need to understand the innovation so that they can: o Help Districts/Divisions “choose” wisely  Address a prevalent and significant need  Worth it to scale-up (e.g. effect size of.50 or greater) o Evaluate the innovation’s readiness for implementation o Assess the implementation capacity of the District in relation to a number of factors related to the innovation

23. For More Information Dean L. Fixsen, Ph.D. 919-966-3892 fixsen@mail.fpg.unc.edu Michelle A. Duda, Ph.D. 919-636-0843 duda@mail.fpg.unc.edu At the Frank Porter Graham Child Development Institute University of North Carolina Chapel Hill, NC www.scalingup.org http://nirn.fpg.unc.edu/ http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/

24. For More Information Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Download all or part of the monograph at: http://www.fpg.unc.edu/~nirn/resources/publications/ Monograph/http://www.fpg.unc.edu/~nirn/resources/publications/ Monograph/ To order the monograph go to: https://fmhi.pro-copy.com/