Michelle A. Duda, Ph.D., BCBA, Dean L. Fixsen, Ph.D. & Karen A. Blase Ph.D., Melissa Van Dyke, LCSW Frank Porter Graham Child Development Institute University of North Carolina – Chapel Hill Working Smarter: The implementation journey to improve student outcomes

Implementation Drivers Part 4

The Work of Implementation Changing the behavior of adult human service professionals is hard work It requires a systematic approach to support behavior change of practitioners, volunteers, administrators, and policy makers It requires the application of Implementation Drivers

Implementation Drivers Create the Infrastructure Implementation Drivers are mechanisms that o Help to develop, improve, and sustain practitioners’ ability to implement an intervention to benefit children o Help ensure sustainability and improvement at the organizational level Implementation Drivers

INTEGRATED & COMPENSATORY SUPERVISION & COACHING STAFF PERFORMANCE EVALUATION FACILITATIVE ADMINISTRATIVE SUPPORTS RECRUITMENT AND SELECTION PRESERVICE & IN SERVICE TRAINING SYSTEMS INTERVENTIONS Implementation Drivers DECISION SUPPORT DATA SYSTEMS

Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System CompetencyOrganization Student Benefits Leadership Adaptive Technical Integrated & Compensatory

© Fixsen & Blase, 2008 Student Benefits Performance Assessment (Fidelity) Coaching Competency Drivers Implementation Lens Selection Training

Recruitment and Selection It’s about “choice” and “fit” for: 1.Current Staff 2.Early Adopters 3.Volunteers 4.New Staff

Recruitment and Selection Purposes: Select for the “unteachables” Screen for pre-requisites Make expectations explicit Allow for mutual selection Improve likelihood of retention after “investment” Improve likelihood that training, coaching and supervision will result in implementation

Recruitment and Selection Resources consumed but no new services provided for children Start up may add 10-20% to first year costs Status Quo gets challenged Creating new realities o “Oh, I didn’t understand that this would mean……”

Recruitment and Selection Implementation Best Practices: Job or role description clarity about accountability and expectations Sampling of skills and experience is related to “new practices” and expectations Interactive Interview Process: –Behavioral Vignettes and Behavior Rehearsals –Assessment of ability to accept feedback –Assessment of ability to change own behavior

© Fixsen & Blase, 2008 Student Benefits Performance Assessment (Fidelity) Coaching Competency Drivers Implementation Lens Selection Training

Pre-Service and In-Service Training Purposes: Knowledge acquisition Basic Skill Development “Buy-in” Implementation Best Practices: Theory grounded (adult learning) Skill-based –Behavior Rehearsals vs. Role Plays –Knowledgeable Feedback Providers –Practice to Criteria Feedback to Selection and Feed Forward to Supervision Data-based (pre and post testing)

© Fixsen & Blase, 2008 Student Benefits Performance Assessment (Fidelity) Coaching Competency Drivers Implementation Lens Selection Training

Coaching Impact OUTCOMES % of Participants who Demonstrate Knowledge, Demonstrate New Skills in a Training Setting, and Use new Skills in the Classroom TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10%5%0%..+Demonstration in Training 30%20%0% …+ Practice & Feedback in Training 60% 5% …+ Coaching in Classroom 95% Joyce and Showers, 2002

Supervision and Coaching Purposes: Ensure implementation Develop good judgment Ensure fidelity Provide feedback to selection and training processes

Coaching Implementation Best Practices: Design a Coaching Service Delivery Plan Develop accountability structures for Coaching – Coach the Coach! –Regular satisfaction feedback from employees and volunteers –Regular review of adherence to Coaching Service Delivery Plan –Look at data – Fidelity, Teacher Satisfaction with Support, Skill Acquisition, and….

© Fixsen & Blase, 2008 Student Benefits Performance Assessment (Fidelity) Coaching Competency Drivers Implementation Lens Selection Training Organization Drivers

Performance Assessment Purposes: Ensure implementation Reinforce teachers and build on strengths Develop skills and abilities Measure fidelity Interpret Outcome Data Feedback to school, District(s), RIT on functioning of –Recruitment and Selection Practices –Training Programs (pre and in-service) –Supervision and Coaching Systems

Performance Assessment Implementation Best Practices: Transparent Processes – Orientation –What, When, How, Why Use of Multiple Data Sources –Context –Content –Competency Tied to positive recognition – not used ‘punitively ’

Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System CompetencyOrganization Student Benefits Leadership Adaptive Technical Integrated & Compensatory

Organizational Drivers

Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System CompetencyOrganization Student Benefits Leadership

Purposes: Provide information to assess effectiveness of new educational practices strategies To guide further program and practice development Celebrate success Engage in continuous quality improvement Be accountable for quality infrastructure (are Drivers ‘working’) and for outcomes Decision Support Data Systems

Implementation Best Practices: Includes intermediate and longer term outcome measures Includes process measures (fidelity) Measures are “socially important” Useful data are: Reliable (standardized protocols, trained data gatherers) Reported frequently (e.g. weekly, quarterly) Reported at relevant and “actionable” levels (e.g. student, classroom, school) Widely shared Practical to collect Useful for and used for making decisions (PDSA)

Decision Support Data Systems Measure Fidelity AND Measure Outcomes BECAUSE you need to know: Are we having an implementation problem? Low fidelity & Poor outcome = Implementation Problem. Are we having an effectiveness problem? High fidelity & Poor outcome = Effectiveness Problem.

Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System CompetencyOrganization Student Benefits Leadership

Facilitative Administrative Supports No such thing as a purely administrative decision!!....They are all decisions about Quality Education!! Purposes: Facilitates moving through Stages Ensures effective use of Competency Drivers Takes the lead on Systems Interventions Utilizes PEP – PIP cycles and data for improvement Looks for ways to make work of teachers and staff easier and more effective!!

Facilitative Administrative Supports Implementation Best Practices: An Implementation Team (e.g. School, District Leadership team) is formed and functional Uses feedback to make changes in Implementation Drivers Revises policies and procedures to support the new way of work Solicits and uses feedback from teachers and staff Reduces administrative barriers

Reflection Would you introduce the concept of Facilitative Administration to a school or Division? 1.Why or Why not? 2.How and When? 3.Challenges? 4.Benefits?

Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System CompetencyOrganization Student Benefits Leadership

Systems Interventions Systems Can Trump Programs! - Patrick McCarthy, Annie E. Casey Foundation At: School level Division level State level Federal level

Systems Interventions Purposes: Identify barriers and facilitators for the new way of work Create a “hospitable” environment for the new way of work Contribute to cumulative learning in multi-site projects. Implementation Best Practices Match leadership level needed to intervene Engage and grow “champions” and “opinion leaders” Objectively document barriers Establishes formal PEP – PIP cycles Uses Transformation Zones to –Identify Systems Issues –Create time-limited, barrier busting processes Make constructive recommendations and assist in implementing and evaluating them (PDSA)

For More Information Dean L. Fixsen, Ph.D Michelle A. Duda, Ph.D At the Frank Porter Graham Child Development Institute University of North Carolina Chapel Hill, NC

For More Information Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Download all or part of the monograph at: Monograph/ Monograph/ To order the monograph go to: