1. Michelle A. Duda, Barbara Sims, Dean L. Fixsen, Karen A. Blase, National Implementation Research Network FPG Child Development Institute University of North Carolina at Chapel Hill Santa Clara County Office of Education August 2012 Leading Systems Development for the Implementation of the Common Core State Standards

2. Introduction and Overview of Implementation Science- Four Active Implementation Frameworks Implementation Team Improvement Cycles Implementation Drivers Implementation Stages Moving from “theory to practice” – How to use implementation science to create the Infrastructure and readiness for CCSS Goals for Today’s Session

3. Reliably Produce Predictable outcomes for children, families and communities – (i.e. achieving outcomes defined by the CCSS) That improve every year For the next 50 years Our Broader Goals

4. The Challenge We work in complex environments with: Unpredictable people Competing demands Shifting priorities Various points of view Education

5. Students cannot benefit from interventions they do not experience FACT

6. Good science (evidence-based practices) leads to Good practices (implementation science) FICTION

7. National Assessment of Education Progress (NAEP) WHAT IS IT GOING TO TAKE?

8. The Challenge…. Science to Service Gap – What is known is not what is adopted to help students Implementation Gap – What is adopted is not used with fidelity and good outcomes – What is used with fidelity is not sustained for a useful period of time – What is used with fidelity is not used on a scale sufficient to broadly impact student outcomes

9. Implementation Gap RESEARCH PRACTICE GAP Implementation is defined as a specified set of activities designed to put into practice an activity or program of known dimensions. IMPLEMENTATION Why Focus on Implementation? SO THAT, ALL students CAN benefit from interventions they DO experience Implementation

10. Implementation Science EffectiveNOT Effective Effective NOT Effective IMPLEMENTATION INTERVENTION Actual Benefits (Institute of Medicine, 2000; 2001; 2009; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education, 1983; Department of Health and Human Services, 1999) Inconsistent; Not Sustainable; Poor outcomes Unpredictable or poor outcomes; Poor outcomes; Sometimes harmful Implementation Science

11. Data Show These Methods, When Used Alone, Do Not Result In Implementation As Intended: – Diffusion/ Dissemination of information – Training – Passing laws/ mandates/ regulations – Providing funding/ incentives – Organization change/ reorganization Necessary But Not Sufficient

12. Evidence- based Actual Supports Years 1-3 Outcomes Years 4-5 Every Teacher Trained Fewer than 50% of the teachers received some training Fewer than 10% of the schools used the CSR as intended Every Teacher Continually Supported Fewer than 25% of those teachers received support Vast majority of students did not benefit Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006 Longitudinal Studies of a Variety of Comprehensive School Reforms Implementation Science Implementation of School Reform

13. Plan for Change It is not a “school problem” District is the point of entry for school improvement – Use short-term infusion of resources – Establish long-term, district-based capacity for quality Plan for Change

14. Shifting Accountability SISEP 2012 Student Practitioner System Shifting Accountability

15. Making It Happen Letting it happen – Recipients are accountable Helping it happen – Recipients are accountable Making it happen – Purposeful use of implementation practice and science – Implementation system is accountable Based on Hall & Hord (1987); Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou (2004); Fixsen, Blase, Duda, Naoom, & Van Dyke (2010) SISEP 2012 Making It Happen

16. “Making It Happen” To successfully implement and sustain evidence-based and evidence-informed interventions, we need to know: – The WHAT - What is the intervention (e.g. effective instruction, effective assessment) – The HOW - Effective implementation and sustainability frameworks (e.g. strategies to change and maintain behavior of adults) – The WHO – Organized, expert implementation assistance Making It Happen

17. HTTP://NIRN.FPG.UNC.EDU Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Implementation Research: A Synthesis of the Literature Implementation Science

18. Active Implementation Frameworks Testable Hypotheses Organize Implementation Knowledge Identify Gaps Guide Development of Implementation Capacity Active Implementation Frameworks

19. So that Work of Implementation positive outcomes are more frequently created for students structures, cultures, and climates change to support new ways of work implementation is supported and outcomes can be achieved Adult human service professionals (teachers, staff) Administrators in organizations (Principals, Assistant Principals) System directors, policy makers, and funders (District Staff, SEA) Changing the thinking and behavior of:

20. Reflection Implementation Science Why is it important to pay attention to the implementation process as we are creating our action plans for CCSS? Supporting New Ways of Work

21. Implementation Teams Improvement Cycles Implementation Drivers Implementation Stages Active Implementation Frameworks Organized, expert assistance to develop and sustain an accountable structure

22. WHO will do the work? Purveyors – a group of individuals representing a program or practice who actively work with organizations and communities to help them implement that practice or program with fidelity and good effect An Intermediary Purveyor Organization that becomes expert in implementation and a “bridge” or expert with multiple EBPs (a new way of doing T & TA) connecting providers and purveyors “Local” Implementation Team with the knowledge, skill, freedom, and authority to act (e.g. within a larger organization or a collaboration of agencies)

23. Impl. TeamNO Impl. Team Effective Making it Happen IMPLEMENTATION INTERVENTION 80%, 3 Yrs 14%, 17 Yrs Balas & Boren, 2000 Green & Seifert, 2005 Fixsen, Blase, Timbers, & Wolf, 2001 Letting it Happen Helping it Happen Implementation Teams

24. School-based Implementation Team School-based District-based District-based Regionally-based Regionally-based State-based State-based “We tend to focus on snapshots of isolated parts of the system and wonder why our deepest problems never seem to get solved. (Senge, 1990) Linked Team Structures

25. Implementation Team : Sphere of Influence Implementation Teams Prepare & Work with Counties, Communities Prepare Staff & Administrators T/TA, Purveyors & Researchers Assure Implementation State, Tribe Assure Child and Family Benefits Family & Stakeholders Create Readiness

26. Membership Teams whose members collectively represent the “system as is” and the “system to be” Know the innovation(s) very well (formal and practice knowledge) Know implementation very well (formal and practice knowledge) Know improvement cycles to make intervention and implementation methods more effective and efficient over time Promote and participate in systems change at multiple levels to create hospitable cultures, policies, and funding streams Implementation Teams

27. What does this mean for us? Implementation Teams ensure local capacity Attend to systems alignment Representation from those who know the “what” and the “how” Use improvement cycles for purposeful adjustments

28. Reflection Implementation Teams Do you currently have an Implementation Team? Who are members you might include on your Implementation Team? Supporting New Ways of Work

29. Implementation Teams Improvement Cycles Implementation Drivers Implementation Stages Active Implementation Frameworks Changing on purpose to support the new way of work

30. Changing on Purpose People, organizations, and systems Cannot change everything at once Cannot stop and re-tool Cannot know what to do at every Many outcomes are not predictable Trial & Learning Improvement Cycles

31. PDSA Cycles 1. Rapid cycle problem solving (Shewhart; Deming) 2. Usability testing (Neilson; Rubin) 3. Transformation Zones 4. Practice-policy communication loops Improvement Cycles

32. SISEP 2012 Improvement Cycles Rapid Cycle Problem Solving

33. Usability Clear description of the program Trial and learning approach Small number of participants with multiple iterations Rapid cycle problem solving applied to program and system Pilot Clear description of the program Trial and learning approach Sufficient number of participants and sufficient time to realize potential results SISEP 2012 Usability vs. Pilot Testing Improvement Cycles

34. Transformation Zone A “vertical slice” of the service system (from the classroom to the District, Region, State) – The “slice” is small enough to be manageable – The “slice” is large enough to include all aspects of the system – The “slice” is large enough to “disturb the system” – a “ghost” system won’t work. Improvement Cycles

35. Policy Practice Feedback Loops Policy (Plan) Practice (Do) Feedback Study - Act Policy Enabled Practices (PEP) Practice Informed Policy (PIP) Expert Implementation Support Policy Practice Structure Procedure FORM SUPPORTS FUNCTION

36. What does this mean for us? Change is difficult Improvement Cycles provide a process for change PDSA cycle is used so we get better and better Practice-Policy Feedback Loops promote hospitable environments for effective practice

37. Reflection Improvement Cycles and Communication Loops How can we make use of improvement cycles in developing and implementing our improvement activities? Linking Communication Protocols Supporting New Ways of Work

38. Implementation Teams Improvement Cycles Implementation Drivers Implementation Stages Active Implementation Frameworks Common features of successful supports to help make full and effective use of a wide variety of innovations

39. What are Implementation Drivers? Mechanisms to: Develop, improve, and sustain one’s ability to implement an intervention to benefit students (Competency Drivers) Create and sustain hospitable organizational and systems environments for effective educational services (Organization Drivers) Ensure continuous improvement cycles are moving information forward and information backward to improve alignment overtime (Leadership Drivers) Implementation Drivers

40. Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System Integrated & Compensatory TechnicalAdaptive Reliable Benefits and Consistent Use of Interventions © Fixsen & Blase, 2007 Staff Competence Organization Supports Leadership

41. Competency Drivers Build Competency and Confidence Develop, improve, and sustain competent & confident use of innovations Competency Drivers

42. Measure fidelity Ensure implementation Reinforce staff and build on strengths Feedback to agency on functioning of – Recruitment and Selection Practices – Training Programs (pre and in-service) – Supervision and Coaching Systems – Interpretation of Outcome Data Competency Drivers Performance Assessment

43. Selection Select for the “unteachables” Screen for pre-requisites Set expectations Allow for mutual selection Improve likelihood of retention after “investment” Improve likelihood that training, coaching, and supervision will result in implementation Competency Drivers

44. Training Timeliness “Buy-in” Knowledge acquisition Skill Development Competency Drivers

45. Training and Coaching OUTCOMES % of Participants who Demonstrate Knowledge, Demonstrate New Skills in a Training Setting, and Use new Skills in the Classroom TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10%5%0%..+Demonstration in Training 30%20%0% …+ Practice & Feedback in Training 60% 5% …+ Coaching in Classroom 95% Joyce and Showers, 2002 Training and Coaching

46. Coaching Ensures fidelity Ensures implementation Develops clinical and practice judgment Provides feedback to selection and training processes Grounded in “Best Practices” Competency Drivers

47. Organization Drivers Change Organizations and Systems Create and sustain hospitable organizational and system environments for effective services Organization Drivers

48. successful organizational change system change What do we need to know about successful organizational change and system change methods? Implementation Drivers

49. Creating Capacity for Competent Change New innovations do not fare well in old organizational structures and systems Develop new position descriptions and job functions in State Departments of Education and in Regional and District systems “Systems trump programs.” Patrick McCarthy, Annie E. Casey Foundation Creating Capacity for Competent Change

50. Decision Support Data Systems Improves student outcomes through data-based decisions Provides information to assess effectiveness of intervention and prevention practices Analyzes the relationship of fidelity to outcomes Guides further program development Engages us in continuous quality improvement Celebrate success Be accountable to consumers and funders Organizational Drivers

51. Facilitative Administration Facilitates installation and implementation of the Drivers Aligns policies and procedures Takes the lead on Systems Interventions Looks for ways to make work of practitioners and supervisors more effective and less “burdensome”!! Organizational Drivers

52. Systems Intervention Identifies barriers and facilitators for the new way of work Creates an externally and internally “hospitable” environment for the new way of work Contributes to cumulative learning in multi-site projects Organizational Drivers

53. EXISTING SYSTEM Effective Innovations are Changed to Fit the System Or Operate in the Shadows (The “Ghost” System) What do we know about System Stability?

54. EXISTING SYSTEM Effective Innovations are Changed to Fit the System Or Operate in the Shadows (The “Ghost” System) What do we know about Effective System Stability? EXISTING SYSTEM IS CHANGED TO SUPPORT THE EFFECTIVENESS OF THE INNOVATION (“Host” System) EFFECTIVE INNOVATION

55. Leadership Drivers Different challenges call for different strategies – Technical Strategies – Adaptive Strategies Leadership Drivers

56. Strategies According to Ron Heifetz and his colleagues at Harvard’s Kennedy School of Government, one of the biggest mistakes “leaders” make is to incorrectly identify the type of challenge they are facing – Using technical approaches for adaptive issues (and vice versa) Leadership Drivers

57. Technical Challenges Perspectives are aligned (views, values) Definition of the problem is clear Solution and implementation of the solution is relatively clear There can be a “primary” locus of responsibility for organizing the work Leadership Drivers

58. Technical Strategies Use established norms/ goals Define problems Provide solutions Clarify roles and responsibilities Assign tasks Manage conflict Maintain order Leadership Drivers

59. Adaptive Challenges Legitimate, yet competing, perspectives emerge Definition of the problem is unclear There are different perspectives on the “issue” at hand Solution and implementation is unclear and requires learning Primary locus of responsibility is not a single entity or person Leadership Drivers

60. Adaptive Strategies Get on the Balcony Identify the Adaptive Challenge Regulate Distress Maintain Disciplined Attention Give the Work Back to the People Protect All Voices Ron Heifetz, Leadership without Easy Answers, 1996 Leadership Drivers

61. Benefits of Driver-Based Action Planning Infrastructure needed becomes visible to all Strengths and progress get celebrated Next right steps are planned and results measured Resources can be aligned and re- purposed to improve implementation Benefits of Driver Based Action Planning

62. Reflection Implementation Drivers Do we need to promote ‘competence and confidence’ of educators? How can we build in- state capacity to select, train, coach, and assess performance well? How are we engaged in promoting more hospitable organizational environments? Do we have support of leadership? How do we know? Supporting New Ways of Work

63. Implementation Teams Improvement Cycles Implementation Drivers Implementation Stages Active Implementation Frameworks Purposeful matching of critical implementation activities to the stage of the process

64. Implementation Takes Time: 2 – 4 Years EXPLORATION INSTALLATION INITIAL IMPLEMENTATION FULL IMPLEMENTATION Stages of Implementation Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 AWARENESS TRANSITION FULL IMPLEMENTATION

65. Exploration Exploration (Awareness) Stage Goals – Create readiness for change Changing hearts and minds – Examine degree to which the proposed strategies and practices meet the needs of our State and our students – Determine whether adoption and implementation are desirable and feasible “Pay now or pay later.” Stages of Implementation

66. Exploration What happens during Exploration (Awareness) Stage? Formalize Team Structures Develop Communication Plan Determine Need and Identify Options Assess “Fit” and Feasibility Promote “Buy in” for the innovation and for implementation supports Make recommendations Stages of Implementation

67. EBP: 5 Point Rating Scale: High = 5; Medium = 3; Low = 1. Midpoints can be used and scored as a 2 or 4. HighMediumLow Need Fit Resources Availability Evidence Readiness for Replication Capacity to Implement Total Score: Need in the Educational Setting, Socially Significant Issues, Parent & Community Perceptions of Need, Objective Data indicating Need Need Fit Fit with current - Initiatives RtI Implementation School and District Priorities Organizational structures Community Values Resource Availability Resources Curricula & Classroom Materials, IT requirements, Staffing, Training and PD, Data Systems, Coaching & Supervision, Administrative & system supports needed Evidence Outcomes – Is it worth it? Fidelity data Cost – effectiveness data Number of studies Population similarities Diverse cultural groups Efficacy or Effectiveness Evidence Assessing Evidence-Based Programs and Practices Readiness Qualified purveyor Expert or TA available Mature sites to observe # of replications How well is it operationalized? Are Imp Drivers operationalized? Intervention Readiness for Replication Capacity Staff meet minimum qualifications Able to sustain Implementation Drivers Financially Structurally Buy-in process operationalized Educators Administrators Families Capacity to Implement © National Implementation Research Network- 2009

68. Reflection Creating Readiness What role can you play in developing readiness for building in CCSS in your district? What are 2 things your team could to tomorrow to asses your current infrastructure? District Initiative Inventory Analysis of Evidence-based Programs or Practices “aka” hexagon tool Supporting New Ways of Work

69. Installation Installation (Transition) Stage Goals Structural and functional changes are made Selection protocols developed First implementers selected Define and initiate training of first implementers Develop coaching plans Evaluate readiness and sustainability of data systems Stages of Implementation

70. Installation What’s Needed: High-level protection, problem solving, and support Reduced expectations and higher costs during start up Help in evolving organizational supports at every level Help in establishing new school, community, and organizational climate and culture Stages of Implementation

71. Initial Implementation Initial Implementation (Transition) Stage Goals -Get started, then get better! Learn from mistakes Celebrate progress Continue “buy-in” efforts Make systemic changes Manage expectations -All the components of the program or innovation are in place and the implementation supports begin to function Stages of Implementation

72. Initial Implementation Work through the Awkwardness – Managing Change – Managing Expectations Provide training and coaching on the evidence-based practice, re-organization of school roles, functions and structures Make use of improvement cycles to resolve systems issues Stages of Implementation

73. Full Implementation Full Implementation Stage Goals – Maintaining and improving skills and activities throughout the system – Components integrated, fully functioning – Skillful practices by front line staff, supervisors, administrators – Changes in policy that are reflected in practice at all levels – Ready to be evaluated for expected outcomes Stages of Implementation

74. Full Implementation “What Change?....This is our way of work!” – Skillful Teaching and School Practices – Skillful Use of the Drivers Drivers experience their own Improvement Cycles – Data Systems in use, reliable, efficient, and used for Decision-Making at multiple levels to regenerate and improve – Policy to Practice and Practice to Policy Feedback Cycles Stages of Implementation

75. Reflection What are you already doing that is “stage-based”? What are the facilitators and barriers to doing stage-based work? – Stages of Implementation Analysis Supporting New Ways of Work

76. California Common Core State Standards District level system change – Information, what it is, is not – Stakeholder buy-in, – Application/selection process Informed agreement, understand and defend the initiative What will it Take?

77. California Common Core State Standards District level system change – Give lots of rationales – Not a project, not patchwork – Focus on CCSS functions – Establish a common vocabulary to ease communication – Build on what folks are doing already – help them get ready for change What will it Take?

78. California Common Core State Standards Guided development – Leadership involvement (require more in the future) – Year of training with follow up coaching (require more on-site visits in the future) – Include leaders in the training (well informed, able to explain and defend, willing to do what is required) What will it Take?

79. California Common Core State Standards Manage expectations and pace based on Stage of Implementation Teachers & Staff directly impact students It is the job of principals, superintendents, and funders to align policies and structures to facilitate effective teacher & staff practices There is no such thing as an “administrative decision” – they are all education decisions What will it Take?

80. The Work of Implementation Changing the behavior of education system professionals is hard work It requires a systematic approach to support behavior change of – Teachers, – School and district personnel – TA providers, – State department of education personnel It requires both effective practices/programs (the “what”) and effective implementation processes (the “how”)

81. Stay Connected! www.scalingup.org SISEP @SISEPcenter For more on Implementation Science http://nirn.fpg.unc.edu www.implementationconference.org

82. For More Information Michelle A. Duda, Ph.D., BCBA-D duda@unc.edu Frank Porter Graham Child Development Institute University of North Carolina Chapel Hill, NC http://nirn.fpg.unc.edu/