Presentation on theme: "Investing in local implementation capacity: Statewide scale up a multi-tiered school reform model for fidelity and sustainability Kim St. Martin Assistant."— Presentation transcript:
1 Investing in local implementation capacity: Statewide scale up a multi-tiered school reform model for fidelity and sustainabilityKim St. MartinAssistant Director, MiBLSiSteve GoodmanDirector, MiBLSiOSEP Project Directors’ ConferenceJuly 2013Investing in local implementation capacity: Statewide scale up a multi-tiered school reform model for fidelity and sustainabilitymiblsi.cenmi.org
2 Session DescriptionThis session describes a scale-able and durable statewide model that emphasizes data-based decision making aroundStudent outcomes,Implementation fidelity of effective reading and behavior support practices, andOrganizational structures of supports.Strategic planning focuses on developing intermediate and local school district capacity to implement with durability in a self-modifying system. Suggestions for scaling-up will be provided.
3 What is MiBLSi?A statewide structure to create capacity for an integrated Behavior and Reading Multi-Tiered System of Support (MTSS) that can be implemented with fidelity, is sustainable over time and utilizes data-based decision making at all levels of implementation support
4 Participating Schools The Early Years2009 Schools (150)2006 Schools (50)2007 Schools (165)2008 Schools (95)2004 Schools (21)2005 Schools (31)Participating Schools2000 Model Demonstrations (5)This slides illustrates moving from model demonstration schools to replication schools. Note that it is more likely to replicate in areas of the state where earlier demonstrations have previously been established.4
5 Emphasis of MiBLSi Work Over Time Focus on providing direct PD and TA to Implementers (Team Training, Focus Training)Focus on providing direct PD and TA to Implementers (Team Training, Focus Training)MiBLSi develop capacity for project implementationFocus on providing direct PD and TA to Implementers (Team Training, Focus Training)MiBLSi develop regional capacity for MiBLSi regional implementationFocus on providing direct PD and TA to Implementers and cascading levels of supportMiBLSi develop local capacity for LEA/ISD implementationFocus on providing direct PD and TA to cascading levels of supportLEA/ISD develop local capacity for LEA/ISD implementation
6 Replications Replications Note: the ducks at the end of the row are less clear that the closest ducks (most recent) as we continue each iteration in replications re become clearer in the work we doThrough each replication, we become more clear in our implementation efforts
7 Why we moved from school to district model… Schools do not always get to criteria for fidelity of implementation.Schools do not always maintain implementation effortsSupporting a school based participation model cannot be scaled and is not durable as district based
8 Implementation Science Longitudinal Studies of a Variety of Comprehensive School ReformsEffective InterventionsActual SupportsYears 1-3OutcomesYears 4-5Every Teacher TrainedFewer than 50% of the teachers received some trainingFewer than 10% of the schools used the CSR as intendedEvery Teacher Continually SupportedFewer than 25% of those teachers received supportVast majority of students did not benefitAladjem, D. K., & Borman, K. M. (2006, April). Summary of Findings from the National LongitudinalEvaluation of Comprehensive School Reform. Paper presented at the Annual meeting of the AmericanEducational Research Association, San Francisco, CA.Vernez, G., Karam, R., Mariano, L. T., & DeMartini, C. (2006). Evaluating comprehensive school reform models at scale:Focus on implementation. Santa Monica, CA: RAND Corporation.Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006
9 Two Categories of Work Access to Effective Practices Ensuring that students (staff) have access to effective practices in an integrated behavior and reading Multi-Tiered System of SupportsPractices are selected based on: Need, Fit, Resource Availability, Evidence, Readiness for Replication, Capacity to ImplementSupport for the PracticesEnsuring that the interventions are implemented correctly with the “right people”, at the “right time”, in the “right amounts” (Implementation Fidelity)
10 Moving from Model Demonstration to Standard Practice Modifying System for Standard PracticeExamine Feasibility/Efficacy of Practice Being DevelopedEmbed w/in existing work: Provide access as current practiceModel Demonstration: Does it Work in a specific setting with a specific population?Replications:Can it be reproduced in different settings with different populations?Scale-Up:Increase “reach” of practice to critical mass of implementationInvest heavily in resources to produce resultsUtilize typical/existing resources
11 Vision for Supporting Scaling MTSS Across Michigan Local capacity to support implementation of MTSS with fidelity will be developed and provided within …47 of 57 Intermediate School Districts (approx. 80%)319 of 798 Local Education Agencies (approx. 40%)1595 or 4192 schools (approx. 38%)Local capacity support means…MTSS CoordinationImplementation TeamCabinet level (executive leadership) supportEffective/efficient data systems for collecting, storing and analyzing data at district levelDistrict trainers (training capacity)Coaching structures (coaching capacity and systems for coaching to occur in an on-going way)
12 Vision for Supporting Scaling MTSS Across Michigan Continued “Scaling up is defined as having enough of something so that it is useful. Scaling up is the process of moving from “exemplars” to the “typical.” The process of scaling involves the development of organizational capacity to move from exemplars sustained by extra-ordinary supports, to typical application with typical supports”.“Scaling up is defined as having enough of something so that it is useful. Scaling up is the process of moving from “exemplars” to the “typical.” The process of scaling involves the development of organizational capacity to move from exemplars sustained by extra-ordinary supports, to typical application with typical supports. While there is no firm agreement about the level at which “scaling” is achieved, we hypothesize that an organization (district) has reached the “tipping point” for functional scaling when approximately 40% of the units in the organization are implementing a practice with fidelity. At that point, the education system would have changed to provide typical supports for evidence-based practices across the (district).” Fixsen et al 2008Fixsen et al 2008
13 MiBLSi add value by providing to districts assistance to address… Michigan School Improvement FrameworkESEA Flexibility WaiverState Board of Education policy on Positive Behavioral Interventions and SupportsMatt’s Safe School Law (bully prevention)Drop Out PreventionState Board of Education Policy on Seclusion and RestraintState Board of Education Resolution to Address School Discipline Issues Impacting Student OutcomesState Board of Education Professional Learning Policy, Standards and GuidanceDisproportionality in race and disciplineMiBLSi collaborates with ISDs to develop local capacity in supporting constituent districts as they work to successfully implement an integrated behavior and reading MTSS model. MiBLSi professional development materials, resources and technical assistance align with the following mandates, priorities and policies of the State of Michigan Department of Education:
14 What makes a program scale-able or scale-worthy? Your TurnYour TurnWhat makes a program scale-able or scale-worthy?
15 MiBLSi Value-Added Work System PoliticalRegulatory/LegalEconomyGeographySociety/CultureEnvironmental FactorsMiBLSi Value-Added Work SystemReceiversStakeholders/FundersDetermine worthManagement/CoordinationCapitalResourcesPeopleMaterialsInformationWork Systems:Providing the MTSS practices and the supports for the these practices to take place successfully within schools and districtsFinancialEvaluationProfessional LearningTechnical AssistanceMiBLSi Value-Added Work SystemInvestments:FundingVisibilityPolitical supportThis is a graphic representation of the MiBLSi value added system. It is important to note that the concept of “value added” is strongly related to the feedback from the stakeholders and the consumers of MiBLSi.Given the current economic environment, it is more important than ever that we wisely invest in practices that demonstrate successful outcomes. Additionally, we need to be good stewards of the resources made available to us. We understand that schools/district can choose to implement MTSS on their own or choose another MTSS program. It is important that we focus on the “value added” through participation with MiBLSi. This value comes from experience in developing MTSS skills and practices that have been implemented across time, locations, and various populations. Added to this is an emphasis on ensuring that structures are in place to successfully support implementation of MTSS practices. If MiBLSi is to improve and scale-up, we need to be cognizant of what brings educators into the program and why they continue on.Returns:Addressing critical issues (Discipline/Ethnicity)Addressing program directives (State Performance Plan)Consumers schools, districts,ISDs/RESAsValued MTSS Products/ ServicesPriority, Plan, Formative Assessment, Continuous Improvement, Risk Management
16 Technical Assistance Regions Provides technical assistance and information dissemination to Intermediate School Districts (ISDs) and Local Education Agencies (LEAs) within a regional structureThis slide displays a map of Michigan with the state sectioned off into various regions for the purpose of providing technical assistance
18 Transformation Zone: Make it easy to change Transformational Process: Developing “Host Environments” for MTSS Implementation with FidelityExisting System of PracticeTransformation Zone: Make it easy to changeDesired System of Practice: Make it easy to sustainThis figure depicts the transformation zone as the place to transition from “what is” (the left hand side) to “what is envisioned” (the right hand side). We call the transformation zone the “zone of discomfort” in recognition of the difficulties inherent in changing the ways of work among teachers, building administrators, district staff, individuals and leaders within the state education system, and parents and stakeholders.Implementation TeamsAnalyze existing system of practiceDevelop PlanCoordinate and monitor Create/modify materialsIdentify barriers to implementationModify plan for continuous improvementAdjust system to address new challenges/pressures to implementation fidelityCoordinate and monitor
19 Statewide Structure of Support Provides guidance, visibility, funding, political support for MiBLSiStudentsBuilding StaffBuilding Leadership TeamLEA District Cabinet and Implementation TeamAcross StateMultiple ISD/LEA TeamsAll staffAll studentsMultiple schools w/in local districtProvides guidance, visibility, funding, political support, and implementation supportsProvides coaching and TA for LEA and/or ISD TeamsProvides guidance and manages implementationProvides effective practices to support studentsImproved behavior and readingISD Cabinet and Implementation TeamRegional Technical AssistanceMichiganDepartment of Education/MiBLSi LeadershipMultiple LEAs w/in intermediate districtWho is supported?Statewide Structure of SupportThis stair step of a cascading system illustrates that there are units within educational systems that build upon each other and are connected through feedback loops and mechanisms of supportHow is support provided?Statewide Structure of Support
20 Feedback Loops Successful Student Outcomes Feedback Loops Develop Organizational StructuresSchools effectively implementing MTSS PracticesSuccessful Student OutcomesInputsOutcomesOutcomesIn the transformation zone, reflective feed back and feed forward loops are embedded in implementation, organization change, and system transformation methods. A key guiding principle for scaling up involves, defining, strengthening, and aligning the interfaces between and among different operating/ system components by increasing the flow of relevant and reflective two-way-information.Develop Staff CompetenciesFeedback LoopsBased on formative and summative measuresCompared with external standards set by stakeholders/consumers and also internal standard set by projectFeedback Loops
21 Leadership Function of District Teams Cabinet Level TeamImplementationPlanning TeamVisionPolitical supportPriorityVisibilityAllocation of resourcesBarrier busting1Collect, summarize, evaluate dataDevelop district planCoordinate and monitor planCreate/modify materials and toolsIdentify barriers to implementationCoordinations between the Cainet and the Implementation Planning Team includes communication around four variable:Need to knowNice to knowDon’t need to knowNeed to act on3Coordination2
22 Implementation Supports District ModelCabinet TeamImplementation TeamVisionPolitical supportPriorityVisibilityAllocation of resourcesBarrier bustingCollect, summarize, & eval dataDevelop planCoordinate/monitor planCreate/modify materialsIdentify barrierLiaisonCoordinationMTSS CoordinatorImplementation SupportsDirectionTrainingCoachingContent ExpertiseMaterialsEvaluationCoordinations between the Cabinet and the Implementation Planning Team includes communication around four variable:Need to knowNice to knowDon’t need to knowNeed to act onSchoolsSchoolsDistrict Model
23 Your TurnYour TurnWhat are critical components necessary to ensure sustainable implementation?
25 Logic of MiBLSiEmphasis is on developing local implementation capacityPartnership between Intermediate School District (ISD) and Local School District focuses on shared ownership and accountability around implementation of an integrated MTSS modelSchools/Districts will be supported through stages of implementation rather than through a training sequenceGoal: Schools/Districts implement MTSS practices with fidelity supported by infrastructure the promotes continuous improvement and sustainability
26 An analogy of implementation stages When you read the album cover, it appears that all track are equal in duration. When you actually look at the record you see that the tracks vary in duration. Additionally, even went moving through the tracks (stages) a scratch may cause you to be stuck in one location for awhile and a bump may send you to an earlier track.
27 Stages of Implementation FocusStageDescriptionExploration/AdoptionDecision regarding commitment to adopting the program/practices and supporting successful implementation.InstallationSet up infrastructure so that successful implementation can take place and be supported. Establish team and data systems, conduct audit, develop plan.Initial ImplementationTry out the practices, work out details, learn and improve before expanding to other contexts.ElaborationExpand the program/practices to other locations, individuals, times- adjust from learning in initial implementation.Continuous Improvement/RegenerationMake it easier, more efficient. Embed within current practices.Should we do it?Work to do it right!Implementation of reading and behavior MTSS is a process that takes place over time (stages of Implementation). Within a school setting, this process may take 2 – 4 years; district-wide, this process will take many more years to achieve. It is important to remember that we should not expect to see change in student outcomes until effective practices have been implemented with fidelity.Work to do it better!
28 Implementation Drivers Successful Student OutcomesImplementation DriversProgram/Initiative(set of practices that are implemented with Fidelity)Staff competency to support students/families with the selected practicesOrganizational capacity to support staff in implementing practices with fidelityCompetencyCapacityThere are three categories of Implementation Drivers: Leadership, Capacity and Competency.When these core components are in place they provide the support to a successful implementation that will be sustained.Competency Drivers are mechanisms that help to develop, improve, and sustain one’s ability to implement an intervention to benefit students. Competency Drivers include: Selection, Training, Coaching, and Performance AssessmentOrganization Drivers are mechanisms to create and sustain hospitable organizational and systems environments for effective educational services. Organization Drivers include: Decision Support Data System, Facilitative Administration, and Systems InterventionLeadershipImplementation DriversAdapted from Fixsen & Blase, 2008Ability to provide direction/vision of process
29 Illustration of Guided Notes for District Meetings
30 Exploration/Adoption Installation Initial Implementation Focus of Work:SepOctNovDecJanFebMarAprMayISD CabinetISD Implementation TeamISD Implementation Team(Phase I & Phase II Conversations)Focus of Work:LEA CabinetLEA Implementation TeamThis slide illustrates a timeline of the focus of the work over the rest of the current school year and into the next.LEA Implementation TeamAdmin TeamsAdmin TeamsSchool TeamsExploration/AdoptionInstallationInitial Implementation
31 Developing ISD Infrastructure Developing the structures to help LEAs develop support structures or to directly support school implementationThis figure illustrates a map of Michigan with dots representing Intermediate School Districts. The intermediate School Districts are clustered together for training purposes as demonstrated by the circles around each cluster of Intermediate School Districts.District Cohort 1District Cohort 2Readiness work for District Cohort 3
32 Focused Planning Sessions Provide for an efficient dissemination of content and technical assistance for developing local implementation infrastructure.These meeting are conducted monthly in several regions throughout the state.The full day sessions involve content delivery and translation to contextual fit.Participants:ISD implementation team membersMiBLSi TAP(s)Other members of from the transformation zone teamTAP takes lead in coordination
33 Developing LEA Infrastructure Accessing implementation support from ISDThis figure illustrates a map of Michigan with dots representing Intermediate School Districts with their constituent local school districts. The intermediate School District along with constituent local school districts are clustered together for training purposes as demonstrated by the circles around each cluster of Intermediate School Districts and constituent local districts.District Cohort 1District Cohort 2Readiness work for District Cohort 3LEA
34 Local Education Agency: Scope and Sequence Topic 6: Evaluating the Effectiveness of MTSS Implementation Supports (District Implementation Team)Pre-Session: Leading District Implementation (District Liaison and MTSS CoordinatorTopic 1: Leading District Implementation (District Implementation Team)Leading District Implementation:(Administrative Team)Topic 2: District Data Review Process (District Implementation Team & Administrative Team)Topic 3: Developing District Capacity for Sustaining MTSS (District Implementation Team)MTSS Common Threads and What Really Works(Administrative Team)Topic 4: Selection Procedures (District Implementation Team & Administrative Team)Topic 5: Supporting Buildings in Stages of MTSS Implementation (District Implementation Team)SessionsTimeline for
35 Training Schools Teams Accessing implementation support from LEA or ISDThis figure illustrates a map of Michigan with dots representing Intermediate School Districts with their constituent local school districts and schools. The intermediate School District along with constituent local school districts and school are clustered together for training purposes as demonstrated by the circles around each cluster of Intermediate School Districts, constituent local districts and schools.District Cohort 1District Cohort 2Readiness work for District Cohort 3LEASchool
36 Training Schools Teams Continued The number of schools to be selected for involvement in your training plan will depend on the constant tension of three factors regarding implementationTraining Schools Teams ContinuedResource AllocationMoving through the stages:How fast you move through these stages will depend on the constant tension of three factors regarding implementation illustrated in the figure. If you want quality implementation, then you may need either more time to engage implementation work or you may need more resources allocated to get the work done.Implementation QualityImplementation Timeframe
37 Your TurnYour TurnWhat are critical components necessary to ensure fidelity of implementation?
39 Supporting Implementation Changes over time Support for your first 50 schools (model demonstrations)Support for your first 500 schools (learning how to scale and support implementation in complex systems)Support for scaling and beyond (becoming standard practice of how we do education)
40 Supporting Infrastructure Use existing structures to scale up the workNo new funding available, need to be creative with existing funding streamRe-allocate staffOrganize around efficiencies to make room for the work
41 Local Context Counts Local Context Counts Emphasize adoption of critical non- negotiable “Big Ideas”Allow for contextual variations in implementation of “Big Ideas”
42 Transformation Zones Transformation Zones Learn from “Transformation Zones” to help in mapping out scaling up planLearn what works from those most willing to implementMore dynamic- changes with need and participation