1. 0 ©2015 U.S. Education Delivery Institute What makes implementation difficult? TodaysMeet.com/delivery

2. 1 ©2015 U.S. Education Delivery Institute How do we get from red to green? The Assessment Framework Likelihood of Success Red Amber/Red Amber/Green Green Highly problematic – requires urgent and decisive action Problematic – requires substantial attention, some aspects need urgent attention Mixed – aspect(s) require substantial attention, some good Good – requires refinement and systematic implementation Key Quality of planning Capacity to drive progress Evidence of progress JudgementRatingRationale Summary

3. 2 ©2015 U.S. Education Delivery Institute One answer comes from a surprising place

4. 3 ©2015 U.S. Education Delivery Institute “Deliverology” is a flexible methodology that can help you strengthen implementation across the board 1 What are we trying to do? 2 How are we planning to do it? 3 At any given moment, how will we know whether we are on track to succeed? 4 If we’re not on track, what are we going to do about it?

5. 4 ©2015 U.S. Education Delivery Institute Why might Deliverology help? It’s proven It’s practical It’s accessible

6. 5 ©2015 U.S. Education Delivery Institute The method was first invented and proven in the UK “…a mandate for reform… and an instruction to deliver ”

7. 6 ©2015 U.S. Education Delivery Institute Tony Blair and his government achieved some remarkable results Percent of targets on track during Tony Blair’s second term

8. 7 ©2015 U.S. Education Delivery Institute In the last five years, these proof points have spread to American education K-12 partners Higher education partners Both

9. 8 ©2015 U.S. Education Delivery Institute Kentucky undertook the approach in 2010… Screenshot of KDE Delivery Website

10. 9 ©2015 U.S. Education Delivery Institute …and has since seen large gains in college and career readiness… CCR trajectory with targets and baselines 38.0% 47.2% 54.0% 62.2%

11. 10 ©2015 U.S. Education Delivery Institute Hawai’i adopted the approach in 2012 Hawai’i’s three goals and six strategies Academic Review Teams Common Core Comprehensive Student Support System Formative Instruction/Data Teams Educator Effectiveness Induction and Mentoring

12. 11 ©2015 U.S. Education Delivery Institute The state has gone from high risk status in Race to the Top to a success story…

13. 12 ©2015 U.S. Education Delivery Institute …and has achieved encouraging results for students (1 of 2) Source: Hawai’i P20 Partnerships for Education

14. 13 ©2015 U.S. Education Delivery Institute …and has achieved encouraging results for students (2 of 2) Source: Hawai’i P20 Partnerships for Education

15. 14 ©2015 U.S. Education Delivery Institute Why might Deliverology help? It’s proven It’s practical It’s accessible

16. 15 ©2015 U.S. Education Delivery Institute “Deliverology” consists of 15 essential elements Plan for delivery Develop a foundation for delivery Understand the delivery challenge A.Evaluate past and present performance B.Understand drivers of performance and relevant activities A.Determine your reform strategy B.Set targets and establish trajectories C.Produce delivery plans A.Establish routines to drive and monitor performance B.Solve problems early and rigorously C.Sustain and continually build momentum Drive delivery A.Define your aspiration B.Review the current state of delivery C.Build the delivery unit D.Establish a “guiding coalition” 2341 Create an irreversible delivery culture 5 A. Build system capacity all the time B. Communicate the delivery message C. Unleash the “alchemy of relationships”

17. 16 ©2015 U.S. Education Delivery Institute The approach includes a number of tools that you can apply to your work The Assessment Framework Likelihood of Success Red Amber/Red Amber/Green Green Highly problematic – requires urgent and decisive action Problematic – requires substantial attention, some aspects need urgent attention Mixed – aspect(s) require substantial attention, some good Good – requires refinement and systematic implementation Key Quality of planning Capacity to drive progress Evidence of progress JudgementRatingRationale Summary

18. 17 ©2015 U.S. Education Delivery Institute When planning goes wrong, why does it go wrong? TodaysMeet.com/delivery

19. 18 ©2015 U.S. Education Delivery Institute A good plan is like a good map “The more accurate the map, the more it resembles the territory. The most accurate map would be the territory, and thus would be perfectly accurate and perfectly useless” -Neil Gaiman

20. 19 ©2015 U.S. Education Delivery Institute But it’s not enough for the plan to be in your head

21. 20 ©2015 U.S. Education Delivery Institute 1.Prioritized Reform Strategies 2. Targets and Trajectories Delivery Plans 3. Delivery Chains The delivery methodology includes tools that help you avoid these challenges

22. 21 ©2015 U.S. Education Delivery Institute These tools help states get clear about what they are doing and how Sample tool: Strategy Profile R OW W HAT IS IT ? DescriptionDescribe the strategy and its purpose in a sentence or two Link to goal(s)On which goals will the strategy have a significant impact? Rationale / Theory of ActionWhy/how do we believe it will have that impact? ScaleHow many students, teachers, campuses, etc. will it affect? Resources requiredWhat people, time, money and technology will be needed for successful implementation? Definition of success / Desired outcome What would success look like for this specific strategy and by when? Estimate of ImpactScale x Efficacy = Impact. Of those affected, how many additional will achieve the outcome we desire? MilestonesWhat are the key events (5 or fewer) in the rollout of this strategy?

23. 22 ©2015 U.S. Education Delivery Institute The approach includes a number of tools that you can apply to your work The Assessment Framework Likelihood of Success Red Amber/Red Amber/Green Green Highly problematic – requires urgent and decisive action Problematic – requires substantial attention, some aspects need urgent attention Mixed – aspect(s) require substantial attention, some good Good – requires refinement and systematic implementation Key Quality of planning Capacity to drive progress Evidence of progress JudgementRatingRationale Summary

24. 23 ©2015 U.S. Education Delivery Institute What is capacity? TodaysMeet.com/delivery

25. 24 ©2015 U.S. Education Delivery Institute Capacity is a tricky term to define

26. 25 ©2015 U.S. Education Delivery Institute A simple (if not universal) definition can help Capacity is a combination of the skill and will of individuals or teams to make the change you need them to make.

27. 26 ©2015 U.S. Education Delivery Institute A delivery chain helps you to get clear about roles, responsibilities, and the capacity needed to reach the field at scale StateRegionalDistrictSchoolClassroom This chain shows the mechanisms for implementation for the CCSS in one state. SEA SEA Curriculum Vendors Regional Network Teams and Curriculum Staff CCSS Assessment Vendors Website Curriculum modules, sample items, other materials Other Curriculum Vendors All other Districts Sups and curriculum staff Large and Districts and Charters with NTEs RICS Students Teachers Coaches and Teacher Leaders Principals and APs Manage Train Provide materials Consult CCSS Instruction State assessments and results Non-state professional development

28. 27 ©2015 U.S. Education Delivery Institute It helps states to develop their theories of action about how education workforce development will actually occur County Delivery Chain for Observer Norming District SchoolClassroom Students Talent Mgmt. (12) Coaches (36) State Students Teachers ILDs (12) Principals (~200) Observers (100+) Train Coach Train: shared language Improve performance All able to observe and come to consistent conclusions; shared language ; more effective instruction Train: shared language Provide supports Receive strong, consistent feedback about how to improve  Improve instruction Train: shared language ▪ Potentials Risks/Weaknesses: – Are coaches’ norming sessions consistent? – Are principals getting required training on language and norming? – Is training/support too compliance oriented and not focused enough on how to coach people to improve?

29. 28 ©2015 U.S. Education Delivery Institute The approach includes a number of tools that you can apply to your work The Assessment Framework Likelihood of Success Red Amber/Red Amber/Green Green Highly problematic – requires urgent and decisive action Problematic – requires substantial attention, some aspects need urgent attention Mixed – aspect(s) require substantial attention, some good Good – requires refinement and systematic implementation Key Quality of planning Capacity to drive progress Evidence of progress JudgementRatingRationale Summary

30. 29 ©2015 U.S. Education Delivery Institute Why is it so hard to measure progress in our work? TodaysMeet.com/delivery

31. 30 ©2015 U.S. Education Delivery Institute The big dilemma is the disconnect between how often outcome data are available and how often we have to make decisions

32. 31 ©2015 U.S. Education Delivery Institute This results in some awkward situations What do the 2013 NAEP results say now?

33. 32 ©2015 U.S. Education Delivery Institute It helps to expand our definition of “evidence”… Already available Availability Created by you Form Quantitative StatisticsNew Collections Field workArtifacts Qualitative

34. 33 ©2015 U.S. Education Delivery Institute …and to think of a spectrum from inputs to outcomes Student learning outcomes: Did students learn as a result? Field use of new knowledge and skills: Are they changing practice? Organizational support and change: Are they set up for success? Field learning: Did they learn what is expected of them? Field reaction: Did they like it? Milestones completed: What did we do? These questions can be asked of stakeholders at each level of the State’s education system: teachers, school leaders, district leaders and regional leaders. Framework for Understanding Implementation Progress

35. 34 ©2015 U.S. Education Delivery Institute Kentucky has used these tools to creatively track progress on TLE implementation This map and tree map show Kentucky check list items by district Total Items 0 1 2 3 Jefferson County

36. 35 ©2015 U.S. Education Delivery Institute Other states have developed educator surveys to gather evidence of implementation progress Principal Survey Question During your most recent 5-10 visits to English/Language Arts classrooms, how often did you observe the following practices? ConsistentlySometimesRarelyNever I don’t know Not applicable The use of text-dependent questions Students writing about texts using personal experiences The use of texts that were the appropriate level of complexity The use of pre-reading strategies to help students fully understand a text prior to reading it Students developing arguments based on the texts they’ve read The use of academic vocabulary during instruction Teacher Survey Question Please indicate your level of agreement with the following statements regarding the feedback you received after receiving a classroom observation. a)The feedback was timely b)The feedback included concrete actions that I can take within my classroom c)The feedback included suggestions to implement the Common Core better d)The feedback related to the information drawn from student data based on common interim assessments or student work e)The feedback directed me to resources that will help me improve my practice f)The feedback included suggestions to help me improve my instructional practice SAMPLE SURVEY QUESTIONS

37. 36 ©2015 U.S. Education Delivery Institute Once you have the evidence, you also have to decide what to do with it Overall Likelihood of Success AttractPrepareSupportRetain and Extend State 12431 State 23334 State 34232 State 43334 State 53133 State 63133 State 73333 State 83311 State 94133 State 103231 State 113424 State 122333 State 133333 State 143344 State 152313 State 163133 State 173332 State 183334 State 194333 State 204331 State 213323 State 223334 State 233331 State 243233 State 254312 State 262333 State 273131 State 283333 State 293343 State 301223 State 311244 State 323133

38. 37 ©2015 U.S. Education Delivery Institute Why might Deliverology help? It’s proven It’s practical It’s accessible

39. 38 ©2015 U.S. Education Delivery Institute If this overview interested you, you’ll have an opportunity to dive deeper on some of the components during the summit The Assessment Framework Likelihood of Success Red Amber/Red Amber/Green Green Highly problematic – requires urgent and decisive action Problematic – requires substantial attention, some aspects need urgent attention Mixed – aspect(s) require substantial attention, some good Good – requires refinement and systematic implementation Key Quality of planning Capacity to drive progress Evidence of progress JudgementRatingRationale Summary Tuesday Breakout: “Implementation: Elements of Delivery Planning” Wednesday Breakout: “Managing the Implementation of Initiatives”

40. Thank You @EdDelivery www.deliveryinstitute.org nrodriguez@deliveryinstitute.org