®® The Impact of Professional Development Models and Strategies on Teacher Practice and Student Achievement in Early Reading IES Research Conference Michael S. Garet June 8, 2009 IES Research Conference Michael S. Garet June 8, 2009

®® 2 PD Impact Study The 5-year study was supported by IES The report is available on the IES website

®® 3 Principal Staff Michael S. Garet (project director) AIR Fred Doolittle (co project director) MDRC Stephanie Cronen (deputy project director) AIR Meredith Ludwig, AIR Terry Salinger, AIR Marian Eaton, AIR Anja Kurki, AIR Howard Bloom, MDRC Rob Ivry, MDRC

®® 4 Partner Organizations American Institutes for Research (AIR) MDRC REDA International, Inc. Sopris West CORE

®® Introduction Test of two intensive, content-focused PD interventions that represent “best practices” to improve 2 nd grade reading achievement Examines impact on ultimate outcome (student achievement) and intermediate outcomes (teacher knowledge and classroom instruction)

®® PD Models Tested Treatment A: A content-focused PD series consisting of eight institute and follow-up seminar days (48 hrs). Based on Language Essentials for Teachers of Reading and Spelling (LETRS) developed by Louisa Moats Treatment B: The eight-day institute and seminar series plus coaching throughout the year (~108 hrs). Coach training conducted by the Consortium on Reading Excellence (CORE) “Business as Usual:” The control condition

®® Research Questions What effect does the institute and seminar series have on 2 nd grade teachers’ knowledge, practices, and students’ reading achievement?  (A vs. C) What effect does the institute and seminar series plus in-school coaching have on these same outcomes?  (B vs. C) What is the added effect of in-school coaching above and beyond the institute and seminar series alone?  (B vs. A)

®® Design School-level random assignment  90 schools located in 6 districts using one of two popular reading programs consistent with the National Reading Panel (NRP)  270 teachers  5,500 students (77% eligible for free or reduced priced lunch; 16% White; 78% African American; 6% Asian, Hispanic, or Other  MDES: 0.20 for student-level outcomes, 0.40 for teacher-level outcomes Timing  Year 1: Implementation of the PD and data collection  Year 2: Follow-up data collection only

®® Data Sources for Outcomes Teacher Knowledge: Reading Content and Practices Survey (RCPS)  Total, word-level, and meaning-level scales Teacher Practices: Classroom Observations  Explicit instruction, independent student activity, and differentiated instruction scales Student Achievement: District Records (Standardized Assessments)

®® Implementation of the PD According to sign-in sheets, teachers attended 35 of the 45 hours of institutes and seminars According to coach logs, the coaching included the planned topics and time (62 hours) According to teacher surveys, treatment A and B teachers attended significantly more institute and seminar hours than control teachers (39 and 47 vs 13 hours), and treatment B teachers received significantly more coaching (71 hours vs 6 hours)

®® Impacts on Total Knowledge by Year **

®® Impacts on Explicit Instruction by Year * *

®® Impacts on Student Activity by Year

®® Impacts on Diff Instruction by Year

®® Impacts on Student Achievement by Year

®® Summary of Study Results Although there were positive impacts on teachers’ knowledge and on one of the three instructional practices promoted by the PD, neither PD intervention resulted in significantly higher student test scores. The added effect of the coaching intervention was not statistically significant. There were no statistically significant impacts on measured teacher or student outcomes in the year following the treatment.

®® Exploratory Analyses We conducted exploratory analyses to examine questions raised by the main impact results:  What might explain why the impacts on teacher knowledge and practice did not translate into impacts on student achievement?  Why were the impacts on teacher outcomes found during the implementation year no longer significant at follow-up?  What might explain why the PD affected teachers’ word- but not meaning-level knowledge?  Why didn’t the coaching plus institutes produce greater impacts relative to the institutes alone?

®® 18 Report The Impact of Two Professional Development Interventions on Early Reading Instruction and Achievement (NCEE ) NCEE NCEE