1. MSRP Year 1 (Preliminary) Impact Research for Better Schools RMC Corporation

2. Overview: Study Design and Analytic Approach Study Design READ 180: Evaluate student outcomes using an experimental design based on randomly assigning eligible students to treatment and control conditions within participating schools Student outcomes include reading achievement (ITBS) and state assessment (TCAP) results in core content areas Study Design MCLA: Evaluate teacher and student outcomes using an experimental design based on randomly assigning schools within matched pairs to treatment and control conditions Teacher outcomes include preparedness and frequency of literacy strategy use Analytic Approach READ 180: Cross-sectional ITT analyses of reading and core content area achievement Two-level models using spring ITBS and TCAP scores as a function of student and school variables Analytic Approach MCLA: Two-level models using spring indices as a function of teacher and school variables Two-level models using spring ITBS and TCAP scores as a function of student and school variables

3. MCLA: Random Assignment of Schools

4. Demographic Characteristics of Year 1 MCLA Student Sample

5. Baseline Comparisons of Students in MCLA Treatment and Control Schools

6. READ 180: Enrolled Students

7. Statistical Model Used to Estimate READ 180 Impacts At the student level,

8. Statistical Model Used to Estimate READ 180 Impacts

9. Variables Included in READ 180 Impact Analytic Models (Year One): Dependent and Independent

10. Variables Included in READ 180 Impact Analytic Models (Year One): Covariates

11. Analytic Decisions Missing Data –students missing pretest score(s) deleted from impact analysis on relevant measure(s) Covariates –include all student- and school-level covariates in the model –run the model –eliminate the school covariate with the lowest significance level (highest p-value) not less than 0.2 – repeat steps 2 and 3 until the remaining covariates had p-values less than 0.2 – repeat steps 2-4 for the student covariates

12. READ 180 Impacts on Students (Year One)

13. Selected Characteristics of the Year 1 Teacher Sample for MCLA Impact Analyses

14. Statistical Model Used to Estimate MCLA Teacher Impacts

16. All Variables Included in MCLA Impact Analytical Models for Year 1

17. Analytic Decisions Missing Data –teachers missing pretest score deleted from impact analysis on measure Covariates –include all teacher and school-level covariates in the model –run the model –eliminate the school covariate with the lowest significance level (highest p-value) not less than 0.2 – repeat steps 2 and 3 until the remaining covariates had p-values less than 0.2 – repeat steps 2-4 for the teacher covariates

18. Comparison of Teachers in MCLA Treatment and Control Schools on Year-End Indices for Preparedness and Frequency of Use

19. Regression Models Employed to Test Year 1 MCLA Impact on Student Achievement

21. Analytic Decisions Missing Data –students missing pretest score(s) deleted from impact analysis on relevant measure(s) Covariates –include all student- and school-level covariates in the model –run the model –eliminate the school covariate with the lowest significance level (highest p-value) not less than 0.2 – repeat steps 2 and 3 until the remaining covariates had p-values less than 0.2 – repeat steps 2-4 for the student covariates

22. MCLA Impacts on Students (Year One)

23. (Brief) Conclusions & Discussion READ 180: No significant Year One student impact Late startup (Most) students will receive two years of intervention Planned Future Analyses: Three-level analyses planned to examine whether teacher characteristics exert a moderating effect on student outcomes Exploratory analyses of relationships between amount of READ 180 instruction and effects on student outcomes

24. (Brief) Conclusions & Discussion MCLA: Significant (moderate) impact on teachers’ frequency and preparedness to use MCLA strategies No significant impact on students’ achievement in reading or core content areas Discuss: –Subjectivity of measure (“Hawthorne Effect”) –Teacher findings support program logic model –Explore relationship between impact and participation in PD