Quasi-experimental Research: Benchmark Assessment and Improved Student Achievement A Tale of Two Studies: Clayton County Public Schools and West Virginia Linda Bond, Research Scientist CTB/McGraw-Hill

By: Daniel Lewis, Senior Research Scientist CTB/McGraw-Hill Clayton County Public Schools Acuity and Growth on the Georgia Criterion- Referenced Competency Tests in Clayton County: A Quasi-Experimental Study By: Daniel Lewis, Senior Research Scientist CTB/McGraw-Hill

Clayton County Public Schools Quasi-Experimental Study: CCPS Student cohorts. Treatment Length N with matched pretest (Spring 2007 CRCT) and posttest (Spring 2009 CRCT) scores Pre-test Treatment Grades Post-Test 1 2 years 2,720 Grade 1 Math CRCT (Spring 2007) Grades 2 and 3 during the 2007-08 and 2008-09 academic years Grade 3 Math CRCT (Spring 2009) 2 2,739 Grade 2 Math CRCT (Spring 2007) Grades 3 and 4 during the 2007-08 and 2008-09 academic years Grade 4 Math CRCT (Spring 2009) 3 2,807 Grade 3 Math CRCT (Spring 2007) Grades 4 and 5 during the 2007-08 and 2008-09 academic years Grade 5 Math CRCT (Spring 2009) 4 1 year 3,217 Grade 1 Math CRCT (Spring 2008) Grade 2 during the 2008-09 academic year Grade 2 Math CRCT (Spring 2009) 5 3,214 Grade 2 Math CRCT (Spring 2008) Grade 3 during the 2008-09 academic year 6 3,257 Grade 3 Math CRCT (Spring 2008) Grade 4 during the 2008-09 academic year 7 3,232 Grade 4 Math CRCT (Spring 2008) Grade 5 during the 2008-09 academic year These seven cohorts represent students with matched CRCT scores from the pretest to the posttest. Approximately 70 – 80% of the students in CCPS who could have been in the cohort had matched CRCT scores and the difference in CRCT scores between the matched CCPS cohorts and the unmatched CCPS cohorts were from .01 to .07 standard deviation units apart.

Equivalent Control Groups 99% student-by-student matching on: Pretest CRCT score Gender Ethnicity IEP Status ELL Status Title I status School SES status Moderate to high level of internal validity District-wide school implementation, testing & professional development School or classroom effects cannot be completely controlled Low to moderate External validity Results may be generalizable to other settings (student by student matching) Replicable results…

Differences in percent of CCPS cohorts Meeting or Exceeding Standards on Posttest Treatment Group % Meets or Exceeds Standards on Posttest Control Group % Meets or Exceeds Standards on Posttest Difference in % Meets or Exceeds Standards on Posttest: Treatment Group - Control Group 1 69.23 67.32 1.9* 2 64.22 64.55 -0.3 3 74.92 71.49 3.4 4 81.04 82.10 -1.1 5 68.33 70.07 -1.7 6 63.19 58.86 4.3 7 73.70 70.39 3.3 * Bolded values are associated with statistically significant cohort differences identified in Table 5.

Acuity Usage and WESTEST 2 Performance A Quasi-Experimental Study by the WVDE Office of Assessment, Accountability and Research

West Virginia DOE Study – Quasi-Experimental

Summary and Conclusions The research supports the efficacy of the Acuity system in CCPS to accelerate achievement over that of the status quo in the rest of the state. Moderate effect sizes were observed in each case of statistical significance. Ongoing Efficacy research planned Randomized, control studies Learning Point Associates research in Indiana Future research efforts – Hambleton & Sireci