1. The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director http://www.dpi.wi.gov/cal/t2bgrant.html MSP Regional Conference February 14-15, 2008

2. Mathematics and Science Partnerships Grant Evaluation Model: Quasi-experimental design Sampling Instruments Outcomes Conclusions Future Studies Program Design What is the true impact of the Mathematics and Science Partnerships Grant on student achievement in Wisconsin?

3. Evaluation Model 1.Participants reaction (Surveys, logs, …) 2.Participants Learning (Assessment, tests,..) 3.Organization support for changes (administration ) 4.Participants use of new knowledge (Observation, performance) 5.Student learning outcomes (Assessment, tests, …) All = Gold 2 to 5 = Silver 2 and 5= Copper Anyone = Tin

4. RO XO ROO R is the program Group and R the Comparison Group Time O refers to the testing X is the treatment Vertical alignment of O shows the pre- and post- testing time Design

5. Sampling Program group: 75 middle school mathematics teachers from high-need LEAs. pre-selected Only 69 teachers participated completed the program Comparison group: 75 middle school teachers from similar LEAs. Matched to the program group. Only 71 teachers completed the program Program group: 5 students from each participating teacher randomly assigned. (325) Comparison group: 5 students from each participating teacher randomly assigned. (325)

6. Instruments 1- Teachers The Test of Mathematics Knowledge for Teaching School Mathematics also known as Learning Mathematics for Teaching (LMT) developed by Deborah Ball and Hyman Bass at the University of Michigan as an NSF project, were used to assess the teacher in both the pre- and post-test. 2- Students The Measures of Academic Progress (MAP) developed by the Northwest Evaluation Association (NWEA) in mathematics has been used to assess student achievement. More than 138 school districts in Wisconsin use MAPs in mathematics and reading to monitor the progress of student achievement.

7. Statistical Formulas

8. Program Teachers Comparison StudentsComparison Teachers Program Students

14. Raw Score Program GroupComparison Group TeachersStudentsTeachersStudents PrePostGainPrePostGainPrePostGainPrePostGain mean81810122311880990 SD1.972.791.903.641.813.301.061.221.001.691.461.33 min3126618364-455-2 Max13241519261811 114124 d 6.444.32----------------------- Participants6733567335 Raw score Analysis

15. Conclusions Positive Impact in mathematics, Program teacher gained more content knowledge, The program impact is on teachers with lower content knowledge, Program students achieved higher, The program impact is on students with lower content knowledge, Effective evaluation designs are necessary for impact programs, and Variables such as the socioeconomic group, ethnic group, and disability should be factored in the evaluation design

16. Disclaimer The instructional practices and assessments discussed or shown in these presentations are not intended as an endorsement by the U.S. Department of Education