1. Evaluation Results 2002-2003 Missouri Reading Initiative

2. MRI’s Evaluation Activities: Surveys *Teacher Beliefs and Practices (pre/post) Annual Participant Questionnaire *Data Collection Test Scores Standardized Tests Classroom Assessments (DRA) MAP Demographics Special Education Information MAP Analyses *For schools beginning in 2002

3. MAP ANALYSES: Map analyses compare schools that have finished the MRI program with randomly chosen samples of non-MRI elementary schools Results indicate MRI schools generally outperform non-MRI schools

4. Notes for MAP Analyses Note: With all the following MAP Analyses charts the numbers are not as important as the comparative performance between MRI and non-MRI schools. This is because: 1. There is a lot of variation in the data from year to year and school to school 2. Statewide 2003 MAP scores went down from 2002 3. The calculation of the baseline is slightly different: For the 2002 schools 1999 was the baseline For the 2003 schools an average of 1999/2000 was the baseline

5. Notes for Chart 1 1. This chart compares MRI and non-MRI schools performance on MAP Communication Arts Index 2. Baseline year is 1999; Outcome year is 2002 3. Each sample has 15 schools = number of schools that finished MRI Spring 2002 4. Total random sample = 150 (large enough number to satisfy statistical significance, high confidence levels)

7. Notes for Chart 2 1. This chart compares MRI and non-MRI schools performance on MAP Reading Index 2. Baseline year is 1999; Outcome year is 2002 3. Each sample has 15 schools = number of schools that finished MRI Spring 2002 4. Total random sample = 150 (large enough number to satisfy statistical significance, high confidence levels)

9. Notes for Chart 3 1. This chart compares MRI and non-MRI schools performance on MAP Communication Arts Index 2003 2. Baseline year is an average of 1999/2000 (smoothes out variations); Outcome year is 2003 3. Each sample has 20 schools = number of schools that finished MRI Spring 2003 4. Total random sample 200 (large enough number to satisfy statistical significance, high confidence levels)

11. Notes for Chart 4 1. This chart compares MRI and non-MRI schools performance on MAP Reading Index 2003 2. Baseline year is an average of 1999/2000 (smoothes out variations); Outcome year is 2003 3. Each sample has 20 schools = number of schools that finished MRI Spring 2003 4. Total sample 200 (large enough number to satisfy statistical significance, high confidence levels)

13. Notes for Chart 5 Left Side: For 20 MRI schools, there was an average 19.5% decrease (the desirable direction) in the number of students scoring in the bottom two categories (Step 1 and Progressing) of the 2003 3 rd Grade Communication Arts test as compared to a 2.8% increase (not the desirable direction) for 200 randomly selected Missouri elementary schools. Right Side: For 20 MRI schools, there was an average 32.5% increase (the desirable direction) in the number of students scoring in the top two categories (Proficient and Advanced) of the 2003 3 rd Grade Communication Arts test as compared to a 6.6% increase for 200 randomly selected Missouri elementary schools.

15. Notes for Chart 6 Left Side: 75% of MRI schools had decreases (the desirable direction) in the number of students who scored in the two lowest categories (Step 1 and Progressing) on the 2003 MAP 3 rd Grade Communication Arts test as compared to 48% of the schools in the random sample. Right Side: 75% of MRI schools had increases (the desirable direction) in the number of students who scored in the two lowest categories (Proficient and Advanced) on the 2003 MAP 3 rd Grade Communication Arts test as compared to 44.5% of the schools in the random sample.

17. 2003 MAP: Average Yearly Progress Statewide: 49.7% Met AYP MRI 81% Met AYP (N=60/74) Of the 14 that did not meet AYP:  Six met AYP for All Students  3 - Did not meet AYP for FRL  1 - Did not meet AYP for IEP  2 - <95% of eligible group (30+)

18. Participant Survey Participants rate the usefulness of component utilization, practice change, "buy in", attitudes toward the program and trainer, etc. Results drive program change; e.g., Orientation

19. Notes for Participant Survey Slide This slide introduces the survey and its uses. The example is the Table in the next slide which demonstrates how the survey is often used. In this case: 1. 2002 survey respondents identified the problem of being “overwhelmed” 2. Program responded by redesigning orientation and other details 3. Program satisfaction for 1 st year schools improved from 02 to 03 surveys

20. Notes for School Rating Slide The year (e.g., 01-02; 02-03) indicates the year of the survey The number (e.g., 1 st, 2 nd, 3 rd ) indicates the MRI year for those schools This slide provides evidence for two statements: 1. There is a higher degree of satisfaction expressed by participants the longer they are in the program. For example, 68% (23/34) of schools who began in 01-02 rated the program better than 4.0 as in their 2 nd year as opposed compared to 47% (16/34) in their 1 st year. At the same time only 9% (3/34) of those schools rated the program below 3.5 in their 2 nd year as opposed to 29% (10/34) of the same schools in their 1 st year. 2. There has been an increase in satisfaction each year by schools at the same point of the program. For example 72% (13/18) of 1 st year schools in 02-03 rated the program better than 4.0 as compared to 47% (16/34) of 1 st year schools in 01/02. (see previous slide notes)

21. School Ratings: 01-03 Reflecting on the effectiveness of the MRI program as a whole, how would you rate it? Category 1 st 01-02 1 st 02-03 2 nd 01-02 2 nd 02-03 3 rd 02-03 2.0-2.4910000 2.5-2.9941310 3.0-3.4952021 3.5-3.9982684 4.0-4.491297158 4.5-5.044789 TOTALS3418233422

22. Special Education We track the effects of MRI on Special education in two ways: 1.Beginning with schools that started MRI in the Fall of 2002, all students with IEPs are identified and the type of IEP is described (Reading, Math, Speech, etc.) 2.Annual reports are made from every school about their IEP evaluation process (see following slide)

23. Notes for Special Education Table This Table is for schools that were in their 3 rd year 2001-2002 Many schools do not have this data, or it is not easily accessed so many schools did not report. 9 of 18 in 2002 5 0f 23 in 2003 The data we do have all points to a decrease in referrals, evaluations, and assignment of IEP over the time schools participate in MRI. We are collecting this data from the onset of schools that began in 2002-2003 so that we will have a complete report in 2005.

24. Special Education Results

25. DRA GRADE LEVEL (GL) KEY ( GL) 1st Grade Fall 4 1st Grade Spring 18 2nd Grade Fall 18 2nd Grade Spring 28 3rd Grade Fall 28 3rd Grade Spring 38

26. FALL SPRING FALL SPRING FALL SPRING