1. Recovering from Reading Recovery: A long-term program evaluation JoAnna Cogan-Ferchalk, Ed.S., NCSP Matthew Ferchalk, Ed.S., NCSP

2. Background & Philosophy Developed in New Zealand, by Dr. Marie Clay in the 1970's The goal of RR is to close the gap between at-risk readers and their average peers (Schwartz, 2005) Incorporates Vygotskys theory of the Zone of Proximal Development, where the student is able to read at a higher rate with help from the teacher than would be possible alone (Clay & Cazden, 1990) Whole language background-- phonics is not taught explicitly to children It is taught incidentally if an opportunity arises (Schwartz, 2004) Details of print and letter/sound relationships should be gained through writing, not reading instruction. This knowledge is considered limited when compared to comprehension of the text (Clay, 1993; Grossen et al., 1997)

3. (CREC, 2004; see also Saginaw Public Schools, 1992) How it works Students are selected for RR by teacher ratings and by screening with the Observational Survey (the lowest students are not always enrolled) Students receive daily 30 minute lessons during their first grade year Instruction includes use of magnetic letters to help write words, and teaching to read for meaning Most students graduate from RR after 12-20 weeks ELCOs RR teachers have adopted a more structured phonics element

4. School District Demographics Approximately 2400 students 16% special education placement rate Rural, farming community in Pennsylvania Reading Recovery has been in place for 11 years

5. Literature on RR Several studies have found RR to be moderately effective (Marina & Gilman, 2003; Pinnell, Lyons, DeFord, Byrk, & Seltzer, 1994; Schwartz, 2005) There are complaints that these studies are flawed (Center & Wheldall, 1992; Heibert, 1994; Hoff, 2002; Rasinski, 1995) Unequal treatment time Not using the lowest readers Other studies have found that students do not attain the same reading proficiency as their peers (Huggins, 1999), which has been attributed to lack of explicit phonics instruction (Pressley, 1998) When explicit instruction in phonics is added to RR researchers have found greater improvement than with traditional RR (Chapman, Tunmer, & Prochnow, 2001; Iverson & Tunmer, 1993) Few studies have examined long term effects of RR. Those that do typically high rates of special education placement in later years (Ramaswami, 1994)

6. Research Questions How do students who have been through the RR program score on the PSSAs and curriculum- based assessments (CBA) in reading? If students demonstrate improvement in reading scores after RR, do these gains persist over time? Do students who have been through RR make the same rate of reading progress as their peers who did not need supplemental instruction? What is the prevalence of placement into programs such as Title I and Special Education after students have graduated or were discontinued from RR?

7. Method Review existing data for students who were enrolled in RR and are currently in 5 th, 6 th, and 7 th grades 8 students per elementary building per year (n= 96 students) 3 rd and 5 th grade PSSA reading scores CBA reading scores from 1 st to 5 th grade Special programs placement rate following RR Review existing data from a randomly selected comparison group (n=96)

8. 5 th Grade PSSA data, Cohort 1 Reading Recovery Below Basic24% Basic38% Proficient19% Advanced19% Total Proficient 38% District wide data Below Basic14% Basic22% Proficient30% Advanced34% Total Proficient 64%

9. 3 rd Grade PSSA Data, Cohort 2 Reading Recovery 1 st quartile28% 2 nd quartile22% 3 rd quartile28% 4 th quartile22% Upper quartiles 50% District wide data 1 st quartile14% 2 nd quartile20% 3 rd quartile34.5% 4 th quartile31.5% Upper quartiles 66%

10. 5 th Grade PSSA Data, Cohort 2 Reading Recovery Below Basic27% Basic14% Proficient50% Advanced9% Total Proficient 59% District wide data Below Basic18% Basic12% Proficient47% Advanced23% Total Proficient 70%

11. Across time PSSA, Cohort 2 PSSA scores from 3 rd to 5 th grade declined from 1298 to 1238 The difference was not significant at 0.05

12. 3 rd Grade PSSA Data, Cohort 3 Reading Recovery 1 st quartile35% 2 nd quartile30% 3 rd quartile22% 4 th quartile13% Upper quartiles 35% District wide data 1 st quartile17% 2 nd quartile22% 3 rd quartile31% 4 th quartile30% Upper quartiles 61%

13. CBA Cohort 1 Short-term, 2 nd grade, 75% of the comparison group were at mastery or instructional levels Compared to 9% of RR students Long-term, 5 th grade 90% of the comparison group were at mastery or instructional 81% mastery Compared to 80% of RR students 20% mastery

15. CBA Cohort 2 Short-term, 2 nd grade 82% of the comparison group were at mastery or instructional levels Compared to 32% of RR students Long-term, 5 th grade 95% of the comparison group were at mastery or instructional levels 42% mastery Compared to 82% of RR students 29% mastery

17. CBA Cohort 3 Short-term, 2 nd grade 43% of the comparison group were at mastery or instructional levels None of the RR students had reached an instructional level Long-term, 4 th grade 90% of the comparison group were at mastery or instructional levels 71% mastery Compared to 67% of RR students 25% mastery

19. Special programs placement rates Special Education Cohort 1 19% Cohort 2 22.7% Cohort 3 12% District wide rate is approximately 16% Title I Cohort 1 1 year- 100% 2 years- 33.3% 3 years- 14.3% Cohort 2 1 year- 100% 2 years- 31.8% 3 years- 27.3% Cohort 3 1 year- 100% 2 years- 40% 3 years- no data

20. How do students who have been through the RR program in the ELCO school district score on standardized tests of achievement and curriculum-based assessments (CBA) in reading? On the PSSA exams, fewer students in RR scored in the proficient range as compared to district wide data This held true across all cohorts On CBA data, fewer students in RR scored at the instructional and mastery levels than did students not in RR This held true across all cohorts

21. Do students who have been through RR make the same rate of reading progress as their peers who did not need supplemental instruction? When examining the CBA data it appears that RR students started out well below their peers, and remained below them over the course of elementary school RRs goal is to close this gap in achievement Over time more RR students scored in the instructional and mastery levels on the CBA measures; however, they did not approach the level of mastery attained by their peers

22. If students demonstrate improvement in reading scores after RR, do these gains persist over time? In general, students did not demonstrate initial gains in reading (as demonstrated by CBA) By 5 th grade, students in RR were still behind their peers (as demonstrated by CBA & PSSA scores) A jump was noted in fourth grade CBA scores; however this cannot be attributed to RR as it occurred three years after the intervention. This may also be the result of CBA scores no longer being kept for students in special education.

23. What is the prevalence of placement into programs such as Title I and Special Education after students have graduated or were discontinued from RR? In cohorts 1 & 2, students were placed into special education at a higher rate than the district average (19 & 22.7% vs. 16%) In cohort 3, the rate was lower (12% vs. 16%) 100% of students in RR received Title I services in 1 st grade 40-60% of students in RR received Title I services in 2 nd and 3 rd grade

24. Conclusions & Implications The goal of RR (bringing at-risk students to the level of their peers in a short period of time) does not appear to have been met Given that RR is not designed for the lowest readers, it excludes students who are in need of help, and may lead to misleading reports of effectiveness The practice of dropping students if they do not make adequate progress leaves some students without much needed intervention School psychologists are in a unique position to recommend programs that have a broader research base and can provide more benefits to more students (i.e.- PHONICS!!!) We can also take an active role as a researcher and/or program evaluator to keep our districts informed of beneficial programs

25. Limitations Quality of data kept in the school No pre RR reading levels of students available Different data sets exists for each cohort Too many confounding variables to accurately determine the effects of RR 100% of students in RR received Title I for at least one year, and many more received years of support Attrition When students in RR were placed in special education, their CBA scores were not recorded

26. Recommendations for future research More long-term research is needed to examine the effects or RR Research should include initial reading levels of at-risk students

27. References Capital Region Education Council. (2004). Who we are and what we do. Retrieved online September 26, from http://www.crec.org/rr/index.shtml#what.http://www.crec.org/rr/index.shtml#what Center, Y., & Wheldall, K. (1992). Evaluating the effectiveness of Reading Recovery: A critique. Educational Psychology, 12(3/4) Clay, M. M. (1985). Observation survey of early literacy achievement. Auckland, New Zealand: Heinemann. Clay, M. M. (1993). Reading Recovery: A guideline for teachers in training. Portsmouth, NH: Heinemann. Clay, M. M. (1998). An observation survey of early literacy achievement. Auckland, New Zealand: Heinemann. Clay, M. M. & Cazden, C. B. (1990). A Vygotskyan interpretation of Reading Recovery. In L. Moll (Ed.), Vygotsky and education: Instructional implications and applications of sociohistorical psychology (pp. 206-222). Cambridge, England: Cambridge University Press. Chapman, J. W., Tunmer, W. E., & Prochnow, J. E. (2001). Does success in the Reading Recovery program depend on developing proficiency in phonological processing skills? A longitudinal study in a Whole Language instructional context. Scientific Studies of Reading, 5(2), 141-177.

28. Dyer, P. C. (1992). Reading Recovery: A cost-effectiveness and educational outcomes analysis. Spectrum: Journal of Research in Education, 10(1), 110-119. Elbaum, B., Vaughn, S., Hughes, M. T. & Moody, S. W. (2000). How effective are one-to-one tutoring programs in reading for elementary students at risk for reading failure? A meta-analysis of the intervention research. Journal of Educational Psychology, 92(4), 605-619. Evidence-based research on Reading Recovery. (2002). Report to House and Senate Education Committees. Retrieved July 1, 2002, from, http://www.nrrf.org/rrletter_5- 02.pdf.http://www.nrrf.org/rrletter_5- 02.pdf Glynn, T., & Crooks, T. (1992). Reading recovery in context: Implementation and outcome. Educational Psychology, 12(3/4), 249-262. Grossen, B., Coulter, G., & Ruggles, B. (1997). Reading Recovery: An evaluation of benefits and costs. Retrieved on July 1, 2002, from, http://darkwing.uoregon.edu/~bgrossen/rr.htm. http://darkwing.uoregon.edu/~bgrossen/rr.htm Hoff, D. J. (2002). Researchers urge officials to reject Reading Recovery. Education Week, 21(39), 1-3. Huggins, R. (1999). Longitudinal study of the Reading Recovery program: 1994-1998. Urban Education. (ERIC Document Reproduction Service No. 430 067). Iversen, S. & Tunmer, W. E. (1993). Phonological processing skills and the Reading Recovery program. Journal of Educational Psychology, 85, 112-126. Marina, B., & Gilman, D. A. (2003). Is Reading Recovery worth the cost? (ERIC Document Reproduction Service No. ED 473 957).

29. Moore, P., Lysy, D. C., Bamford, R., & Cook, L. A. (1999). A look at Reading Recovery programs in Maine schools: A supplement to the State of Maine Reading Recovery report and evaluation, 1997-1998. Maine University: Center for Early Literacy. (ERIC Document Reproduction Service No. ED 463 533). Munoz, M.A. (1999) Reading Recovery and the at-risk first grade students. (ERIC Document Reproduction Service No. ED 471 186). Reading Recovery Executive Summary, 1984 to 1998. (1999). Columbus, OH: Reading Recovery Council of North America. (ERIC Document Reproduction Service No. ED 434 318). Report of the National Reading Panel: Teaching Children to Read. (April 2000). National Institute of Child Health and Human Development. NIH Pub. No. 00-4769. Retrieved July 1, 2002, from, http://www.nichd.nih.gov/publications/nrp//findings.htm.http://www.nichd.nih.gov/publications/nrp//findings.htm Pinnell, G. S., Lyons, C. A., DeFord, D. E., Bryk, A. S., & Seltzer, M. (1994). Comparing instructional models for the literacy education of high-risk first graders. Reading Research Quarterly, 29, 9-39. Pressley, M. (1998). Reading instruction that works: The case for balanced teaching. New York: Guilford. Ramaswami, S. (1994). The differential impact of Reading Recovery on achievement of first graders in the Newark School District. Newark, NJ: Newark Board of Education, Office of Planning, Evaluation and Testing. (ERIC Document Reproduction Service No. ED 374 180). Rasinski, T. V. (1995). Commentary on the effects of Reading Recovery: A response to Pinnell, Lyons, DeFord, Bryk, and Seltzer. Reading Research Quarterly, 30, 264- 270.

30. Saginaw Public Schools, Department of Evaluation Services. (1992). Compensatory education product evaluation: Reading Recovery Program 1991-92. Saginaw: Author. (ERIC Document Reproduction Service No. ED 350 587). Shanahan, T., & Barr, R. (1995). Reading Recovery: An independent evaluation of the effects of an early instructional intervention for at-risk learners. Reading Research Quarterly, 30, 958-995. Schawrtz, R. M. (2005). Literacy learning of at-risk first-grade students in the Reading Recovery early intervention. Journal of Educational Psychology, 97(2), 257-267. Tunmer, W. E., & Chapman, J. W., (2001). The Reading Recovery approach to preventative early intervention: As good as it gets? Palmerston North, New Zealand: Massey University.