2. Using handheld computers to support the collection and use of reading assessment data Naomi Hupert

3. Why technology to support assessment? Educators need data that is: Specific: identifying where students need help Accessible: available to teachers in a timely manner and without restriction or logistical obstacles Comprehensible: visually clear, aligned to practice, represented in multiple formats designed to be used for different purposes and by different stakeholders

4. Example: DIBELS on a handheld computer Diagnostic Inventory of Basic Early Literacy Skills (DIBELS): early literacy screening assesses students’ early reading skills and identifies those not on track for grade level reading. includes benchmark assessments (usually administered 3 x per year) and a progress monitoring assessment administered based on needs of individual. handheld computer version replicates paper & pencil administration & provides increased guidance and instant calculation of results.

5. What it looks like

7. During the 2004-05 school year an estimated 7,865 schools within 2,447 districts in 49 states and Canada used the DIBELS to assess over 1.7 million students, kindergarten through 3 rd grade.

8. Benefits of the Handheld Computers During interviews, teachers, reading coaches and program coordinators reported the following benefits: –Efficiency of the technology –Immediacy of the feedback loop –Easy access to data for multiple stakeholders –Data that are comprehensible and specific

10. Progress Monitoring The DIBELS progress monitoring consists of a short text that is leveled according to grade and time of year (beginning or end of year). A student reads the passage provided by the teacher and is timed for one minute. The number of words read correctly are recorded during that time.

13. Recommended Goals Good and Kaminski (2003) recommend goals, represented as the number of words read per minute, necessary for students to read to be considered on track to eventually read at grade level. Students in the spring of third grade must meet the minimal goal of reading 110 words per minute to be on track for reading at grade level, and will be considered in need of intensive support if they read below 70 words per minute on a specified grade-level text.

14. What Does Progress Monitoring Look Like? Takes place in a quiet area where students sit for a few minutes to complete the assessment task. Generally a classroom teacher will progress monitor her students one-on-one, while other students work independently at centers, or with a classroom aid. Individual students pulled out for progress monitoring over several days. In some schools an interventionist or reading coach administers progress monitoring. Students read from a leveled text for 1 minute.

15. Student Response Once one minute has passed the teacher will place a marker on the handheld screen at the point in the text where the student stopped reading. The handheld screen will display a graphic indicating the number of words the student read in one minute, represented by an image of a running figure. In nearly all instances of progress monitoring students immediately ask to see “where they are” on the screen after completing the assessment.

17. Is there a Measurable Impact of Progress Monitoring on Student Reading Outcome? Our study conducted multiple analyses using a national sample of 200,000 students Students in the sample received either: –no progress monitoring –minimal progress monitoring –frequent progress monitoring.

18. Our Findings The results of these analyses indicate that frequency of progress monitoring has a positive impact on student outcomes. Where fewer progress monitoring administrations took place, smaller effect sizes were observed. Where larger numbers of progress monitoring administrations took place, greater effect sizes were seen. This was true regardless of a student’s reading level.

19. What is going on? Administration of an assessment, in isolation, is unlikely to change student reading outcomes. Between the administration of the assessment and the student’s change in reading outcome, something occurs that precipitates a change in reading. –What is the “it” that takes place between the assessment and the student outcome? –What takes place during and after progress monitoring? –What kinds of interactions do teachers and their students have during this activity? –What do teachers and their students take away from this activity?

20. What we have learned from observations and interviews During progress monitoring: teacher and student sit quietly and read a connected text, with the teacher paying close attention to the students’ reading; students participate in the assessment process by viewing the outcome of the assessment immediately and noting progress or lack of progress; and teachers have access to clear grade-appropriate goals for students.

21. Continued… Each of these may seem like common events within the course of schooling, yet for many teachers and students the opportunity for these kinds of experiences is rare.

22. Summary Who should be progress monitored? It has been widely accepted that progress monitoring and use of progress monitoring data have important benefits for struggling readers, but these findings suggest there are important benefits for strong readers as well. Learning why and how progress monitoring impacts reading outcomes will provide valuable information for schools working to sustain strong reading growth for all students.

23. Summary, continued How much progress monitoring is enough? How do these findings contribute to the careful balance that teachers must create between time spent on assessment and time spent on instruction. These findings indicate a consistent correlation between frequency of progress monitoring and improved student reading performance, and support the implementation of progress monitoring at least at the recommended levels by DIBELS authors. However, further investigation may support that for optimal outcomes for all students, the minimally recommended frequency of progress monitoring should be raised.

24. Thank you For more information please contact: EDC Center for Children and Technology 96 Morton Street, 7th Floor New York, NY 10014 http://www.cct.edc.org/ Naomi Hupert Nhupert@edc.org 707-829-8532