1. 1 RTI: Grade-Level Data Meetings for Elementary Literacy Data Analysis Oakland Schools Michele Farah, Joan Firestone, Diane Katakowski, and Susan Koceski

2. Pre-requisite skills for successful Grade-Level Data Meetings 2  Understanding and commitment to the problem-solving process  Understanding of the purposes of assessment and a variety of assessment tools  Organization and preparation  Facilitation skills including understanding group dynamics, norm, and how to deal with “challenging behavior”  Ability to display and analyze multiple sources of data  Ability to examine current curriculum and instruction to determine effectiveness Oct Nov

3. Grade-Level Data Meetings: Winter 3 Questions “Are we making enough growth from Fall Benchmark to Winter Benchmark? If NOT, what is our plan to make instructional changes?” Purpose Reviewing progress at multiple levels: grade-level, classroom-level and small groups of students. – Fall to Winter Differentiating instruction as well as grouping students by instructional need.

4. Evaluate Student Response (Progress Monitoring & Outcome Assessments) Monitor & Adjust the Plan. Analyze the Problem Identify Variables that Contribute to the Problem. Develop a Hypothesis. Identify & Clarify the Problem (Screening & Diagnostic Assessments) Define and Validate the Problem in Measurable Terms. Develop & Implement a Plan (Goal Setting & Planning) Carry Out the Intervention & Monitor Progress. Problem Solving Process

5. Reflecting on Instruction Using the Problem-Solving Process Anywhere Elementary – Fall 2012 Steps 1 & 2: Define & Clarify the Problem  Universal screening in September showed 55% of 1 st grade students were making benchmark, as measured by PSF… which meant that 45% of 1 st grade students were NOT on their way to mastering phonemic awareness by the end of the year.

6. Reflecting on Instruction Using the Problem-Solving Process Anywhere Elementary – Fall 2012 Step 3: Analyze the Problem, Formulate a Hypothesis, and Devise a Plan  Analyze: 1 st grade reading instruction focused on phonemic awareness, phonics, and comprehension skills -20 min per skill per day, 5 days per week.  Hypothesize: Increasing the amount of instructional time devoted to blending and segmenting sounds (instead of larger units such as syllables) in words will improve student mastery of phonemic awareness by the end of the year.  Plan: 1 st grade reading instruction was re-allocated so that phonemic awareness instruction targeted blending and segmenting individual sounds in words, both during talking and writing activities. This instruction was provided in whole-class as well as small group formats (rather than only whole-class).

7. Reflecting on Instruction Using the Problem-Solving Process Anywhere Elementary – Fall 2012 Step 4: Implement the Plan 1 st grade teachers modified their reading instruction plan by targeting blending and segmenting activities at the sound level. Sound blending and segmenting were taught both during talking and writing activities. This instruction was provided in whole- class as well as small group formats, 20 min per day, 5 days a week

8. Reflecting on Instruction Using the Problem-Solving Process Anywhere Elementary – Fall 2012 Step 5: Evaluate After the universal screening in January, teachers looked at the data and asked… Did increasing the intensity of core phonics instruction improve the percentage of students mastering alphabetic principle and making benchmark at the end of the year???

9. Sample Demonstration: Data Packet for a Grade-Level Meeting Middle of Year 1 st Grade

10. Data Review Process – TIER I Data SourceWhat can we Learn? 1) Grade level View Box Plots Histograms Summary Report School-wide Effectiveness What big ideas of Core Instruction are priorities? How effective is our Core instruction, curriculum, & resources at meeting the needs of all kids? Who is learning as expected? Who is not? In what proportions? What is working well? What are our challenges and concerns? 2) Classroom Level View 3) Small groups and Individuals

11. Nonsense Word Fluency 1 st GradeMiddle of Year Is the phonics goal reasonable for this time of year in 1 st grade? How many made the benchmark in the fall? In the winter? How many are not yet at benchmark? How are the most at-risk students faring with this phonics goal? What are the implications for phonics instruction and this grade-levels’ next steps? Benchmark About 55% meeting Benchmark About 40% meeting Benchmark 5 th 20 th 95 th 80 th 50 th

12. Phoneme Segmentation Fluency 1 st GradeMiddle of Year Is the PA goal reasonable for this time of year in 1 st grade? How many made the benchmark in the fall? In the winter? Are you happy with this growth? How many are not yet at benchmark? How are the most at-risk students faring with this PA goal? How effectively is your core meeting the PA needs of students across the year? Benchmark About 90% meeting Benchmark About 55% meeting Benchmark 5 th 20 th 95 th 80 th 50 th

13. Oral Reading Fluency 1 st Grade Middle of Year Is the accuracy and fluency goal reasonable for this time of year in 1 st grade? How many made the benchmark in the winter? How many are not yet at benchmark? How are the most at-risk students faring with their accuracy and fluency in connected text? What are the implications for instruction and this grade-levels’ next steps? About 60% meeting Benchmark Benchmark 5 th 20 th 95 th 80 th 50 th

14. www.oakland.k12.mi.us 14 Purpose: Box plot is a quick, overall view of grade level student performance in relation to the benchmark goal. 90% student scores are captured on one graph. Box Plot Key Roland Good and Ruth Kaminski (2005) www.dibels.uoregon.edu What can be learned from this multiyear box plot? Are we making a difference, and if so, where? 95 th 80 th 50 th 20 th 5 th

15. Phoneme Segmentation Fluency 1 st GradeMiddle of Year 90 % Benchmark (n = 81) 10% Emerging (n=9) 0% Deficit (n=0) Benchmark Goal: 35 to 45 on Phoneme Segmentation Fluency by the end of Kindergarten or the beginning of First Grade. Middle Status: 35-45 sounds per minute on Phoneme Segmentation Fluency. Further progress on PSF above 45 does not appear to be necessary. How close are students to moving to the next category? Where are students within each category? How do the percentages compare to the distribution of students in each category? What story does it tell?

16. Benchmark Goal: 50 or more on Nonsense Word Fluency by the middle of First Grade. Middle Status: 50 or more correct letter sounds per minute on Nonsense Word Fluency. Nonsense Word Fluency 1 st GradeMiddle of Year 41% Benchmark (n = 37) 39% Emerging (n=35) 20% Deficit (n=18) How close are students to moving to the next category? Where are students within each category? How do the percentages compare to the distribution of students in each category? What story does it tell?

17. Benchmark Goal: 40 or more words per minute on Oral Reading Fluency by the end of First Grade. Middle Status: Students with at least 20 correct words per minute are likely to achieve the end-of-First Grade benchmark goal. 59% Benchmark (n = 53) 36% Emerging (n=32) 6% Deficit (n=5) Oral Reading Fluency 1 st GradeMiddle of Year How close are students to moving to the next category? Where are students within each category? How do the percentages compare to the distribution of students in each category? What story does it tell? End of Year Expectation

18. Developmental Reading Assessment (DRA) Benchmark: 3-4 Emerging: 2 Deficit: <2 Reported DRA scores represent student’s highest independent level & include accuracy, fluency, and comprehension consideration 10-12 6-8 <6 How has the percentage of students at Benchmark changed from Fall to Winter? What must students do to read at a level 8-10 with accuracy, fluency & comprehension? What information does this data add to the screening results to tell us about student needs related to accuracy, fluency & comprehension at mid-first grade?

19. Problem Analysis Activity: As a Member of this 1 st Grade Team, Think of 5 Questions Related to… What We Teach How We Teach

20. What we teach… What concepts/ big ideas/ outcomes are targeted by the core? Is there a scope & sequence for teaching the skill/concept? Is there an organizational framework or units of study to arrange the outcomes? Are materials available to meet a range of student instructional levels? Guiding Questions When Problem Solving with Grade-Level Data How we teach… What explicit instructional activities from core program target the concepts/ big ideas/ outcomes? How much time is allocated towards big ideas? Is there sufficient teacher modeling, guidance and corrective feedback? Is transfer of new skills to real reading and writing planned for and scaffolded? Is sufficient time allocated for practice of new skills? Is corrective feedback sufficient to assure accurate and fluent practice of skills? How will we monitor progress?

21. Activity: Reflect on the Process How did the data conversation feel? Fast? Slow? Confusing? Clear? Did you spend more time looking at & processing the data or more time engaged in dialogue? Why? Was it different from how your team currently dialogues with data? How?

22. Data Review Process- TIER I Data SourceWhat can we Learn? 1) Grade level View 2) Classroom Level View Summary of Effectiveness by Class Class Progress Graphs Class Lists What Big Ideas were priorities this past semester? Who is making growth and who is not? How effective is Core Instruction in maintaining students at benchmark? How effective are strategic and intensive instruction at accelerating students to benchmark? What are learning strengths and challenges (needs) in the classroom? 3) Small groups and Individuals

23. Did students at benchmark in Fall stay at benchmark in Winter? Did students who were strategic in Fall reach benchmark in Winter? Did students who were intensive in Fall reach benchmark? How effective is the classroom system in moving students with different levels of skills to the end of year goal? Summary of Effectiveness by Class

24. Which students made good gains? Which students are you concerned about? Is there a pattern between where students started and where they are at the middle of the year? Do growth spurts relate to your instructional emphasis?

25. 1 st Grade Winter Smith Classroom Organizer WORD STUDY: Phonemic Awareness, Alphabetic Principle, and Phonics MLPP Segmenting Goal = 7+ Phoneme Segmentation Fluency* Benchmark = 40 MLPP Letter ID Goal = 54 MLPP Sound ID Goal = 26 MLPP Hearing & Recording Sounds Goal = 36 Nonsense Word Fluency – CLS* < 49 DORF Accuracy* < 73% Anthony- 4, 3 Shyanne- 6, 4 class avg: 7 Anthony- 16 Shyanne- 25 Ryan- 30 Craig- 33 class avg: 43 Ryan- 17 Logan- 20 Sean- 20 class avg: 20 Ryan- 20 Anthony- 23 Craig- 25 class avg: 30 Craig- 20 Logan- 23 Ryan- 25 Sean- 26 Shyanne- 29 Riley- 31 Adam- 32 Rick- 32 Tessa- 36 class avg: 42 EMERGENT BEHAVIORS SELF- MONITORING: Application SELF-MONITORING & BOOK UNDERSTANDING-COMPRHENSION: Meaning, Structure, and Visual Cueing Systems MLPP Concepts of Print Goal = 22 Nonsense Word Fluency – WWR* < 5 DORF* < 19 wcpm MLPP Sight/ Decodable Word List- Preprimer: < 13 DRA with Retelling Goal = 10+/ F&P F+ (with 95% accuracy & adequate retell) MLPP Oral Language < 2 (note syntax, vocab, or fluency) Adam- 8 Shyanne- 9 Riley- 10 Logan- 13 Ryan- 13 Anna- 15 Dan- 17 Craig- 19 class avg: 47 Adam- 2 Shyanne- 2 Logan- 4 Riley- 4 Dan- 4 Ryan- 6 Craig- 6 Ryan- 8

26. Problem Analysis Activity: As a Member of this 1 st Grade Team, Think of 5 Questions Related to… What We Teach How We Teach

27. Guiding Questions When Problem-Solving with Classroom Data What we teach… What concepts/ big ideas/ outcomes are targeted by the core? Is there a scope & sequence for teaching the skill/concept? Is there an organizational framework or units of study to arrange the outcomes? Are materials available to meet a range of student instructional levels?

28. Guiding Questions When Problem-Solving with Classroom Data How we teach… How is my classroom core instruction meeting the needs of all kids? Are there commonalities among students who did not make growth? How does this translate into my instruction? Could I use this information for flexible groupings? Am I getting the growth that was expected given my instructional emphasis? Do I need to make changes? Is the purpose of each activity clearly stated and understood by staff and students? Is there sufficient teacher modeling, guidance, corrective feedback and practice? Is transfer of new skills to real reading and writing planned for and scaffolded? How will we manage the resources within the classroom to match the severity of the problem? Are activities sufficiently motivating to assure active engagement in learning? Are there other variables (classroom management,) that need to be addressed to improve academic engagement in materials? How will I monitor progress?

29. Winter Classroom Data Summary: SMITH Problem-Solving Step 5: Evaluate student response to my fall instructional priorities; Percent of students at Benchmark in fall who remain at Benchmark in winter:57% Percent of students at Strategic in fall who moved to Benchmark in winter:29% Percent of students at Intensive in fall who moved to Benchmark in winter:0 Problem-Solving Steps 1 & 2: Define & validate the problem using a classroom organizer to integrate screening (i.e.: DIBELS) and diagnostic (i.e.: MLPP, DRA) data. Problem-Solving Step 3: Analyze the problem by summarizing classroom data… Strengths:I am rocking the house with phonemic awareness- blending & segmenting skills are a strength Needs:I need to spend more time on explicit letter-sound instruction and connecting that instruction to real reading in instructional and independent text levels during small group instruction and conferences Questions that require more information: If I have students spend more time writing (eg: individual words/labels) in their journals, will it make a positive difference in their letter-sound and reading skills? Hypotheses Statements: 1. Increasing small group instruction time for students with letter-sound fluency needs and linking the instruction to writing and reading instructional level texts will improve overall reading accuracy and fluency. 2. I was using whole-group instruction targeting phonics skills 30 min/day this fall… Now I think that spending at least 15 min of the 30 min on small group instruction formats will accelerate students who are still below benchmark on NWF. Problem-Solving Step 4: Plan and implement instruction by stating what the priorities are for student learning based on the data and my hypotheses… Tier 1: I will continue to target phonics skills 30 min per day, spending 15 min on whole-group instruction (F&P Phonics Lessons) and 15 min on small group instruction differentiated for students still not automatic with letter-sounds and carryover to real reading. I will also target phonemic awareness segmenting skills 5-10 min per day and comprehension (strategy instruction) 20-25 min per day. Tier 2 (include student names and interventionist): Tier 3 (include student names and interventionist):

30. Activity: Reflect on the Process How did the data conversation feel? Fast? Slow? Confusing? Clear? Did you spend more time looking at & processing the data or more time engaged in dialogue? Why? Was it different from how your team currently dialogues with data? How? How might this classroom-level dialogue be different than the grade-level dialogue?

31. Data Review Process- TIER II and Tier III Data SourceWhat can we Learn? 1) Grade level View 2) Classroom Level View 3) Small groups and Individuals Class Progress Graphs Student History Progress Monitoring Graphs Who is responding? Who is not? Is there a performance problem or a skills problem? Do we have the right intervention or strategy, at the right time, for the right need, at the right level of intensity (Torgesen, 1998)? How much is this student responding compared to the average response in the group or the class mean? Are they likely to reach the goal without making further changes?

32. What information do you need to collect prior to holding Grade-Level Meetings? What grade-levels will you collect information on? Who will do this and how? Is there professional learning that staff needs to engage in before data can be collected or Grade- Level Meetings can take place? Activity: Action Planning

33. www.oakland.k12.mi.us/rti