1. Interpreting Literacy Data for Effective Instructional Grouping Grades 2-5 Moira K. McKenna North West Positive Behavior Intervention and Support Network Conference March 8, 2010 McKenna (2010)

2. Academic SystemsBehavioral Systems 1-5% 5-10% 80-90% Intensive, Individual Interventions Individual Students Assessment-based High Intensity Intensive, Individual Interventions Individual Students Assessment-based Intense, durable procedures Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Universal Interventions All students Preventive, proactive Universal Interventions All settings, all students Preventive, proactive Designing School-Wide Systems for Student Success McKenna (2010)

3. 1-5% 5-10% 80-90% Intensive, Individual Interventions Small Group/Individual students Assessment-based High Intensity Intensive, Individual Interventions Individual students Assessment-based Intense, durable procedures Targeted Group Interventions Some students (some risk) High efficiency Rapid response Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Universal Interventions All students Preventive, proactive Universal Interventions All settings, all students Preventive, proactive EBISS EBISS Effective Behavioral and Instructional Support Systems Academic SystemsBehavioral Systems McKenna (2010)

4. Curriculum-Based Measurement (CBM) Definition and Evidence-Base CBM is a brief, standardized assessment that documents student achievement through a systematic sampling of skills that represent the annual curriculum (Fuchs, 2004; Shinn, 2002, 1998, 1989; Deno, 1986) Alternate passages are of equivalent difficulty, whereby each measure is represented by the same level of complexity, gaining an accurate measure of student growth Growth is measured by Universal Screening and Progress Monitoring McKenna (2010)

5. Types of CBM General Outcome Measures (GOM) – application of skill to independent task – leveled passages that can be used for progress monitoring Skills-Based Measures (SBM) – leveled measures that assess proficiency on a specific set of skills that students are expected to perform per grade-level standards – Most commonly seen in mathematics/mixed math computation Mastery Measures (MM) – Focuses on student attainment of finite skills – not appropriate for progress monitoring McKenna (2010)

6. Utility of CBMs Screening Decisions – identify which students may need instructional support Progress Monitoring Decisions – decide when to modify instruction, teach new skills, and/or revise goals Diagnostic Decisions – to target specific skill(s) for support Outcome Decisions – to modify instruction, change intervention, or reintegrate back into general education support McKenna (2010)

7. CBM as Convergent Data Technically reliable and valid GOMs and SBMs will be used for Universal Screening and Progress Monitoring of student performance MM will be used to determine if a student is able to present skills taught in a lesson or unit Student performance measures from these, and other relevant sources of information, will be used to determine student growth as aligned with standards McKenna (2010)

8. Big idea General Outcome Measure 1.Phonological Awareness 2.Alphabetic Principal 3.Fluency (& accuracy) with connected text 4.Vocabulary 5.Comprehension Initial Sound Fluency Nonsense Word Fluency DIBELS Oral Reading Fluency (DORF) DIBELS Oral Reading Fluency (DORF); AIMSweb MAZE Phoneme Segmentation DIBELS: Dynamic Indicators of Basic Early Literacy Skills DIBELS is an example of a measurement system AIMSweb: Letter Sound Fluency (LSF; Alphabetic Principle ) (Fuchs & Fuchs, 2004): Word Identification Fluency (WIF; Alphabetic Principle) McKenna (2010)

9. Early Literacy Skills of Phonemic Awareness assessed in fall, winter and spring – Ability to manipulate sounds auditorily at the phoneme level – foundation skill set required to become a proficient reader – phoneme segmentation as capstone skill representing knowledge in rhyme, on-set rime, and blending – general outcome measures include Initial Sound Fluency (ISF), and Phoneme Segmentation Fluency (PSF) McKenna (2010)

10. Early Literacy Letter naming – Adds confidence that a student is on track to be a successful reader – Does not link to one of the five essential components of beginning reading Phonics / Alphabetic Principle – the ability to link letters to their representative sounds in text – general outcome measures (GOMs) include letter- sound fluency (LSF) and nonsense word fluency (NWF). McKenna (2010)

11. Early Literacy Beginning Reading Automaticity with Code – Reading accurately and fluently with prosody – Measured by rates of Oral Reading Fluency (ORF) – “Best” versus “fastest” reading Vocabulary – Gain meaning from words and word parts – Technically adequate general outcome measure not available; unstandardized measure through error analysis of ORF – Measured by in-program assessment Comprehension – Gaining meaning from text – Measured efficiently by MAZE task McKenna (2010)

12. Oral Reading Fluency Mid-First grade through 6 th grade – Norms available through 8 th, 12 th grade using 8 th grade passages DIBELS and AIMSweb: 2 examples of measurement systems – http://www.rti4success.org/ Students who’ve met benchmark, maintain instructional program – differentiate through small group instruction McKenna (2010)

13. MAZE for Upper Elementary Measure of reading comprehension Comprehension is typically linked with phonics deficits Every 7-10 words in a passage, student chooses the best fit of 3 words in the context of what is being read Correct, exact word from passage Near Distracter – word of the same ‘type’ of word from passage (e.g. noun, verb, adverb) Far Distracter – word not of the same type, selected randomly from the passage MAZE recommended grade 4 and beyond, better predictor of future reading performance (Hosp, Hosp, & Howell, 2007). MAZE can be used as a supplemental measure [with ORF] to provide a more complete picture of a student’s reading skills (Shinn & Shinn, 2002) McKenna (2010)

14. Outcomes Driven Model Good, Gruba, & Kaminski (2002) McKenna (2010)

15. Data Team Meetings Reviews Universal Screening and Progress Monitoring data – 3x’s per year for benchmark data review – at least once in between benchmark periods to review progress monitoring data; monthly meetings, best practice when intervention programs are established – align meeting schedule with decision-making guidelines for frequency of data review in your school district Data Team membership includes principal, literacy coach, school psychologist, special education teacher, ELL instructor, other as locally indicated McKenna (2010)

16. Data Team Meetings Options for meeting with grade-level teams (1)Grade level teams rotate throughout day Hire substitutes who will rotate with teachers efficient, occurs in 1 day Approximately 45 minute meetings (2)Schedule end-of-day meeting across a week 45 minute sessions, grade level teams organized by day during the week Schedule meeting times at the beginning of the year! Plan effectively McKenna (2010)

17. Data Team Meetings Data team meets first, reviews data, and identifies specific discussion targets and outcomes for specific grade levels Data team meets with Grade-Level teams – Identify successes through use of data – Identify skill areas where students are not performing proficiently – Discuss potential adjustments that can be made to increase instructional efficacy as realized by student gain; problem-solve challenges – Review progress for concerns of specific students McKenna (2010)

18. Required Data for Grouping Summary of Effectiveness report per grade level – Student movement from fall to winter – Student movement from winter to spring Summary of Effectiveness report for same skill Class list reports Individual student performance data from Universal Screening and Progress Monitoring measures Progress Monitoring data for students receiving differentiated and targeted instruction and intervention ELPA scores for students who are ELL Grade-level teams bring in-program assessment data McKenna (2010)

19. Summary of Effectiveness Overview Review student movement over time – fall to winter – winter to spring How do I read it? Celebrate movement! Identify what worked to move students forward and maintain gains Review student regression: go deeper with other reports and actual student performance data McKenna (2010)

20. Summary of Effectiveness Systems Focus: Identify Precision Statements Identify precision statements to effectively answer questions from the data Precision statements have answers for who, what, when, where, and why – Who: What group of students – What: Skill area; specific skills – When: …and for how long does instruction occur? – Where: Does instruction occur in classroom or through pull-out support? Is environment conducive to learning? – Why: What variables that we can control affect learning? How can wrap-around support be provided to mediate other variables? McKenna (2010)

21. Summary of Effectiveness Systems Focus: Questions to Answer Did 100% of students remain at benchmark? – How effective is the core program? What percentage of students moved out of a strategic range of support to benchmark? – What support did students receive? Differentiation through Core? Intervention? What percentage of students moved out of an intensive range of support, performing at a strategic or intensive level? – How was their instructional program defined? McKenna (2010)

22. Summary of Effectiveness Systems Focus: Questions to Answer What percentage of students did not remain at benchmark? – If over 20% moved back, review effectiveness of core program and make adjustments Professional Development? Time, grouping, instruction? Question to answer: How do we support program fidelity? – Review individual student performance data to determine support that may be required McKenna (2010)

23. Summary of Effectiveness Systems Focus: Questions to Answer What percentage of students moved from a strategic range of support to requiring intensive support? – How was instructional programming defined? – Assess fidelity of program implementation and opportunity to learn – Review variables of time, grouping, and instruction to increase program effectiveness – Creative problem-solving, ‘Think Outside the Box’ LEAVE WITH A PLAN McKenna (2010)

24. Summary of Effectiveness Lets look at some data! McKenna (2010)

25. Approaching Class Lists Summary of Effectiveness is critical to review student movement and performance on a systems-level – inform adjustments that need to be made within and across grade levels to enhance core instruction – School-wide review of intervention programming Class lists and individual student performance data allows us to take closer look at where the break down is for students who are not demonstrating proficiency McKenna (2010)

26. Guidelines for the Interpretation of Multiple Measures Identify the GOMs that represent pre-requisite skills in order to build a foundation for the attainment of other skills as aligned with the big ideas of beginning reading Review performance across all measures within the benchmark period fall 2 nd grade, NWF and ORF; mid-second through 5 th, ORF Identify how discrepant a student is from the benchmark or normative data; validate concern Discrepancy Ratio = Expected Performance / Student Performance Severely discrepant is 2x’s or greater We want to intervene much earlier Magnitude of discrepancy and need for skill instruction indicates level of intensity of intervention required McKenna (2010)

27. Guidelines for Multiple Measures What level of support is required for students to be successful? Grade-Level Intervention / Walk to Read – differentiate within core program – replacement core if enough students in grade level require support to be successful Specific Skill Instruction – Reteaching, practice and repetition of skills not mastered – Phonics inventory to target skill instruction – Differentiation and/or intervention Widely Discrepant performance – Provide intensive support with explicit and systemic intervention program – replacement core McKenna (2010)

28. Questions to Answer How effective is our Core program? – are at least 80% of students responding to core instruction? For which students is the core program effective and not effective? – Strategic – Intensive – How discrepant is student performance? What skills need to be targeted for support? – Enhancement of Core program for all students – Small group skill instruction in core program – Supplemental program and Intervention McKenna (2010)

29. In review of student performance for all… Identify specific populations of students – English Language Learners – Special Education – Review individual/group instructional programs for strength in alignment of specificity of need within system of support Convergence of Evidence Validate need for support – do other measures of student performance also indicate skill deficit? – Reassess student to confirm need for support and to more closely review patterns in student performance McKenna (2010)

30. Universal Screening and Progress Monitoring Data Has the student/instructional group demonstrated gains in skill with progress monitoring? What is the current instructional program? – Is the student responding? Performance above the aim line (4 data points)? Performance below the aimline (4 data points)? No growth? Stable growth along the aimline (data hovers above and below)? – Is the program being implemented as intended? Identify alterable variables of time, grouping and instruction to either intensify support, maintain support, or fade support/reintegrate Review data sources for Convergence of Evidence McKenna (2010)

31. Second – Fifth Grade: Big Ideas Students ‘not established’, requiring strategic support to make gains would benefit from small group instruction targeted to specific skill deficits that supports phonics skills in-classroom support beyond the core reading block may be indicated differentiate between higher and lower performance in strategic range use phonics inventory to identify specific skill sets for instruction McKenna (2010)

32. Second – Fifth Grade: Big Ideas Students ‘not established’, requiring strategic support to make gains progress monitor every other week if in the classroom; pull-out, progress monitor weekly (2x’s/wk, best practice) monitor accuracy and fluency – Look for patterns in performance – The Devil’s in the details Bottom line: It’s about phonics support. McKenna (2010)

33. Second – Fifth Grade: Big Ideas Student’s requiring an intensive level of support to make gains – Identify median rate of accuracy across passages – Identify how discrepant a student is from his/her peers: * Discrepancy Ratio = performance goal/ student performance Student performance is considered severely discrepant if at or above 2 times discrepant Complete Phonics Inventory. Skill deficits are typically across the board McKenna (2010)

34. Second – Fifth Grade: Big Ideas Student’s requiring an intensive level of support to make gains Typically, an explicit and systematic program will be required to support meaningful gains Progress Monitor Weekly – 2x’s/wk, best practice – intensify progress monitoring when more data is needed sooner; for example, the data team meets in a month to review progress for specific students Monitor accuracy and fluency – Look for patterns in performance McKenna (2010)

35. Second – Fifth Grade: Big Ideas Low Accuracy, Low Rate of Oral Reading Fluency If student is widely discrepant (2x’s) consider administering a measure of NWF in order to determine is established in meeting the performance expectation of 50 letter sounds per minute and in the ability to recode/blend 15 CVC words. Assess if goal for PSF has been met (Phoneme Segmentation Fluency) Provide remediation specific to skill deficit(s) via explicit instruction through: – differentiation in the core or in a curriculum specific to phonics development Small Group instruction Pre-teach lesson; target specific skill sets for repetition toward mastery McKenna (2010)

36. Second – Fifth Grade: Big Ideas Low Accuracy, Low Rate of Oral Reading Fluency – Supplemental instruction through intervention beyond the core program (e.g. Phonics for Reading) – Intensive Support through a core program that addresses the essential components of beginning reading (e.g. Reading Mastery, Corrective Reading) – Additional opportunities for practice and repetition occur throughout the day to reinforce explicit skills taught (on the way to lunch, from recess, 5 minute wait before bus, check out before leave for activities and end of day) McKenna (2010)

37. Second – Fifth Grade: Big Ideas Accurate, Low Rate of Oral Reading Fluency If student is widely discrepant, yet accurate, review existent error patterns. – Accurate, meaning at least a rate of 97% or higher – Student(s) may benefit from instruction in syllabication and/or intervention that teaches an efficient approach to reading multisyllabic words. – Assess if grade-level class has been taught skills of syllabication. If not, teach set to class. Otherwise, provide small group instruction within class. – In addition to support for approaching multisyllabic words, build fluency through repeated reading program. McKenna (2010)

38. Second Grade Student’s not at benchmark, and for some who are, the primary issue will be one of phonics Identify skills that have been taught and that students are expected to perform with classroom teacher and interventionist If students have not met benchmark for ORF, determine if they have met benchmark for NWF in the fall – review previous data for NWF and PSF Determine skill deficits through Phonics Screener and error analysis of NWF McKenna (2010)

39. Second Grade Students not meeting benchmark on NWF Consider degree of discrepancy If widely discrepant (2x+) and multiple types of errors, collect enough data to adequately identify problem 1) Define instructional program to meet level of intensity of support needed – Is Reading Mastery required to support skill attainment? – Can support be met using Phonics for Reading? 2) Preteach or Reteach components of instructional program in which the student will participate 3) Prioritize and consider the most valuable use of instructional time; add instructional time for additional practice in most discrepant skill areas McKenna (2010)

40. Second Grade Students not meeting benchmark on NWF If students are not widely discrepant (< 2x’s), – analyze performance on NWF probe for accuracy, error patterns, and blending (words recoded) Identify error patterns – vowel errors – decoding accurately, not blending – specific sound errors across nonsense words Provide small group instruction specific to error patterns identified, blending, or both (collect more data if unsure to validate problem identification) Pre-teach phonics elements of lesson in core program Align intervention program to instructional need (see Interventions at end of presentation) McKenna (2010)

41. Second Grade Students meeting benchmark on NWF in fall not on ORF in Winter Is the student reading with 97% accuracy or higher? – if yes, reinforce skills initially mastered with additional practice, reteach error patterns that are identified, and build fluency – if no, phonics screener and differentiate within class – depending on degree of discrepancy and breadth of skills for which the student is not proficient, consider supplemental or intensive intervention program In a review of error patterns, had the skill been taught and is the student expected to perform it? McKenna (2010)

42. Third-Fifth Grade Reference notes for Grades 2-5, big ideas If ORF is not been met in 3 rd grade, consider discrepancy from expected performance, complete phonics inventory – Has the goal for NWF been met? – Syllabication typically taught in 3 rd grade If error patterns in 4 th and 5 th grade are primarily found with multisyllabic words, reteach to class/group in class – If error patterns are beyond that of multisyllabic words, complete phonics inventory McKenna (2010)

43. Vocabulary? Comprehension? If students do not have skills and strategies to approach reading text fluently, they will not be able to gain meaning from words or text Focus intervention on prerequisite skills in order to allow for independent access to content If a student reads with 97% accuracy or higher and would benefit from repeated reading, provide intervention beyond that of core reading instruction – Students with high accuracy, low fluency will frequently need to re-read text to gain meaning – Repeated reading will assist to build fluency McKenna (2010)

44. Vocabulary and Comprehension Taught through anthology in core program Schedule intervention groups to provide access to instruction for vocabulary and comprehension in the core program – learners requiring intensive intervention Specific strategies for specific word instruction, word-learning strategies, and comprehension can be embedded in the core program to enhance it’s effectiveness Common language for use of skills and strategies should be cued and pre-corrected throughout content areas McKenna (2010)

45. Vocabulary Students requiring support beyond that of differentiation within the classroom will require support in the area of phonics Embed in Core Instruction Language for Thinking and Language for Writing – Training required for fidelity – Particularly useful for students that see SLPs and teachers of ELL McKenna (2010)

46. Comprehension Comprehension Strategy Instruction – Focus on teaching 1 strategy at a time, well – Teach few, versus many, over time with depth – Background knowledge, summarization, comprehension monitoring, sentence comprehension, story comprehension Soar to Success – Grades 3-8: comprehension strategies, reciprocal skills, graphic organizers – Effective for Comprehension and Fluency McKenna (2010)

47. Forming Instructional Groups The goal is to identify what instructional program will work for the student/group of students Align intervention by need for skill instruction – Differentiate in class or provide intervention – Many programs have placement tests Group size approximately 6 students – fewer students with more intensive need for support If support is being provided outside of the classroom, intervention time will be indicated by program. Most intervention programs will require at least 40 minutes of instructional time – Additional guidelines for instructional time in Carnine et al. (2006) McKenna (2010)

48. Forming Instructional Groups Often occurs with building principal and literacy coach – it is helpful to coordinate scheduling with SPED All teachers have a copy of the intervention schedule Considerations for efficiency – Students walk to intervention room – progress monitor 1-2 students daily at end of lesson McKenna (2010)

49. Considerations for the Interpretation of Spring Data Is spring data used as a formative or summative measure of student performance? What questions are being answered by the data? Will instructional groups shift this school year? – Does flexible grouping occur through scheduled review of progress monitoring data? Will tentative groups be formed for the fall? McKenna (2010)

50. Evidence-Based Supplemental and Intervention Programs http://oregonreadingfirst.uoregon.edu/ http://www.fcrr.org/FCRRReports/index.aspx McKenna (2010)

51. Supplemental and Intervention Programs versus Replacement Core Supplemental to enhance core program for all students prevent/remediate skills for students in core who are somewhat below grade level Intervention prevent/remediate skills for students in core who are somewhat/significantly below grade level Intervention to replace core for students who are substantially below grade level Replacement core addresses all 5 big ideas of reading more explicit instruction of finite skills, moves at slower pace with some exceptions McKenna (2010)

52. Phonics Explode the Code (1 st – 3 rd grade) – Independent work; no research – Practice on explicit skills that have been taught Phonics For Reading 1 st grade- 5 th grade; Adult’s Learning English Language use for skill gaps, strategic support inexpensive and easy to implement Daily 40-50 minutes or split lessons in 1/2 REWARDS ◦ multisyllabic words ◦ PLUS, focuses on Science and Social Studies See Florida Center on Reading Research and Oregon Reading First websites for reviews of other programs for phonics support McKenna (2010)

53. Fluency Read Naturally (50 words correct per minute to adult) – Intervention program, not instruction – Primarily fluency-building; strong research – 97% accuracy or higher – 3, 30 minute lessons/week minimum Great Leaps (grades k-adult) – Fluency-only, easy to implement, 10-15 minutes daily Six Minute Solution (grades K-9) – Derived from a strong research base; 3 levels Only 6 minutes of instructional time daily McKenna (2010)

54. Replacement Core Programs Reading Mastery Classic l, ll, lll (SRA) / Fast Cycle ◦ Modified orthography used in first level to emphasize the specific sounds of letters ◦ Particularly powerful for children with significant language as well as literacy deficits Horizons (SRA), can include Funnix (computer-based) ◦ A, B; A/B covers 2 years in 1 to catch kids up to peers ◦ Uses traditional orthography vs. Reading Mastery Corrective Reading for Decoding and Comprehension (SRA) – 3 rd -12 th grade McKenna (2010)

55. Replacement Core: Critical Points Continue to teach skills of vocabulary and comprehension from evidence-based core program Not meant to be a long-term solution for large groups of students – e.g. use Reading Mastery Fast Cycle to MOVE students forward and to ACCELERATE their growth toward criterion and benchmark standards Use progress monitoring DATA to determine if students are gaining skills and generalizing them Define a plan for REINTEGRATION back into the core program (e.g. semester break in January) McKenna (2010)

56. Instructional Grouping Take Home Points Use instructional recommendations as a guide; Do not go by instructional recommendation alone Look more closely at the data and specific skill areas of deficit and proficiency; be discerning Leave meetings with a plan for each grade level Think creatively, outside of the box Identify the smallest change that can be made to make the biggest impact in student performance McKenna (2010)