Using Strategies, Protocols, and Tools to Analyze Data A Presentation of the National Reading Technical Assistance Center (NRTAC) Speaker’s notes Additional information This module has been designed to expand Reading First participants’ knowledge and expertise related to assessment. Key information is stated directly on the slides with additional information provided in the presenter notes. PRESENTER: Lexie Domaradzki DATE: May 28, 2009 MEETING: Alaska School Leadership Institute

Using Strategies, Protocols, and Tools to Analyze Data Understand how to deepen data analysis skills to inform instruction Develop an institutionalized protocol for data analysis Identify CBM reports that are most critical as you move forward with reading improvement

State Outcome Assessment Results: Only a part of the whole picture One point in time during the school year

Our Opportunity Use frequent assessment results to see a more complete picture of the child’s skill development Understand how to support the child with efficient, effective instruction that increases their opportunity for success

The path we take .....

Quote .... “You can have the results you say you want, or you can have the reasons why you can’t have them. But you can’t have both. Reasons or results. You get to choose.” Susan Scott

Assessment System Outcome Diagnostic Screening Progress Monitoring

Diagnostic Assessment Assessment System Outcome Assessment Measure against standards Screening Assessment Examines who is at risk, compared to peers of same age Diagnostic Assessment Answers “why is this students struggling” Each part of the system is important! Progress Monitoring Is this intervention working? Is the student growing? Program Is the student learning what has been taught? 8

Universal Screening All students Usually 3 times a year http://ies.ed.gov/ncee/wwc/publications/practiceguides/ Measure performance compared to students of their own age http://ies.ed.gov/ncee/wwc/publications/practiceguides/ 9

Diagnostic Assessment Assessment System Outcome Assessment Measure against standards Screening Assessment Examines who is at risk, compared to peers of same age Diagnostic Assessment Answers “why is this students struggling” Each part of the system is important! Progress Monitoring Is this intervention working? Is the student growing? Program Is the student learning what has been taught? 10

Diagnosing-(looking for vulnerabilities) Formal or Informal Diagnosis

Diagnostic Assessment Assessment System Measure against standards Outcome Assessment Screening Assessment Examines who is at risk, compared to peers of same age Diagnostic Assessment Answers “why is this students struggling” Each part of the system is important! Progress Monitoring Is this intervention working? Is the student growing? Program Is the student learning what has been taught? 12

Several diagnostic tools to use Informal: CORE phonics survey Quick Phonics-Jan Hasbrouck Diagnostic Decoding Surveys- Really Great Reading Company Formal Woodcock CTOPP

Is the intervention having a positive impact? Progress monitoring http://ies.ed.gov/ncee/wwc/publications/practiceguides/ Is the intervention having a positive impact?

Bridging the achievement gap Data Analysis

Tools we use to analyze data

Keys to being efficient with data analysis Establish a data analysis protocol Implement protocol consistently Provide training for new teachers/leaders Understand the limitations of each assessment tool you are using

Purpose of Data Analysis Protocol

Data Analysis Protocol

Discussing the Protocol What did you notice? What questions did the protocol raise for you? What is relevant or meaningful in this type of protocol?

Critical elements of using a data protocol Analyze from broad to narrow District-district and school patterns School-school and grade level patterns Grade level-grade level and classroom patterns Classroom- overall class and student patterns Create an environment of trust and openness

My Experience ... 

3 100

Important Leadership Decisions Which portions of the assessment are all teachers expected to give and how often? Who will be the leader in learning the intricacies of the CBM data reports/implications of results? How will we build consistent collaboration into our schedules? What is expected of teachers regarding bringing and sharing data?

Important Leadership Behaviors Establish expectations for “how we do business here” with CBM assessments and analysis Analyze data prior to attending meetings if possible Classroom observation focused on struggling students and instructional success Celebrate successes regularly

3 100

CBM reports

General Questions when using data reports Achievement % report What do you notice? What questions does this data bring up?

Achievement % report Report shows you: Percent of students Deficient, Emerging, Established on a given sub-skill Number of students Deficient, Emerging, Established on a given subskill Reports Fall, Winter, Spring results Shows all subskills All students included (not matched scores) Limitations: Doesn’t show which students are at risk Doesn’t show individual schools on one report

Use the district data protocol Achievement % Use the district data protocol Answer as many questions as possible with the overall District questions. Which questions could you answer? Which are left unanswered?

Use the district data protocol Achievement % Use the district data protocol Answer as many questions as possible with the overall School questions. Which questions could you answer? Which are left unanswered?

Percent of Students Above & Below Target Report shows you: What percent of students are above and below target on a given subskill All grades All students (not matched scores) Fall, Winter, Spring Limitations: Only shows one subskill at a time

Percent of Students Above and Below Target Examine your own data: Use the appropriate questions from the Data Protocol What percent of students are performing on target or “at benchmark”? What percent of students moved out of the “at risk” categories?

Percent of Students Above and Below Target Examine your own data: Use the appropriate questions from the Data Protocol Which grade levels are performing exceptionally well on this subskill? Which grade levels appear to be struggling? Which sub-skill performance is showing success? Which sub-skill performance is concerning?

Summary of Impact of Instructional Program Report shows you: Which students have shown growth on a given subskill Which students hit the target on a given subskill Which students continue to struggle on a given subskill Matched score/only shows those students with two sets of scores Limitations- Only shows one subskill at a time

Summary of Impact of Instructional Program Use Data Protocol to analyze your own data Which questions were able to be answered? Which were left unanswered?

Teacher Early Lit. scores and Instructional Recommendations Report tells you: Which students are scoring in the Benchmark, Strategic, Intensive categories Student scores on each subskill Limitations: Percentile rank is locally normed-sample size too small to be reliable Target not noted for reference

Secondary Reading System What steps should be taken to assess reading skills in Middle School and High School? How do we use the Assessment triangle to help us know where to start? How can we be efficient with assess at risk students? How do we really find the root cause of our Secondary students reading difficulty?

Secondary Reading System Identify students at risk: State outcome assessment Program/district assessments Give students who score well/ grade level fluency assessment Give students who didn’t meet the target a diagnostic assessment Give Oral Reading Fluency Screening Below grade level

Tip of the Iceberg ... 

Our opportunity to help our students soar

It takes courage and skill to be unambiguous and clear Quote ...  It takes courage and skill to be unambiguous and clear Peter Senge

Thank you .... Lexie Domaradzki Research Associate RMC Research Corporation Portland Oregon Office ldomaradzki@rmccorp.com