1. Using Strategies, Protocols, and Tools to Analyze Data A Presentation of the National Reading Technical Assistance Center (NRTAC)
Speaker’s notes
Additional information
This module has been designed to expand Reading First participants’ knowledge and expertise related to assessment. Key information is stated directly on the slides with additional information provided in the presenter notes.
PRESENTER: Lexie Domaradzki
DATE: May 28, 2009
MEETING: Alaska School Leadership Institute

2. Using Strategies, Protocols, and Tools to Analyze Data
Understand how to deepen data analysis skills to inform instruction
Develop an institutionalized protocol for data analysis
Identify CBM reports that are most critical as you move forward with reading improvement

3. State Outcome Assessment Results:
Only a part of the whole picture
One point in time during the school year

4. Our Opportunity
Use frequent assessment results to see a more complete picture of the child’s skill development
Understand how to support the child with efficient, effective instruction that increases their opportunity for success

5. The path we take .....

6. Quote ....
“You can have the results you say you want, or you can have the reasons why you can’t have them. But you can’t have both. Reasons or results. You get to choose.”
Susan Scott

7. Assessment System
Outcome
Diagnostic
Screening
Progress Monitoring

8. Diagnostic Assessment
Assessment System
Outcome Assessment
Measure against standards
Screening Assessment
Examines who is at risk, compared to peers of same age
Diagnostic Assessment
Answers “why is this students struggling”
Each part of the system is important!
Progress Monitoring
Is this intervention working? Is the student growing?
Program
Is the student learning what has been taught? 8

9. Universal Screening All students Usually 3 times a year
Measure performance compared to students of their own age 9

10. Diagnostic Assessment
Assessment System
Outcome Assessment
Measure against standards
Screening Assessment
Examines who is at risk, compared to peers of same age
Diagnostic Assessment
Answers “why is this students struggling”
Each part of the system is important!
Progress Monitoring
Is this intervention working? Is the student growing?
Program
Is the student learning what has been taught? 10

11. Diagnosing-(looking for vulnerabilities)
Formal or Informal Diagnosis

12. Diagnostic Assessment
Assessment System
Measure against standards
Outcome Assessment
Screening Assessment
Examines who is at risk, compared to peers of same age
Diagnostic Assessment
Answers “why is this students struggling”
Each part of the system is important!
Progress Monitoring
Is this intervention working? Is the student growing?
Program
Is the student learning what has been taught? 12

13. Several diagnostic tools to use
Informal:
CORE phonics survey
Quick Phonics-Jan Hasbrouck
Diagnostic Decoding Surveys- Really Great Reading Company
Formal
Woodcock
CTOPP

14. Is the intervention having a positive impact?
Progress monitoring
Is the intervention having a positive impact?

15. Bridging the achievement gap
Data Analysis

16. Tools we use to analyze data

17. Keys to being efficient with data analysis
Establish a data analysis protocol
Implement protocol consistently
Provide training for new teachers/leaders
Understand the limitations of each assessment tool you are using

18. Purpose of Data Analysis Protocol

19. Data Analysis Protocol

20. Discussing the Protocol
What did you notice?
What questions did the protocol raise for you?
What is relevant or meaningful in this type of protocol?

21. Critical elements of using a data protocol
Analyze from broad to narrow
District-district and school patterns
School-school and grade level patterns
Grade level-grade level and classroom patterns
Classroom- overall class and student patterns
Create an environment of trust and openness

22. My Experience ... 

23. 3
100

24. Important Leadership Decisions
Which portions of the assessment are all teachers expected to give and how often?
Who will be the leader in learning the intricacies of the CBM data reports/implications of results?
How will we build consistent collaboration into our schedules?
What is expected of teachers regarding bringing and sharing data?

25. Important Leadership Behaviors
Establish expectations for “how we do business here” with CBM assessments and analysis
Analyze data prior to attending meetings if possible
Classroom observation focused on struggling students and instructional success
Celebrate successes regularly

26. 3
100

27. CBM reports

28. General Questions when using data reports
Achievement % report
What do you notice?
What questions does this data bring up?

29. Achievement % report Report shows you:
Percent of students Deficient, Emerging, Established on a given sub-skill
Number of students Deficient, Emerging, Established on a given subskill
Reports Fall, Winter, Spring results
Shows all subskills
All students included (not matched scores)
Limitations:
Doesn’t show which students are at risk
Doesn’t show individual schools on one report

30. Use the district data protocol
Achievement %
Use the district data protocol
Answer as many questions as possible with the overall District questions.
Which questions could you answer?
Which are left unanswered?

31. Use the district data protocol
Achievement %
Use the district data protocol
Answer as many questions as possible with the overall School questions.
Which questions could you answer?
Which are left unanswered?

32. Percent of Students Above & Below Target
Report shows you:
What percent of students are above and below target on a given subskill
All grades
All students (not matched scores)
Fall, Winter, Spring
Limitations:
Only shows one subskill at a time

33. Percent of Students Above and Below Target
Examine your own data:
Use the appropriate questions from the Data Protocol
What percent of students are performing on target or “at benchmark”?
What percent of students moved out of the “at risk” categories?

34. Percent of Students Above and Below Target
Examine your own data:
Use the appropriate questions from the Data Protocol
Which grade levels are performing exceptionally well on this subskill?
Which grade levels appear to be struggling?
Which sub-skill performance is showing success?
Which sub-skill performance is concerning?

35. Summary of Impact of Instructional Program
Report shows you:
Which students have shown growth on a given subskill
Which students hit the target on a given subskill
Which students continue to struggle on a given subskill
Matched score/only shows those students with two sets of scores
Limitations-
Only shows one subskill at a time

36. Summary of Impact of Instructional Program
Use Data Protocol to analyze your own data
Which questions were able to be answered?
Which were left unanswered?

37. Teacher Early Lit. scores and Instructional Recommendations
Report tells you:
Which students are scoring in the Benchmark, Strategic, Intensive categories
Student scores on each subskill
Limitations:
Percentile rank is locally normed-sample size too small to be reliable
Target not noted for reference

38. Secondary Reading System
What steps should be taken to assess reading skills in Middle School and High School?
How do we use the Assessment triangle to help us know where to start?
How can we be efficient with assess at risk students?
How do we really find the root cause of our Secondary students reading difficulty?

39. Secondary Reading System
Identify students at risk:
State outcome assessment
Program/district assessments
Give students who score well/ grade level fluency assessment
Give students who didn’t meet the target a diagnostic assessment
Give Oral Reading Fluency Screening
Below grade level

40. Tip of the Iceberg ... 

41. Our opportunity to help our students soar

42. It takes courage and skill to be unambiguous and clear
Quote ... 
It takes courage and skill
to be unambiguous and
clear
Peter Senge

43. Thank you .... Lexie Domaradzki Research Associate
RMC Research Corporation
Portland Oregon Office