1. Progress Monitoring in Reading: Why, What, and How
A module for pre-service and in-service
professional development
MN RTI Center
Authors: Lisa Habedank Stewart, PhD & Adam Christ, graduate student Minnesota State University Moorhead
click on RTI Center

2. MN RTI Center Training Modules
This module was developed with funding from the MN legislature
It is part of a series of modules available from the MN RTI Center for use in preservice and inservice training: 2 2

3. Overview This module is Part 1 of 2
Part 1: Why, What, How to Progress Monitor
Why do it?
What do we mean by progress monitoring?
How do you do it?
Part 2: Using Progress Monitoring Data
Introductory slide showing overall flow of the materials. The instructor could choose to do these two modules on different days/class periods depending on the length of the class session, sophistication of the learner, and activities and discussion time needed. 3

4. Is this student making progress?
Discussion could be added here for the class or small groups to look at and discuss what they think this graph means… is the student making progress? How do we know? What does the vertical line (intervention change/phase line) mean…. Is there a difference in the data before and after the change? Why do you think the teacher changed what was happening with Adam?
Can Use this also to teach about the graphing conventions (axes, date on bottom, individual data points, etc etc) 4

5. Assessment: One of the Key Components in RTI
Curriculum and Instruction
Assessment
School Wide Organization & Problem Solving Systems
(Teams, Process, etc)
Adapted from Logan City School District, 2002 5

6. Assessment and Response to Intervention (RTI)
A core feature of RTI is identifying a measurement system
Screen large numbers of students
Identify students in need of additional intervention
Monitor students of concern more frequently
1 to 4x per month
Typically weekly
Diagnostic testing used for instructional planning to help target interventions as needed
2 primary measurement needs; screening and progress monitoring. This module will focus progress monitoring. There is another module specific to screening. Diagnostic testing expertise is also needed, but typically teachers are more familiar with tools and the process for assessment for instructional planning/diagnostic testing as the term is being used here, than with screening and progress monitoring. 6 6

7. Screening Data and Progress Monitoring can be linked
The goal is to have a cohesive system.
If possible, use the same measures for both screening and progress monitoring (e.g, CBM).
Screen ALL students 3x per year (F, W, S)
Strategic Support and Monitoring
Students at Some Risk
What do we mean by cohesive system?
Ideally the progress monitoring data should be linked to other types of data and data collection in the school. For example if the school does universal screening(benchmarking) it is very nice if those same measures can be used for progress monitoring.
What do we mean by CBM General Outcome Measures? See General Outcome Measures CBM module. GOMs can be used to screen all students in the grade level 3x per year and they can also be used to do weekly progress monitoring with students who are at-risk.
Note: Other types of information (diagnostic information, high stakes tests, etc) are not good for progress monitoring but obviously also have their own purpose and place in the school’s decision making process
I like this slide here. I don’t think cohesive system is too technical. If anything, I would say converging data is more technical. I think that if you write down here in the notes what you mean by each, I think the facilitator will be able to understand and explain to the students what is meant by this slide.
Intensive Support & Monitoring for Students at Extreme Risk 7 7

8. Why Monitor Progress? When teachers USE progress monitoring
Students learn more!
Teachers design better instructional programs
Teacher decision making improves
Students become more aware of their performance Safer & Fleishman, 2005 8

9. Why Monitor Progress? “In God we trust… All others must have data.”
Dr. Stan Deno
Main point- as professional educators, this is an expectation.
Dr. Stan Deno is one of the original CBM researchers and a long-time advocate of data-based decision making. 9

10. Why Monitor Progress?
We do NOT KNOW ahead of time whether an intervention will be successful for an individual student
Do they ASSUME in the hospital that your heart is working just fine after your bypass surgery? After all… the surgery works well for MOST patients…..
Main point- as professionals we use data to make sure a student is benefiting
Also makes point that the more worried we are about a student (and the more resources being pumped into instruction) the more we should be sure to monitor progress closely 10

11. Why Progress Monitor Frequently?
To change what you are doing with a student if it is not working (formative assessment) so you are effective and efficient with your time and instruction
To help make decisions about instructional goals, materials, levels, and groups
To aid in communication with parents
To document progress for special education students as required for periodic and annual reviews
Main Point: Frequent Progress Monitoring has many uses, but the MOST important one is the first reason
Formative Assessment means you gather information over time and change what you are doing in response to the data
DRAFT May 27, 2009
Credit: based on slide by Dr. Kim Gibbons, SCRED 11

12. Change (and Keep Changing) if Instruction isn’t Working…
This graph demonstrates that just implementing an intervention does not mean that the student will get better. It gets at the point of how valuable frequent progress monitoring is and shows that if the first couple of interventions were kept in place the rest of the school year, the student probably would not be reading at his/her current level.
Credit: SCRED 12

13. You Can Also Use Monitoring Data to Celebrate Success!
Main Point- Data also help us demonstrate when a student is making progress and can also lead to sharing “what worked” with fellow staff.
Note: This is a graph of “real” data on phonemic awareness for a K student (weekly phoneme segmentation fluency data)- but not all students make growth that is this dramatic!) This graph was generated in the AIMSweb web-based software program.

14. What is Progress Monitoring?
Standardized measures
Reliable
Valid
Tied to important educational outcomes
Given frequently (e.g., weekly)
Simple, brief, efficient, and cheap 
Sensitive to growth over short periods of time
Main point are the 3 bullets- all 3 need to be in place for the type of progress monitoring we are talking about in this module
What makes a measure reliable for progress monitoring purposes?
Interrater and Test-Retest and Alternate Form consistency (if you have two people score it, if you give it twice within a short period of time or if you give two different probes that are supposed to be the same difficulty you get very similar or identical scores – correlations of .80 or higher, .90+ is gold standard.)
What makes a measure valid for progress monitoring purposes?
The measure has content and construct validity (measures the domain of interest and has evidence it measures it as well or better than other established measures) and has evidence it is useful specifically for progress monitoring (evidence it is sensitive to interventions and when educators use it to monitor progress and make decisions, students do better)
What do we mean by “tied to important educational outcomes”?
The progress monitoring measures are indicators of really important outcomes (e.g., becoming an overall really good reader!, passing the state reading “test”) See General Outcome Measure (CBM) ppt. You also should be able to use progress monitoring to aid in RTI decisions about intervention effectiveness and to monitor progress in terms of IEP goals. 14

15. Measurement and RTI: Progress Monitoring
Reliability coefficients of r=.90+
Well documented treatment validity!
Test and scores are very sensitive to increases or decreases in student skills over time
Evidence of what slope of progress (how much growth in a day, week or a month) is typical under what conditions can greatly increase your ability to make decisions
VERY brief, easy to use, affordable, alternate forms, and results/reports are accessible immediately
See Measurement Overview for more related information

16. Standards for Scientifically Based Progress Monitoring
National Center for Progress Monitoring
Reliability
Quality of a Good Test
Validity
Sufficient Number of Alternate Forms and of Equal Difficulty
Essential for Progress Monitoring
Evidence of Sensitivity to Intervention Effects
Critical for Progress Monitoring
Benchmarks of Adequate Progress and Goal Setting
Rates of Improvement are Specified
Evidence of Impact on Teacher Decision-Making
Critical for Formative Evaluation
Evidence of Improved Instruction and Student Achievement
Gold Standard
Provides information about center criteria and the link about the center for progress monitoring
Main point here is that there is an easily available resource- the national center for progress monitoring, that has information on progress monitoring and has reviews of different tests and test providers and whether they meet these criteria. Can then click on the tests to get more information….
Suggestion: If time allows and you have an internet link, go to the web site and explore the web site for a few minutes to show the students what is on the web site.
DRAFT May 27, 2009 16

17. Many tools that meet standards are members of the Curriculum-Based Measurement (CBM)
“family.”
click on “Tools”
Main Point- some of the best tools out there for progress monitoring in reading are in the CBM “family” (see General Outcome Measure CBM module). AIMSweb and DIBELS are both in the CBM “family” as are the MBSP measures and Progress Pro and other tools on the list that do not appear in this screen shot.
Note that this graphic shows that Accelerated Reader-Reading does NOT have adequate reliability, and some of the measures are “better” (meet all standards) than others
This graphic is an excerpt from the December 2007 version of the studentprogress.org Tools review (this is page 1 of 3 total pages on the website). Some tools reviewed are not in the CBM family (e.g., TOWRE and the TOSWRF and the STAR) but some of these do not meet standards. Also some of the CBM “family” measures (e.g., DIBELS retell fluency, word use fluency) also do not meet standards… 17

18. Buyer Beware!
Many tools may make claims about monitoring progress but…
Is it reliable and valid?
Is it tied to important educational outcomes?
Can it be given frequently (e.g., weekly)?
Simple, brief, efficient, and cheap 
Sensitive to growth over short periods of time
Optional Activity: Hand out information on the TORC-4.
Have students look at information on the TORC-4 with discussion questions at the end
?is it reliable- can you tell? Do you need more information? Where could you find it?
?is it valid-
…..
? Based on just the information provided, do you think this tool would be good for progress monitoring
(answer is NO…. One easy “reason” the answer is no is that it is too long…) 18

19. Short Term (Mastery) and Long Term Progress Monitoring
Mastery Monitoring
Test subskill mastery
and individual lesson effectiveness
Ex: Q&A, worksheets
following directions
unit tests, “hot” reads
accuracy, skills “checks”
CBE, cba
Long Term
General Outcome Measures
Test retention,
generalization and progress
toward overall
general outcome (reading)
Ex: CBM, DIBELS
Main Point- Schools need to use long term measurement for high quality progress monitoring
When it comes to progress monitoring… can use lots of things to monitor progress. For RTI what you need is at least ONE method of progress monitoring that is a GOM (General Outcome Measure)…. A Term used to describe the “CBM family”… see the General Outcome Measure CBM module for more information
Both types of progress monitoring (short term and long term) are very important for the professional teacher.
Mastery monitoring includes getting 10 of 10 math facts correct, being able to pick out the key parts of a story for comprehension, completing an in-class writing assignment with all the key writing components, passing the unit test on the Native Americans of NW Minnesota, etc. Mastery monitoring is also needed and can provide good data about whether a student is able to demonstrate understanding about the concepts and skills presented in that lesson or unit. This is very useful information for lesson planning, scaffolding for students, differentiating instruction, etc. However, this type of progress monitoring does not give us good information about whether our students are retaining and generalizing those skills over time to make progress toward long-term important key educational outcomes like becoming successful overall readers. You can lose the forest for the trees…
Long term monitoring of student progress allows the teacher to make decisions about whether the intervention is working over time- is the student becoming a better reader, writer, problem solver etc…
See next slide for another example of the difference between the two and why monitoring progress in general outcome measures 19

20. Both Mastery Monitoring and Long Term Progress Monitoring are Important
Sometimes mastering subskills doesn’t generalize to the general outcome or students don’t retain the information over time
For example:
Melissa is very good at decoding letters and reading individual words, but is not generalizing these skills to reading text with automaticity and comprehension.
Adam was really good at using his comprehension strategies and using those when they were working on these skills in class (showed mastery), but when they moved on to another unit he quit using the strategies.
Main point- mastery monitoring is great, but you need general outcome measures too.

21. How Often Do I Need to Monitor Progress?
Informally we do this all the time!
For small instructional adjustments (repeat the lesson, how much help to give, etc.)
In a standardized way to make sure we are “on track” with this student?- depends on level of concern
For students in reading who are behind already, monitor progress toward generalized outcome to 4x per month, ideally weekly
The facilitator might need to reiterate the difference between GOMs and Mastery Monitoring here so that the students keep in mind that looking at grades from class work does not constitute progress monitoring. 21

22. How Do You Collect Frequent Progress Monitoring Data?
Which students?
What measures?
What materials?
How often?
Who collects the data? Where? When?
Organizer slide 22

23. Which students? Students of concern Below target
Getting “extra” intervention or help
Tier 2 or Tier 3 services
Note: it is ok to start “practicing progress monitoring” by just picking a handful of students and collecting data. 23

24. What measures?
Web-based sources for information on measures as well as access to materials, web-based data management, etc.
dibels.uoregon.edu
Disclaimer: More data management systems are being developed and marketed all the time to provide information, resources, and data management. This is just a sample of the types of web-based resources available. Be a critical consumer!
If time and internet connect allow- could go online to show students what is available or give them an assignment to do this. 24

25. What materials?
When possible, students are monitored using grade level materials
E.g., student reads a different grade level passage or “probe” each week
If this is not possible due to frustration “test down” and use the highest grade level of materials possible
Periodically “check” how the student is doing on grade level materials and move into grade level materials as soon as possible
Instructional level may be different again!!!
(Note: Test “down” grade levels with 3 grade level passages at each grade and compare to the grade level targets of the materials. This is referred to as a survey level assessment or SLA)
In Math… SLA may look very different….

26. Example of Testing Down: Reading “Survey Level” Data for Gus, Gr 4
Note:
-Error rates high
(5-15) in Gr 4,
-slightly lower (4-8) in Gr 3 & 2,
-and much lower (1-4) in Gr 1.
Walk through what is on this slide… Gus was tested in grade level materials (gr 4) with 3 passages, then in Gr 3, then Gr2 and finally because he still wasn’t super successful, he was testing in Gr 1 where he finally read with few errors. He was tested over several days- not all at once! (Note: Testing “down” grade levels with 3 grade level passages and comparing to the grade level targets of the materials is referred to as a survey level assessment or SLA)
The graph also shows grade level “norms” for reading in this district.
Gus Doesn’t read much better in Gr 4, 3 or even 2…better in Gr 1
Errors disrupt meaning in Grades 4, 3, some in 2… not in Gr 1

27. What Material Should We Use…
To “instruct” Gus?
To monitor Gus’ progress?
Use this for class discussion- may choose to instruct Gus in Gr 1, 2 or perhaps even 3, but monitor probably in Grade 3, moving up to Grade 4 as soon as Gus could handle that without being frustrated.

28. How Often Do We Monitor Progress?
Depends on the sensitivity of the measure and the level of concern we have about the student, but 1 to 4x per month typically
For CBM Oral Reading Fluency
Weekly with 1 passage (this is most common)
Every 3 weeks with 3 passages
Typically schools do 1 passage a week and collect data weekly
However, some recent research (Jenkins, J, Graff, J., Miglioretti, D Exceptional Children 75(2), ) indicated 3 passages every 3 weeks may be an option 28

29. Who does it? When? Where?
Anyone trained in the procedures can collect progress monitoring data
Classroom teachers, special education teachers, Title teachers, aides, related services staff, adult volunteers
Be creative but careful
When and Where?
At a time and place that will provide valid information
Use common sense
Main point- this will depend on resources and physical space, but as long as people are correctly trained and the data are collected in a reasonable place/time it should be fine…. 29

30. Sharing the Data
Just having progress monitoring data is not enough. You need to USE it.
Scheduled graph review dates
Grade level meetings
Problem solving meetings
Without scheduled times and support for looking at data, it will not happen. 30

31. Using Progress Monitoring Data: Is this intervention working?
See MN RTI Center Module 2 for information on graphs, decision rules, etc
Module 2 has lots of graphs, activities, etc… 31

32. Remember: Garbage IN…. Garbage OUT….
Make sure your data have integrity or they won’t be good fer nuthin…
Training
Integrity checks/refreshers
Well chosen measures and materials
Good measures and reliable data collection lead to data you can use for decisions. 32

33. Avoid Common Mistakes Don’t use the same passage/probe every week! 
Have an organized system in place
Progress monitoring schedule for students
Preprinted passages/probes in a binder
An easy way to graph and look at the data
Scheduled time to share/look at the data 33

34. Remember: When teachers USE progress monitoring Students learn more!
Teachers design better instructional programs
Teacher decision making improves
Students become more aware of their performance Safer & Fleishman, 2005 34

35. Web Resources Research Institute on Progress Monitoring
Includes…
A Study Group Content Module with 15 sections on CBM including activities
Progress Monitoring Leadership Team Content Module with 6 sections (e.g. measureable goals, decision making) including activities
Handouts, videos, and power point presentations
Technical reports of CBM measures

36. Web Resources, Cont’d www.studentprogress.org
click on Progress monitoring on right side
look for information on CBM, graphing, etc.
dibels.uoregon.edu
Look for information about progress monitoring as well as access to materials and graphing for progress monitoring
DRAFT May 27, 2009
Research Matters: How Student Progress Monitoring Improves Instruction 36

37. Recommended Texts
Riley-Tillman & Burns. (2009). Evaluating Educational Interventions. Guilford Press.

38. Print Resources available with this module
Safer & Fleishman. (2005). How student progress monitoring improves instruction, Educational Leadership, 62(5),
Fuchs. Progress monitoring within a multi-level prevention system. Retrieved June 5, 2009, from RTI Action Network Web site:
Fuchs & Fuchs What is scientifically-based research on progress monitoring? From the National Center on Student Progress Monitoring (studentprogress.org). Retrieved June 14, 2009,
Jenkins, Hudson, & Hee Lee. Using CBM-Reading assessments to monitor progress. Retrieved June 5, 2009, from RTI Action Network Web site: CBM/1 38

39. Activity for Teachers or Practicum Students
Obtain progress monitoring probes and graphs
Passages and graphing materials self-created or downloaded
dibels.uoregon.edu
Sign up for an account with AIMSweb (instructor accounts and student accounts available)
Practice administration and scoring
Progress Monitor a “real” kid (ideally 2-4 kids of varying risk levels monitored for at least 7-10 weeks)
Graph, analyze, and use data

40. Quiz
1.) What are at least four reasons why teachers should monitor progress?
2) What is the difference between mastery monitoring and frequency progress monitoring toward a general outcome?
3) How often should you monitor progress for students receiving extra help ?
Answers:
1.) What are four reasons why teachers should monitor progress?
-students learn more
-teachers design better instructional programs
-teacher decision making improves
-students become more aware of their performance
-we do not know ahead of time whether an intervention will be successful for an individual student
-to change what you are doing with a student if it is not working so you are effective and efficient with your time and instruction
-to help make decisions about instructional goals, materials, levels, and groups
-to aid in communication with parents
-to document progress for special education students as required for periodic and annual reviews
-to celebrate success!
2) mastery monitoring can tell you whether a student is mastering or learning subskills or individual lessons over the short term, but does not test for retention and generalization of skills. General outcome progress monitoring tests regularly for progress toward an important more general outcome such as evidence the student is improving in overall reading skill over a long period of time.
3.) How often should you monitor progress for students receiving extra help?
1 to 4x per month, ideally weekly
DRAFT May 27, 2009

41. Quiz (Cont’d) 4.) When possible, students are monitored using…
A.) above grade level materials.
B.) grade level materials.
C.) below grade level materials.
5.) Who can collect progress monitoring data?
Answers (cont’d)
4.) When possible, students are monitored using…
B.) grade level materials.
5.) Who can collect progress monitoring data?
anyone trained in the procedures, e.g. classroom teachers, special education teachers, Title teachers, aids, related services staff, adult volunteers

42. Quiz (Cont’d)
6.) To have integrity, your data collection must include what?
A.) trained data collectors
B.) integrity checks/refreshers
C.) well chosen measures and materials
D.) all of the above
Answers (cont’d)
6.) To have integrity, your data must have what?
D.) all of the above

43. Quiz (Cont’d)
7.) True or False? The most important use of frequent progress monitoring is to aid in communication with parents.
Answers (Cont’d)
7.) True or False? The most important use of frequent progress monitoring is to aid in communication with parents.
-False- the most important reason is to change what you are doing with a student if what you are doing is not working so you are effective and efficient with your time and instruction

44. Note: The MN RTI Center does not endorse any particular product
Note: The MN RTI Center does not endorse any particular product. Examples used are for instructional purposes only.
Special Thanks:
Thank you to Dr. Ann Casey, director of the MN RTI Center, for her leadership
Thank you to Aimee Hochstein, Kristen Bouwman, and Nathan Rowe, Minnesota State University Moorhead graduate students, for editing, writing quizzes, and enhancing the quality of these training materials