1. Nuts and Bolts of Progress Monitoring
Laura Boynton Hauerwas
Kristen Matthes
May 31, 2006

2. Topics for the morning Progress Monitoring Assessment: Sample PM tools
What?
Why?
Sample PM tools
Graphing Data
Data-Base Decisions
Setting Goals
Performance Level
Rate of Learning
Linking cases to RI RTI processes

3. Assessment in a RTI model
Benchmarking
To screen and identify students who are at-risk and in need of interventions
All students
Three times a year
All areas
At grade-level
Progress Monitoring
To monitor progress of individual students and determine rate of improvement and need for adaptation of intervention
Students who are not achieving benchmarks (PLP, IEP)
Weekly, biweekly, monthly assessments
In area of need
At instructional level

4. What assessments do you use?
Reading
Math
Social-Emotional
Behavior
What measures do you use?
Who do you assess?
When do you assess?
How is the information used? Benchmark/
Progress Monitoring

5. Progress Monitoring Benefits of Progress Monitoring
Parents and students know what is expected
Teachers know what is working or not working with their instruction based on data
Easy to understand way to show parents progress
Teams have comprehensive data on student performance for decision making

6. Progress Monitoring CBMs Are assessments to monitor progress
Are designed to serve as “indicators” of general reading achievement. R-CBM doesn’t measure everything, but measures the important things.
Are Standardized tests to be given, scored, and interpreted in a standard way.
Are researched with respect to psychometric properties to ensure accurate measures of learning.
Are Sensitive to improvement in Short Periods of time.
Designed to be as short as possible to ensure its “do ability.”
Are linked to decision making for promoting positive achievement and Problem-Solving

7. Progress Monitoring
National Center on Student Progress Monitoring

8. Samples of CBMs
Reading
Math
Writing
Spelling

10. MAZE - CBM
AIMSweb
Reading Comprehension Measure

11. DIBELS Phoneme Segmentation Fluency

12. MATH COMPUTATION
Taken from Fuchs, L. S., Hamlett, C. A., & Fuchs, D. (1998). Monitoring Basic Skills Progress: Basic Math Computation (2nd ed.). [computer program]. Austin, TX: ProEd.
Available: from

13. Concepts and Applications
Sample page from a three-page test for Grade 2 Math Concepts and Applications
From Monitoring Basic Skills Progress

14. CBM - Writing Total Words Written Correct Word Sequences
Words Correctly Spelled

15. Spelling
AIMSweb Spelling Probes

18. Graphing Graphing is an essential part of PM
Without graphic displays, the decision making process is difficult
Teacher graphing vs. Student graphing

19. How to Develop Graphs Hand Graphing Excel and Chart Dog
Graphing Data: Beetle, SUV, Race Car
Hand Graphing
Excel and Chart Dog
Web-based data systems

20. Hand Graphing Establish Baseline (Median score) Set up graph Set Goal
Draw Aimline
Measure Student Progress
Plot Student Performance
Connect Indicators of Student Performance
Analyze Student Performance
Make Instructional Changes
Continue to Measure and Monitor Student Performance

21. Hand Graphing
Goal 44

22. Hand Graphing 50 45 40 35 30 Testing Sessions
Number of Words Read Correctly 35 30
Baseline
Session 1
Session 2
Session 3
Session 4
Session 5
Session 6
Session 7
Session 8
Testing Sessions

23. Hand Graphing Advantages Easy to do No technology required
Students can easily maintain their own graphs
Can be done immediately
Free
Disadvantages
Added paper
Organization required
No long-term storage
Not automatic

24. Excel Excel Template Website to access Excel Templates
To add trend line:
Select Data Points
Right Click …Add Trend Line
Choose Linear,
To find slope –Options Add Equation to the chart (y= slopex+ intercept)

25. Chart Dog
Tool at

26. Excel and Chart Dog Advantage Automatic Storage capability built-in
Easy to do
Clear displays of data
Free if you have EXCEL, Chart Dog is free
Disadvantage
Requires technology
Time to enter data
Students may not be able to do data entry themselves
Requires some understanding of EXCEL or Chart Dog

27. Web Based System

28. Web-Based System Advantage Web based data entry from anywhere
Storage capability built-in
Trend line drawn automatically
Can annotate graphs interventions/goals
Norms –benchmarks and Rate of Improvement
Lots of flexibility
graphs
Disadvantage
Requires technology
Cost
Students may not be able to do data entry themselves
Requires some training

29. BREAK
To do this will take new
learning for everyone

30. Data-Based Decisions Performance Level Rate of Learning
Gaps in Performance
PLP Not at Grade Level
Special Education Significant Discrepancy
Rate of Learning
Trend in performance (slope)
Response to Instruction
General Direction, Rate of Change

31. Performance Level: Gap/ Discrepancy
Be objective. Does it refer to an observable/measurable characteristic of behavior?
Use numbers to define the discrepancy.
Percentile rank
Discrepancy Ratios
Cut scores
Norms

32. Norms… What to use? Local, National
Local norms can be helpful to determine local performance levels and rates of progress
Time consuming and costly to develop
National norms and research norms are available.
BUT….

33. 1. Performance Level Percentile Ranks
Requires a Larger Normative Data Base, Preferably Benchmark Data
< 25th At Risk, Consider Problem-Solving at the Group Level
<10th Potential Severe Problem, Consider Individual Problem Solving

34. Performance Level Discrepancy Ratios
Sample 5-7 Students or Whole Class, Grade
Figure Median and Graph
Divide by 2 and Graph
Students Who Performance Below the Line May Need Problem Solving

35. = Discrepancy of 3.6x Can Compute… Peer Median Target Student Median
145 40
= Discrepancy of 3.6x

36. Performance Level Cut Scores
A number which represents the point at which scores can be divided into different groups (for example does not meet, meets, and exceeds expectations) for decision-making purposes.
May be based on research (e.g., a correlation between scoring at or above a certain level on a CBM or DIBELS task and future academic success) or expectation (e.g., grades at C or above, no more than 3 office referrals).

37. Data-base decisions on performance level
enables team to make decisions about
levels of support and resource from the start.
Generally speaking…
- A student who is 1.5x discrepant from his/her peers may benefit from intensive group interventions.
A student who is 2-2.5x discrepant from his/her peers is appropriate for individualized problem-solving and intensive intervention resources may be appropriate.
Example: Jessica is 2.1x discrepant from peers
on the Math CBM and may benefit from intensive interventions in math.

38. Rebecca 2nd grader List all areas of concern:
Off-task behavior
Reading difficulties
Poor handwriting
Identify primary area of concern and define it in observable and measurable terms:
Reading
Definition: number words read correctly when reading a grade level passage orally
Collect baseline data on primary area of concern and state discrepancy statement:
Baseline data collected in the area of test from CBM reading probes
Discrepancy Statement: Rebecca reads 41 WRC per minute in Fall of 2nd grade while her peers read ____ WRC per minute ___________
_______________________________________________________

39. Rob 7th grader List all areas of concern:
Calling out
Lack of homework completion
Poor handwriting
Identify primary area of concern and define it in observable and measurable terms:
Work Completion
Definition: Turning in teacher assigned work at beginning of class period on the day that it is due.
Collect baseline data on primary area of concern and state discrepancy statement:
Baseline data collected in the area of review from teacher grade books
Discrepancy Statement: Rob currently turns in homework 54% of the time while his peers turn in homework 86% of the time (_____ discrepant)

40. Data-Based Decisions Performance Level Rate of Learning
Gaps in Performance
PLP Not at Grade Level
Special Education Significant Discrepancy
Rate of Learning
Trend in performance (slope)
Response to Instruction
General Direction, Rate of Change

41. 2. Rate of Learning
Why?
Determine when what we are doing isn’t working and intervene early
Better able to predict student success at meeting goals
Better able to identify who needs more intensive instruction

42. Tracking Student Outcomes Using Initial Performance Discrepancies
Rate of Learning
Tracking Student Outcomes Using Initial Performance Discrepancies
Name
Grade
Area
Initial Performance Discrepancy
Follow Up Performance Discrepancy
Outcome Decision
Bill 3
Reading
10th percentile
20th percentile
Satisfactory; Maintain Intervention
Susie 2
Math
1.2x NA
No Severe Problem
Rob 4
Homework
65% completion
64% completion
No Progress, Problem solve and Modify intervention plan

43. Rate of Learning Analyzing Rate using PM Data Rules: Setting Goals
Data Point Rules
Trend Line Rules
Slope

44. Setting Goals End of the Year Benchmarks
GLEs for Reading Fluency (2nd grade WPM, 5th grade )
AIMSweb Math Computation Norms (1st grade 17 DPM, 5th grade 52 DPM)
2. National Norms for Improvement
Math Calculations (>.3 DPM 2nd and 3rd grade, >.5 DPM 4-6th grade)
(Fuchs, 2006)
Reading Fluency (Deno, 2005)
Grade
Modest
Reasonable
Ambitious
1-2
1 Word Per Week
1.5 Word Per Week
2.0 Word Per Week
3-6
.5 Word Per Week
1.0 Word Per Week

45. Setting Goals 3. Individual ROI
Weekly rate of improvement in “baseline slope” calculated from 8 data points (Slope: Difference of highest and lowest/#weeks)
Baseline multiplied by 1.5
Product multiplied by number of weeks until end of year
Add to student’s final baseline score to produce end of year goal.
Baseline Reading scores: 52, 54, 52, 53, 55, 58, 55, 56
Difference: =5
Divide by number of weeks: 5/8 =.625 (SLOPE)
Baseline multiplied by 1.5: × 1.5 = .9375
Number of weeks left (6 weeks): ×6 =5.6
Add to final baseline score: = 61.6
End of the year goal 62

46. Jim – 5th grader What goal would you set
for Jim in math for the end of year?
Jim – 5th grader
Math Calculation
December-January Monitoring
36, 37, 36, 36, 37, 38, 39, 37
January Benchmark 38
Base determination on 18 weeks left in the year

47. Jim – 5th grader What goal would you set
for Jim in math for the end of year?
Jim – 5th grader
End of Year Benchmarks DPM (.77 ROI)
National Norms DPM (.5 ROI)
(38+18*.5)
Individual ROI DPM (.56 ROI)
3/8*1.5 =.56 ROI
38 + (18*.56)

48. Decisions based on data-points
Decisions are based on at least 4 data points
If all 4 scores fall above goal-line, responding to instruction (increase goal if continues for 4 more data points)
If scores are hovering about the goal line, continue what you are doing.
If all 4 scores are below goal-line, but parallel, decide to “wait” for 4 more points to see if student performance accelerates in level to reach original goal.
If all 4 scores fall below goal-line, not responding to instruction, revise plan and implement different teaching strategy.
Mark change on graph with vertical line.
Derived from: Fuchs and Fuchs (2006) and Shapiro (2006)

50. Decisions based on trend lines
Trend lines based on 6-8 data-points
If trend line is steeper than goal line, increase the goal.
If trend line is flatter than goal line, revise instruction
If trend line equals goal line, make no change at this time.

52. Data-base decision on slope
Rate of Improvement = slope or r(in statistics)
Y = slopeX + intercept
Consider discrepancy from ROI norms

54. Is Jim responding
to the intervention?

55. Is Jim Responding to the Math Intervention?
Checkpoint:
Is Jim Responding to the Math Intervention?
Are modifications necessary?
End of Year:
What would you say about Jim’s Math?

56. How can you use the literacy, math and social-emotional/behavior assessments that you have? Do they provide information about
a student’s performance level
rate of learning
area of instructional need?

57. Your chart might include one of these literacy measures?
R-CBM
Rigby
DRA
Performance Level
YES
YES?
Rate of Learning NO
Instructional needs

58. What is the difference between data-driven and data informing?

59. Rebecca- 2nd grader Fall benchmark some concern Instructional Plan
41 WPM, <25th percentile, 1.96X Discrepant
Instructional Plan
small group guided reading
book bag for home with leveled readers

60. Rebecca- 2nd grader

61. What additional data is needed to modify interventions and meet Rebecca’s needs?
Possible
Hypotheses
Review
Interview
Observe
Test
Rebecca reads 41 WRC per minute in the Fall of 2nd grade while her peers read ____WRC per minute and with small group guided reading limited rate of improvement because …

62. Case Example: Rebecca Possible Hypotheses Review Interview Observe
Rebecca reads 41 WRC per minute in the Fall of 2nd grade while her peers read
80 WRC per minute and with small group guided reading limited rate of improvement because …
Possible
Hypotheses
Review
Interview
Observe
Test
She had insufficient opportunity to practice
Interview Teacher/Parent
Observe during SSR
CBM
She lacks the phonological skills to read the text fluently.
Review her Records
R-CBM scores
Phonological Assessment
Instructional materials are above her reading level.
Review her Records/CBM scores
CBM survey level assessment

63. Case Example: Rebecca Predicted Hypotheses Review Interview Observe
Test
She had insufficient opportunity to practice
Instructional Planning Form revealed Rebecca had 15 minutes of independent reading practice per day
Observation revealed that during SSR, Rebecca often played with items on desk.
CBM probes indicated that she was not performing comparable to peers
She lacks the phonological skills to read the text fluently.
Review of records indicated that she had done well in remedial phonics program the previous year and was graduated out of the program.
IPF suggested that instructional time was given to phonological skills.
CBM probes indicated that she was not performing comparable to peers; DIBELS assessments did not point to a phonological awareness or phonics problem.
Her reading level is not instructionally appropriate.
Teacher reports showed that she was at the 25th percentile in the Spring of 1st grade
Teacher felt that curriculum was appropriate
Survey level assessment revealed that Rebecca had mastered 1st grade text

64. Case Example: Rebecca
Rebecca reads 41 WRC per minute in the Fall of 2nd grade while her peers read 80 WRC per minute and is improving at a limited rate (.5 ROI) because …
She had insufficient opportunity to practice.

65. What was the impact of progress monitoring assessments for Rebecca and her teachers?

66. Rebecca’s progress during Reader’s Theatre

67. 3 - 2 - 1 3 things you learned today
3 things you learned today
2 things you still have questions about
1 statement about how you feel about PM now

68. We will have tough choices to make – we’ll decide based on what’s best for our kids

69. If you’re not hopelessly confused, you’re out of touch
If you’re not hopelessly confused, you’re out of touch! If you are hopelessly confused, then you only have one choice — try stuff Tom Peters, Embracing Chaos, 1993

70. We will make mistakes – and fix them along the way

71. Thank You!
Questions: Laura Hauerwas Kristen Matthes

72. References
Deno, S., Lembke, E. and Anderson, A. (2005) Progress Monitoring Study Group Content Module available at
Fuchs and Fuchs (April, 2006) Progress Monitoring: Identifying LD and Improving Student Outcomes. Presentation at National SEA Conference on SLD Determination. Kansas City, MO.
Hawkins, and Tilly, D. (Sept. 2005). Response to Intervention On the Ground: Diagnosing the Learning Enabled. Pacific Northwest Institute on Special Education and the Law.
Shapiro, E. (2006). Advanced Issues in Monitoring Student Progress in Reading. Presentation for Pennsylvania Training and Technical Assistance Network on on RTI.
Tilly (Sept, 2005) Problem Identification and Analysis. Presentation at the Principal Lead Problem Solving Initiative, Highland Park, IL.