Presentation is loading. Please wait.

Presentation is loading. Please wait.

Nuts and Bolts of Progress Monitoring

Similar presentations


Presentation on theme: "Nuts and Bolts of Progress Monitoring"— Presentation transcript:

1 Nuts and Bolts of Progress Monitoring
Laura Boynton Hauerwas Kristen Matthes May 31, 2006

2 Topics for the morning Progress Monitoring Assessment: Sample PM tools
What? Why? Sample PM tools Graphing Data Data-Base Decisions Setting Goals Performance Level Rate of Learning Linking cases to RI RTI processes

3 Assessment in a RTI model
Benchmarking To screen and identify students who are at-risk and in need of interventions All students Three times a year All areas At grade-level Progress Monitoring To monitor progress of individual students and determine rate of improvement and need for adaptation of intervention Students who are not achieving benchmarks (PLP, IEP) Weekly, biweekly, monthly assessments In area of need At instructional level

4 What assessments do you use?
Reading Math Social-Emotional Behavior What measures do you use? Who do you assess? When do you assess? How is the information used? Benchmark/ Progress Monitoring

5 Progress Monitoring Benefits of Progress Monitoring
Parents and students know what is expected Teachers know what is working or not working with their instruction based on data Easy to understand way to show parents progress Teams have comprehensive data on student performance for decision making

6 Progress Monitoring CBMs Are assessments to monitor progress
Are designed to serve as “indicators” of general reading achievement. R-CBM doesn’t measure everything, but measures the important things. Are Standardized tests to be given, scored, and interpreted in a standard way. Are researched with respect to psychometric properties to ensure accurate measures of learning. Are Sensitive to improvement in Short Periods of time. Designed to be as short as possible to ensure its “do ability.” Are linked to decision making for promoting positive achievement and Problem-Solving

7 Progress Monitoring National Center on Student Progress Monitoring

8 Samples of CBMs Reading Math Writing Spelling

9

10 MAZE - CBM AIMSweb Reading Comprehension Measure

11 DIBELS Phoneme Segmentation Fluency
https://dibels.uoregon.edu

12 MATH COMPUTATION Taken from Fuchs, L. S., Hamlett, C. A., & Fuchs, D. (1998). Monitoring Basic Skills Progress: Basic Math Computation (2nd ed.). [computer program]. Austin, TX: ProEd. Available: from

13 Concepts and Applications
Sample page from a three-page test for Grade 2 Math Concepts and Applications From Monitoring Basic Skills Progress

14 CBM - Writing Total Words Written Correct Word Sequences
Words Correctly Spelled

15 Spelling AIMSweb Spelling Probes

16

17

18 Graphing Graphing is an essential part of PM
Without graphic displays, the decision making process is difficult Teacher graphing vs. Student graphing

19 How to Develop Graphs Hand Graphing Excel and Chart Dog
Graphing Data: Beetle, SUV, Race Car Hand Graphing Excel and Chart Dog Web-based data systems

20 Hand Graphing Establish Baseline (Median score) Set up graph Set Goal
Draw Aimline Measure Student Progress Plot Student Performance Connect Indicators of Student Performance Analyze Student Performance Make Instructional Changes Continue to Measure and Monitor Student Performance

21 Hand Graphing Goal 44

22 Hand Graphing 50 45 40 35 30 Testing Sessions
Number of Words Read Correctly 35 30 Baseline Session 1 Session 2 Session 3 Session 4 Session 5 Session 6 Session 7 Session 8 Testing Sessions

23 Hand Graphing Advantages Easy to do No technology required
Students can easily maintain their own graphs Can be done immediately Free Disadvantages Added paper Organization required No long-term storage Not automatic

24 Excel Excel Template Website to access Excel Templates
To add trend line: Select Data Points Right Click …Add Trend Line Choose Linear, To find slope –Options Add Equation to the chart (y= slopex+ intercept)

25 Chart Dog Tool at

26 Excel and Chart Dog Advantage Automatic Storage capability built-in
Easy to do Clear displays of data Free if you have EXCEL, Chart Dog is free Disadvantage Requires technology Time to enter data Students may not be able to do data entry themselves Requires some understanding of EXCEL or Chart Dog

27 Web Based System

28 Web-Based System Advantage Web based data entry from anywhere
Storage capability built-in Trend line drawn automatically Can annotate graphs interventions/goals Norms –benchmarks and Rate of Improvement Lots of flexibility graphs Disadvantage Requires technology Cost Students may not be able to do data entry themselves Requires some training

29 BREAK To do this will take new learning for everyone

30 Data-Based Decisions Performance Level Rate of Learning
Gaps in Performance PLP Not at Grade Level Special Education Significant Discrepancy Rate of Learning Trend in performance (slope) Response to Instruction General Direction, Rate of Change

31 Performance Level: Gap/ Discrepancy
Be objective. Does it refer to an observable/measurable characteristic of behavior? Use numbers to define the discrepancy. Percentile rank Discrepancy Ratios Cut scores Norms

32 Norms… What to use? Local, National
Local norms can be helpful to determine local performance levels and rates of progress Time consuming and costly to develop National norms and research norms are available. BUT….

33 1. Performance Level Percentile Ranks
Requires a Larger Normative Data Base, Preferably Benchmark Data < 25th At Risk, Consider Problem-Solving at the Group Level <10th Potential Severe Problem, Consider Individual Problem Solving

34 Performance Level Discrepancy Ratios
Sample 5-7 Students or Whole Class, Grade Figure Median and Graph Divide by 2 and Graph Students Who Performance Below the Line May Need Problem Solving

35 = Discrepancy of 3.6x Can Compute… Peer Median Target Student Median
145 40 = Discrepancy of 3.6x

36 Performance Level Cut Scores
A number which represents the point at which scores can be divided into different groups (for example does not meet, meets, and exceeds expectations) for decision-making purposes. May be based on research (e.g., a correlation between scoring at or above a certain level on a CBM or DIBELS task and future academic success) or expectation (e.g., grades at C or above, no more than 3 office referrals).

37 Data-base decisions on performance level
enables team to make decisions about levels of support and resource from the start. Generally speaking… - A student who is 1.5x discrepant from his/her peers may benefit from intensive group interventions. A student who is 2-2.5x discrepant from his/her peers is appropriate for individualized problem-solving and intensive intervention resources may be appropriate. Example: Jessica is 2.1x discrepant from peers on the Math CBM and may benefit from intensive interventions in math.

38 Rebecca 2nd grader List all areas of concern:
Off-task behavior Reading difficulties Poor handwriting Identify primary area of concern and define it in observable and measurable terms: Reading Definition: number words read correctly when reading a grade level passage orally Collect baseline data on primary area of concern and state discrepancy statement: Baseline data collected in the area of test from CBM reading probes Discrepancy Statement: Rebecca reads 41 WRC per minute in Fall of 2nd grade while her peers read ____ WRC per minute ___________ _______________________________________________________

39 Rob 7th grader List all areas of concern:
Calling out Lack of homework completion Poor handwriting Identify primary area of concern and define it in observable and measurable terms: Work Completion Definition: Turning in teacher assigned work at beginning of class period on the day that it is due. Collect baseline data on primary area of concern and state discrepancy statement: Baseline data collected in the area of review from teacher grade books Discrepancy Statement: Rob currently turns in homework 54% of the time while his peers turn in homework 86% of the time (_____ discrepant)

40 Data-Based Decisions Performance Level Rate of Learning
Gaps in Performance PLP Not at Grade Level Special Education Significant Discrepancy Rate of Learning Trend in performance (slope) Response to Instruction General Direction, Rate of Change

41 2. Rate of Learning Why? Determine when what we are doing isn’t working and intervene early Better able to predict student success at meeting goals Better able to identify who needs more intensive instruction

42 Tracking Student Outcomes Using Initial Performance Discrepancies
Rate of Learning Tracking Student Outcomes Using Initial Performance Discrepancies Name Grade Area Initial Performance Discrepancy Follow Up Performance Discrepancy Outcome Decision Bill 3 Reading 10th percentile 20th percentile Satisfactory; Maintain Intervention Susie 2 Math 1.2x NA No Severe Problem Rob 4 Homework 65% completion 64% completion No Progress, Problem solve and Modify intervention plan

43 Rate of Learning Analyzing Rate using PM Data Rules: Setting Goals
Data Point Rules Trend Line Rules Slope

44 Setting Goals End of the Year Benchmarks
GLEs for Reading Fluency (2nd grade WPM, 5th grade ) AIMSweb Math Computation Norms (1st grade 17 DPM, 5th grade 52 DPM) 2. National Norms for Improvement Math Calculations (>.3 DPM 2nd and 3rd grade, >.5 DPM 4-6th grade) (Fuchs, 2006) Reading Fluency (Deno, 2005) Grade Modest Reasonable Ambitious 1-2 1 Word Per Week 1.5 Word Per Week 2.0 Word Per Week 3-6 .5 Word Per Week 1.0 Word Per Week

45 Setting Goals 3. Individual ROI
Weekly rate of improvement in “baseline slope” calculated from 8 data points (Slope: Difference of highest and lowest/#weeks) Baseline multiplied by 1.5 Product multiplied by number of weeks until end of year Add to student’s final baseline score to produce end of year goal. Baseline Reading scores: 52, 54, 52, 53, 55, 58, 55, 56 Difference: =5 Divide by number of weeks: 5/8 =.625 (SLOPE) Baseline multiplied by 1.5: × 1.5 = .9375 Number of weeks left (6 weeks): ×6 =5.6 Add to final baseline score: = 61.6 End of the year goal 62

46 Jim – 5th grader What goal would you set
for Jim in math for the end of year? Jim – 5th grader Math Calculation December-January Monitoring 36, 37, 36, 36, 37, 38, 39, 37 January Benchmark 38 Base determination on 18 weeks left in the year

47 Jim – 5th grader What goal would you set
for Jim in math for the end of year? Jim – 5th grader End of Year Benchmarks DPM (.77 ROI) National Norms DPM (.5 ROI) (38+18*.5) Individual ROI DPM (.56 ROI) 3/8*1.5 =.56 ROI 38 + (18*.56)

48 Decisions based on data-points
Decisions are based on at least 4 data points If all 4 scores fall above goal-line, responding to instruction (increase goal if continues for 4 more data points) If scores are hovering about the goal line, continue what you are doing. If all 4 scores are below goal-line, but parallel, decide to “wait” for 4 more points to see if student performance accelerates in level to reach original goal. If all 4 scores fall below goal-line, not responding to instruction, revise plan and implement different teaching strategy. Mark change on graph with vertical line. Derived from: Fuchs and Fuchs (2006) and Shapiro (2006)

49

50 Decisions based on trend lines
Trend lines based on 6-8 data-points If trend line is steeper than goal line, increase the goal. If trend line is flatter than goal line, revise instruction If trend line equals goal line, make no change at this time.

51

52 Data-base decision on slope
Rate of Improvement = slope or r(in statistics) Y = slopeX + intercept Consider discrepancy from ROI norms

53

54 Is Jim responding to the intervention?

55 Is Jim Responding to the Math Intervention?
Checkpoint: Is Jim Responding to the Math Intervention? Are modifications necessary? End of Year: What would you say about Jim’s Math?

56 How can you use the literacy, math and social-emotional/behavior assessments that you have? Do they provide information about a student’s performance level rate of learning area of instructional need?

57 Your chart might include one of these literacy measures?
R-CBM Rigby DRA Performance Level YES YES? Rate of Learning NO Instructional needs

58 What is the difference between data-driven and data informing?

59 Rebecca- 2nd grader Fall benchmark some concern Instructional Plan
41 WPM, <25th percentile, 1.96X Discrepant Instructional Plan small group guided reading book bag for home with leveled readers

60 Rebecca- 2nd grader

61 What additional data is needed to modify interventions and meet Rebecca’s needs?
Possible Hypotheses Review Interview Observe Test Rebecca reads 41 WRC per minute in the Fall of 2nd grade while her peers read ____WRC per minute and with small group guided reading limited rate of improvement because …

62 Case Example: Rebecca Possible Hypotheses Review Interview Observe
Rebecca reads 41 WRC per minute in the Fall of 2nd grade while her peers read 80 WRC per minute and with small group guided reading limited rate of improvement because … Possible Hypotheses Review Interview Observe Test She had insufficient opportunity to practice Interview Teacher/Parent Observe during SSR CBM She lacks the phonological skills to read the text fluently. Review her Records R-CBM scores Phonological Assessment Instructional materials are above her reading level. Review her Records/CBM scores CBM survey level assessment

63 Case Example: Rebecca Predicted Hypotheses Review Interview Observe
Test She had insufficient opportunity to practice Instructional Planning Form revealed Rebecca had 15 minutes of independent reading practice per day Observation revealed that during SSR, Rebecca often played with items on desk. CBM probes indicated that she was not performing comparable to peers She lacks the phonological skills to read the text fluently. Review of records indicated that she had done well in remedial phonics program the previous year and was graduated out of the program. IPF suggested that instructional time was given to phonological skills. CBM probes indicated that she was not performing comparable to peers; DIBELS assessments did not point to a phonological awareness or phonics problem. Her reading level is not instructionally appropriate. Teacher reports showed that she was at the 25th percentile in the Spring of 1st grade Teacher felt that curriculum was appropriate Survey level assessment revealed that Rebecca had mastered 1st grade text

64 Case Example: Rebecca Rebecca reads 41 WRC per minute in the Fall of 2nd grade while her peers read 80 WRC per minute and is improving at a limited rate (.5 ROI) because … She had insufficient opportunity to practice.

65 What was the impact of progress monitoring assessments for Rebecca and her teachers?

66 Rebecca’s progress during Reader’s Theatre

67 3 - 2 - 1 3 things you learned today
3 things you learned today 2 things you still have questions about 1 statement about how you feel about PM now

68 We will have tough choices to make – we’ll decide based on what’s best for our kids

69 If you’re not hopelessly confused, you’re out of touch
If you’re not hopelessly confused, you’re out of touch! If you are hopelessly confused, then you only have one choice — try stuff Tom Peters, Embracing Chaos, 1993

70 We will make mistakes – and fix them along the way

71 Thank You! Questions: Laura Hauerwas Kristen Matthes

72 References Deno, S., Lembke, E. and Anderson, A. (2005) Progress Monitoring Study Group Content Module available at Fuchs and Fuchs (April, 2006) Progress Monitoring: Identifying LD and Improving Student Outcomes. Presentation at National SEA Conference on SLD Determination. Kansas City, MO. Hawkins, and Tilly, D. (Sept. 2005). Response to Intervention On the Ground: Diagnosing the Learning Enabled. Pacific Northwest Institute on Special Education and the Law. Shapiro, E. (2006). Advanced Issues in Monitoring Student Progress in Reading. Presentation for Pennsylvania Training and Technical Assistance Network on on RTI. Tilly (Sept, 2005) Problem Identification and Analysis. Presentation at the Principal Lead Problem Solving Initiative, Highland Park, IL.


Download ppt "Nuts and Bolts of Progress Monitoring"

Similar presentations


Ads by Google