Presentation is loading. Please wait.

Presentation is loading. Please wait.

Nuts and Bolts of Progress Monitoring Laura Boynton Hauerwas Kristen Matthes May 31, 2006.

Similar presentations


Presentation on theme: "Nuts and Bolts of Progress Monitoring Laura Boynton Hauerwas Kristen Matthes May 31, 2006."— Presentation transcript:

1 Nuts and Bolts of Progress Monitoring Laura Boynton Hauerwas Kristen Matthes May 31, 2006

2 Topics for the morning 1.Progress Monitoring Assessment: What? Why? 2.Sample PM tools 3.Graphing Data 4.Data-Base Decisions Setting Goals Performance Level Rate of Learning 5.Linking cases to RI RTI processes

3 Assessment in a RTI model Benchmarking To screen and identify students who are at-risk and in need of interventions All students Three times a year All areas At grade-level Progress Monitoring To monitor progress of individual students and determine rate of improvement and need for adaptation of intervention Students who are not achieving benchmarks (PLP, IEP) Weekly, biweekly, monthly assessments In area of need At instructional level

4 What assessments do you use? ReadingMathSocial-Emotional Behavior What measures do you use? Who do you assess? When do you assess? How is the information used? Benchmark/ Progress Monitoring

5 Benefits of Progress Monitoring Parents and students know what is expected Teachers know what is working or not working with their instruction based on data Easy to understand way to show parents progress Teams have comprehensive data on student performance for decision making

6 Progress Monitoring CBMs Are assessments to monitor progress indicators importantAre designed to serve as indicators of general reading achievement. R- CBM doesnt measure everything, but measures the important things. Standardized tests standard way.Are Standardized tests to be given, scored, and interpreted in a standard way. researchedAre researched with respect to psychometric properties to ensure accurate measures of learning. SensitiveShort PeriodsAre Sensitive to improvement in Short Periods of time. as short as possibledo ability.Designed to be as short as possible to ensure its do ability. linked to decision makingAre linked to decision making for promoting positive achievement and Problem-Solving

7 Progress Monitoring National Center on Student Progress Monitoring

8 Samples of CBMs Reading Math Writing Spelling

9

10 MAZE - CBM AIMSweb Reading Comprehension Measure

11 DIBELS Phoneme Segmentation Fluency https://dibels.uoregon.edu

12 Taken from Fuchs, L. S., Hamlett, C. A., & Fuchs, D. (1998). Monitoring Basic Skills Progress: Basic Math Computation (2nd ed.). [computer program]. Austin, TX: ProEd. Available: from edinc.com MATH COMPUTATION

13 Concepts and Applications Sample page from a three- page test for Grade 2 Math Concepts and Applications –From Monitoring Basic Skills Progress

14 CBM - Writing Total Words Written Correct Word Sequences Words Correctly Spelled

15 Spelling AIMSweb Spelling Probes

16

17

18 Graphing Graphing is an essential part of PM Without graphic displays, the decision making process is difficult Teacher graphing vs. Student graphing

19 How to Develop Graphs Graphing Data: Beetle, SUV, Race Car Hand Graphing Excel and Chart Dog Web-based data systems

20 Hand Graphing 1.Establish Baseline (Median score) 2.Set up graph 3.Set Goal 4.Draw Aimline 5.Measure Student Progress 6.Plot Student Performance 7.Connect Indicators of Student Performance 8.Analyze Student Performance 9.Make Instructional Changes 10.Continue to Measure and Monitor Student Performance

21 Hand Graphing Goal 44

22 Hand Graphing Testing Sessions BaselineSession 1 Session 3 Session 2 Session 4 Session 6 Session 5 Session 7Session 8 Number of Words Read Correctly

23 Hand Graphing Advantages Easy to do No technology required Students can easily maintain their own graphs Can be done immediately Free Disadvantages Added paper Organization required No long-term storage Not automatic

24 Excel Excel Template Website to access Excel Templates To add trend line: 1.Select Data Points 2.Right Click …Add Trend Line 3.Choose Linear, 4.To find slope –Options Add Equation to the chart (y= slopex+ intercept)

25 Chart Dog Tool at

26 Excel and Chart Dog Advantage Automatic Storage capability built-in Easy to do Clear displays of data Free if you have EXCEL, Chart Dog is free Disadvantage Requires technology Time to enter data Students may not be able to do data entry themselves Requires some understanding of EXCEL or Chart Dog

27 Web Based System

28 Web-Based System Advantage Web based data entry from anywhere Storage capability built-in Trend line drawn automatically Can annotate graphs interventions/goals Norms –benchmarks and Rate of Improvement Lots of flexibility graphs Disadvantage Requires technology Cost Students may not be able to do data entry themselves Requires some training

29 To do this will take new learning for everyone BREAK

30 Data-Based Decisions 1.Performance Level Gaps in Performance PLP Not at Grade Level Special Education Significant Discrepancy 2.Rate of Learning Trend in performance (slope) Response to Instruction General Direction, Rate of Change

31 1.Performance Level: Gap/ Discrepancy Be objective. Does it refer to an observable/measurable characteristic of behavior? Use numbers to define the discrepancy. –Percentile rank –Discrepancy Ratios –Cut scores Norms

32 Norms… What to use? Local, National Local norms can be helpful to determine local performance levels and rates of progress Time consuming and costly to develop National norms and research norms are available. BUT….

33 Percentile Ranks Requires a Larger Normative Data Base, Preferably Benchmark Data < 25th At Risk, Consider Problem-Solving at the Group Level <10th Potential Severe Problem, Consider Individual Problem Solving 1. Performance Level

34 Discrepancy Ratios 1.Sample 5-7 Students or Whole Class, Grade 2.Figure Median and Graph 3.Divide by 2 and Graph 4.Students Who Performance Below the Line May Need Problem Solving Performance Level

35 Can Compute… Peer Median Target Student Median = Discrepancy of 3.6x

36 Cut Scores A number which represents the point at which scores can be divided into different groups (for example does not meet, meets, and exceeds expectations) for decision-making purposes. May be based on research (e.g., a correlation between scoring at or above a certain level on a CBM or DIBELS task and future academic success) or expectation (e.g., grades at C or above, no more than 3 office referrals). Performance Level

37 Generally speaking… -A student who is 1.5x discrepant from his/her peers may benefit from intensive group interventions. –A student who is 2-2.5x discrepant from his/her peers is appropriate for individualized problem- solving and intensive intervention resources may be appropriate. Example: Jessica is 2.1x discrepant from peers on the Math CBM and may benefit from intensive interventions in math. Data-base decisions on performance level enables team to make decisions about levels of support and resource from the start.

38 Rebecca 2 nd grader List all areas of concern: –Off-task behavior –Reading difficulties –Poor handwriting Identify primary area of concern and define it in observable and measurable terms: –Reading –Definition: number words read correctly when reading a grade level passage orally Collect baseline data on primary area of concern and state discrepancy statement: –Baseline data collected in the area of test from CBM reading probes –Discrepancy Statement: Rebecca reads 41 WRC per minute in Fall of 2nd grade while her peers read ____ WRC per minute ___________ _______________________________________________________

39 Rob 7 th grader List all areas of concern: –Calling out –Lack of homework completion –Poor handwriting Identify primary area of concern and define it in observable and measurable terms: –Work Completion –Definition: Turning in teacher assigned work at beginning of class period on the day that it is due. Collect baseline data on primary area of concern and state discrepancy statement: –Baseline data collected in the area of review from teacher grade books –Discrepancy Statement: Rob currently turns in homework 54% of the time while his peers turn in homework 86% of the time (_____ discrepant)

40 Data-Based Decisions 1.Performance Level Gaps in Performance PLP Not at Grade Level Special Education Significant Discrepancy 2.Rate of Learning Trend in performance (slope) Response to Instruction General Direction, Rate of Change

41 2. Rate of Learning Why? –Determine when what we are doing isnt working and intervene early –Better able to predict student success at meeting goals –Better able to identify who needs more intensive instruction

42 NameGradeAreaInitial Performance Discrepancy Follow Up Performance Discrepancy Outcome Decision Bill3Reading10th percentile20th percentile Satisfactory; Maintain Intervention Susie2Math1.2xNANo Severe Problem Rob4Homewor k 65% completion64% completion No Progress, Problem solve and Modify intervention plan Rate of Learning Tracking Student Outcomes Using Initial Performance Discrepancies

43 Rate of Learning Analyzing Rate using PM Data Rules: –Setting Goals –Data Point Rules –Trend Line Rules –Slope

44 Setting Goals 1.End of the Year Benchmarks GLEs for Reading Fluency (2 nd grade WPM, 5 th grade ) AIMSweb Math Computation Norms ( 1 st grade 17 DPM, 5th grade 52 DPM) 2. National Norms for Improvement Math Calculations (>.3 DPM 2 nd and 3 rd grade, >.5 DPM 4-6 th grade) (Fuchs, 2006) Reading Fluency (Deno, 2005) GradeModestReasonableAmbitious 1-21 Word Per Week1.5 Word Per Week2.0 Word Per Week Word Per Week1.0 Word Per Week1.5 Word Per Week

45 Setting Goals 3. Individual ROI Weekly rate of improvement in baseline slope calculated from 8 data points (Slope: Difference of highest and lowest/#weeks) Baseline multiplied by 1.5 Product multiplied by number of weeks until end of year Add to students final baseline score to produce end of year goal. Baseline Reading scores: 52, 54, 52, 53, 55, 58, 55, 56 Difference: =5 Divide by number of weeks: 5/8 =.625 (SLOPE) Baseline multiplied by 1.5:.625 × 1.5 =.9375 Number of weeks left (6 weeks):.9375 ×6 =5.6 Add to final baseline score: = 61.6 End of the year goal 62

46 Jim – 5 th grader Math Calculation December-January Monitoring 36, 37, 36, 36, 37, 38, 39, 37 January Benchmark 38 Base determination on 18 weeks left in the year What goal would you set for Jim in math for the end of year?

47 Jim – 5 th grader End of Year Benchmarks 52 DPM (.77 ROI) National Norms 47 DPM (.5 ROI) (38+18*.5) Individual ROI 48 DPM (.56 ROI) 3/8*1.5 =.56 ROI 38 + (18*.56) What goal would you set for Jim in math for the end of year?

48 Decisions based on data-points Decisions are based on at least 4 data points If all 4 scores fall above goal-line, responding to instruction (increase goal if continues for 4 more data points) If scores are hovering about the goal line, continue what you are doing. If all 4 scores are below goal-line, but parallel, decide to wait for 4 more points to see if student performance accelerates in level to reach original goal. If all 4 scores fall below goal-line, not responding to instruction, revise plan and implement different teaching strategy. Mark change on graph with vertical line. Derived from: Fuchs and Fuchs (2006) and Shapiro (2006)

49

50 Decisions based on trend lines Trend lines based on 6-8 data-points If trend line is steeper than goal line, increase the goal. If trend line is flatter than goal line, revise instruction If trend line equals goal line, make no change at this time.

51

52 Data-base decision on slope Rate of Improvement = slope or r(in statistics) Y = slopeX + intercept Consider discrepancy from ROI norms

53

54 Is Jim responding to the intervention?

55 Checkpoint: Is Jim Responding to the Math Intervention? Are modifications necessary? End of Year: What would you say about Jims Math?

56 How can you use the literacy, math and social-emotional/behavior assessments that you have? Do they provide information about –a students performance level –rate of learning –area of instructional need?

57 Your chart might include one of these literacy measures? R-CBMRigbyDRA Performance Level YESYES?YES Rate of Learning YESNO Instructional needs YES?YES

58 What is the difference between data-driven and data informing?

59 Rebecca- 2 nd grader Fall benchmark some concern 41 WPM, <25 th percentile, 1.96X Discrepant Instructional Plan –small group guided reading –book bag for home with leveled readers

60 Rebecca- 2 nd grader

61 What additional data is needed to modify interventions and meet Rebeccas needs? Possible Hypotheses ReviewInterviewObserveTest Rebecca reads 41 WRC per minute in the Fall of 2nd grade while her peers read ____WRC per minute and with small group guided reading limited rate of improvement because …

62 Case Example: Rebecca Possible Hypotheses ReviewInterviewObserveTest She had insufficient opportunity to practice Interview Teacher/Parent Observe during SSR CBM She lacks the phonological skills to read the text fluently. Review her Records R-CBM scores Interview Teacher/Parent Phonological Assessment Instructional materials are above her reading level. Review her Records/CBM scores Interview Teacher/Parent CBM survey level assessment Rebecca reads 41 WRC per minute in the Fall of 2 nd grade while her peers read 80 WRC per minute and with small group guided reading limited rate of improvement because …

63 Case Example: Rebecca Predicted Hypotheses ReviewInterviewObserveTest She had insufficient opportunity to practice Instructional Planning Form revealed Rebecca had 15 minutes of independent reading practice per day Observation revealed that during SSR, Rebecca often played with items on desk. CBM probes indicated that she was not performing comparable to peers She lacks the phonological skills to read the text fluently. Review of records indicated that she had done well in remedial phonics program the previous year and was graduated out of the program. IPF suggested that instructional time was given to phonological skills. CBM probes indicated that she was not performing comparable to peers; DIBELS assessments did not point to a phonological awareness or phonics problem. Her reading level is not instructionally appropriate. Teacher reports showed that she was at the 25 th percentile in the Spring of 1 st grade Teacher felt that curriculum was appropriate Survey level assessment revealed that Rebecca had mastered 1 st grade text

64 Case Example: Rebecca Rebecca reads 41 WRC per minute in the Fall of 2 nd grade while her peers read 80 WRC per minute and is improving at a limited rate (.5 ROI) because … She had insufficient opportunity to practice.

65 What was the impact of progress monitoring assessments for Rebecca and her teachers?

66 Rebeccas progress during Readers Theatre

67 things you learned today 2 things you still have questions about 1 statement about how you feel about PM now

68 We will have tough choices to make – well decide based on whats best for our kids

69 If youre not hopelessly confused, youre out of touch! If you are hopelessly confused, then you only have one choice try stuff. Tom Peters, Embracing Chaos, 1993

70 We will make mistakes – and fix them along the way

71 Questions: Laura Hauerwas Kristen Matthes Thank You!

72 References Deno, S., Lembke, E. and Anderson, A. (2005) Progress Monitoring Study Group Content Module available at Fuchs and Fuchs (April, 2006) Progress Monitoring: Identifying LD and Improving Student Outcomes. Presentation at National SEA Conference on SLD Determination. Kansas City, MO. Hawkins, and Tilly, D. (Sept. 2005). Response to Intervention On the Ground: Diagnosing the Learning Enabled. Pacific Northwest Institute on Special Education and the Law. Shapiro, E. (2006). Advanced Issues in Monitoring Student Progress in Reading. Presentation for Pennsylvania Training and Technical Assistance Network on on RTI. Tilly (Sept, 2005) Problem Identification and Analysis. Presentation at the Principal Lead Problem Solving Initiative, Highland Park, IL.


Download ppt "Nuts and Bolts of Progress Monitoring Laura Boynton Hauerwas Kristen Matthes May 31, 2006."

Similar presentations


Ads by Google