Presentation on theme: "Nuts and Bolts of Progress Monitoring"— Presentation transcript:
1 Nuts and Bolts of Progress Monitoring Laura Boynton HauerwasKristen MatthesMay 31, 2006
2 Topics for the morning Progress Monitoring Assessment: Sample PM tools What?Why?Sample PM toolsGraphing DataData-Base DecisionsSetting GoalsPerformance LevelRate of LearningLinking cases to RI RTI processes
3 Assessment in a RTI model BenchmarkingTo screen and identify students who are at-risk and in need of interventionsAll studentsThree times a yearAll areasAt grade-levelProgress MonitoringTo monitor progress of individual students and determine rate of improvement and need for adaptation of interventionStudents who are not achieving benchmarks (PLP, IEP)Weekly, biweekly, monthly assessmentsIn area of needAt instructional level
4 What assessments do you use? ReadingMathSocial-EmotionalBehaviorWhat measures do you use?Who do you assess?When do you assess?How is the information used? Benchmark/Progress Monitoring
5 Progress Monitoring Benefits of Progress Monitoring Parents and students know what is expectedTeachers know what is working or not working with their instruction based on dataEasy to understand way to show parents progressTeams have comprehensive data on student performance for decision making
6 Progress Monitoring CBMs Are assessments to monitor progress Are designed to serve as “indicators” of general reading achievement. R-CBM doesn’t measure everything, but measures the important things.Are Standardized tests to be given, scored, and interpreted in a standard way.Are researched with respect to psychometric properties to ensure accurate measures of learning.Are Sensitive to improvement in Short Periods of time.Designed to be as short as possible to ensure its “do ability.”Are linked to decision making for promoting positive achievement and Problem-Solving
7 Progress MonitoringNational Center on Student Progress Monitoring
12 MATH COMPUTATIONTaken from Fuchs, L. S., Hamlett, C. A., & Fuchs, D. (1998). Monitoring Basic Skills Progress: Basic Math Computation (2nd ed.). [computer program]. Austin, TX: ProEd.Available: from
13 Concepts and Applications Sample page from a three-page test for Grade 2 Math Concepts and ApplicationsFrom Monitoring Basic Skills Progress
14 CBM - Writing Total Words Written Correct Word Sequences Words Correctly Spelled
18 Graphing Graphing is an essential part of PM Without graphic displays, the decision making process is difficultTeacher graphing vs. Student graphing
19 How to Develop Graphs Hand Graphing Excel and Chart Dog Graphing Data: Beetle, SUV, Race CarHand GraphingExcel and Chart DogWeb-based data systems
20 Hand Graphing Establish Baseline (Median score) Set up graph Set Goal Draw AimlineMeasure Student ProgressPlot Student PerformanceConnect Indicators of Student PerformanceAnalyze Student PerformanceMake Instructional ChangesContinue to Measure and Monitor Student Performance
22 Hand Graphing 50 45 40 35 30 Testing Sessions Number of Words Read Correctly3530BaselineSession 1Session 2Session 3Session 4Session 5Session 6Session 7Session 8Testing Sessions
23 Hand Graphing Advantages Easy to do No technology required Students can easily maintain their own graphsCan be done immediatelyFreeDisadvantagesAdded paperOrganization requiredNo long-term storageNot automatic
24 Excel Excel Template Website to access Excel Templates To add trend line:Select Data PointsRight Click …Add Trend LineChoose Linear,To find slope –Options Add Equation to the chart (y= slopex+ intercept)
26 Excel and Chart Dog Advantage Automatic Storage capability built-in Easy to doClear displays of dataFree if you have EXCEL, Chart Dog is freeDisadvantageRequires technologyTime to enter dataStudents may not be able to do data entry themselvesRequires some understanding of EXCEL or Chart Dog
28 Web-Based System Advantage Web based data entry from anywhere Storage capability built-inTrend line drawn automaticallyCan annotate graphs interventions/goalsNorms –benchmarks and Rate of ImprovementLots of flexibilitygraphsDisadvantageRequires technologyCostStudents may not be able to do data entry themselvesRequires some training
29 BREAKTo do this will take newlearning for everyone
30 Data-Based Decisions Performance Level Rate of Learning Gaps in PerformancePLP Not at Grade LevelSpecial Education Significant DiscrepancyRate of LearningTrend in performance (slope)Response to InstructionGeneral Direction, Rate of Change
31 Performance Level: Gap/ Discrepancy Be objective. Does it refer to an observable/measurable characteristic of behavior?Use numbers to define the discrepancy.Percentile rankDiscrepancy RatiosCut scoresNorms
32 Norms… What to use? Local, National Local norms can be helpful to determine local performance levels and rates of progressTime consuming and costly to developNational norms and research norms are available.BUT….
33 1. Performance Level Percentile Ranks Requires a Larger Normative Data Base, Preferably Benchmark Data< 25th At Risk, Consider Problem-Solving at the Group Level<10th Potential Severe Problem, Consider Individual Problem Solving
34 Performance Level Discrepancy Ratios Sample 5-7 Students or Whole Class, GradeFigure Median and GraphDivide by 2 and GraphStudents Who Performance Below the Line May Need Problem Solving
35 = Discrepancy of 3.6x Can Compute… Peer Median Target Student Median 14540= Discrepancy of 3.6x
36 Performance Level Cut Scores A number which represents the point at which scores can be divided into different groups (for example does not meet, meets, and exceeds expectations) for decision-making purposes.May be based on research (e.g., a correlation between scoring at or above a certain level on a CBM or DIBELS task and future academic success) or expectation (e.g., grades at C or above, no more than 3 office referrals).
37 Data-base decisions on performance level enables team to make decisions aboutlevels of support and resource from the start.Generally speaking…- A student who is 1.5x discrepant from his/her peers may benefit from intensive group interventions.A student who is 2-2.5x discrepant from his/her peers is appropriate for individualized problem-solving and intensive intervention resources may be appropriate.Example: Jessica is 2.1x discrepant from peerson the Math CBM and may benefit from intensive interventions in math.
38 Rebecca 2nd grader List all areas of concern: Off-task behaviorReading difficultiesPoor handwritingIdentify primary area of concern and define it in observable and measurable terms:ReadingDefinition: number words read correctly when reading a grade level passage orallyCollect baseline data on primary area of concern and state discrepancy statement:Baseline data collected in the area of test from CBM reading probesDiscrepancy Statement: Rebecca reads 41 WRC per minute in Fall of 2nd grade while her peers read ____ WRC per minute __________________________________________________________________
39 Rob 7th grader List all areas of concern: Calling outLack of homework completionPoor handwritingIdentify primary area of concern and define it in observable and measurable terms:Work CompletionDefinition: Turning in teacher assigned work at beginning of class period on the day that it is due.Collect baseline data on primary area of concern and state discrepancy statement:Baseline data collected in the area of review from teacher grade booksDiscrepancy Statement: Rob currently turns in homework 54% of the time while his peers turn in homework 86% of the time (_____ discrepant)
40 Data-Based Decisions Performance Level Rate of Learning Gaps in PerformancePLP Not at Grade LevelSpecial Education Significant DiscrepancyRate of LearningTrend in performance (slope)Response to InstructionGeneral Direction, Rate of Change
41 2. Rate of LearningWhy?Determine when what we are doing isn’t working and intervene earlyBetter able to predict student success at meeting goalsBetter able to identify who needs more intensive instruction
42 Tracking Student Outcomes Using Initial Performance Discrepancies Rate of LearningTracking Student Outcomes Using Initial Performance DiscrepanciesNameGradeAreaInitial Performance DiscrepancyFollow Up Performance DiscrepancyOutcome DecisionBill3Reading10th percentile20th percentileSatisfactory; Maintain InterventionSusie2Math1.2xNANo Severe ProblemRob4Homework65% completion64% completionNo Progress, Problem solve and Modify intervention plan
43 Rate of Learning Analyzing Rate using PM Data Rules: Setting Goals Data Point RulesTrend Line RulesSlope
44 Setting Goals End of the Year Benchmarks GLEs for Reading Fluency (2nd grade WPM, 5th grade )AIMSweb Math Computation Norms (1st grade 17 DPM, 5th grade 52 DPM)2. National Norms for ImprovementMath Calculations (>.3 DPM 2nd and 3rd grade, >.5 DPM 4-6th grade)(Fuchs, 2006)Reading Fluency (Deno, 2005)GradeModestReasonableAmbitious1-21 Word Per Week1.5 Word Per Week2.0 Word Per Week3-6.5 Word Per Week1.0 Word Per Week
45 Setting Goals 3. Individual ROI Weekly rate of improvement in “baseline slope” calculated from 8 data points (Slope: Difference of highest and lowest/#weeks)Baseline multiplied by 1.5Product multiplied by number of weeks until end of yearAdd to student’s final baseline score to produce end of year goal.Baseline Reading scores: 52, 54, 52, 53, 55, 58, 55, 56Difference: =5Divide by number of weeks: 5/8 =.625 (SLOPE)Baseline multiplied by 1.5: × 1.5 = .9375Number of weeks left (6 weeks): ×6 =5.6Add to final baseline score: = 61.6End of the year goal 62
46 Jim – 5th grader What goal would you set for Jim in math for the end of year?Jim – 5th graderMath CalculationDecember-January Monitoring36, 37, 36, 36, 37, 38, 39, 37January Benchmark38Base determination on 18 weeks left in the year
47 Jim – 5th grader What goal would you set for Jim in math for the end of year?Jim – 5th graderEnd of Year Benchmarks DPM (.77 ROI)National Norms DPM (.5 ROI)(38+18*.5)Individual ROI DPM (.56 ROI)3/8*1.5 =.56 ROI38 + (18*.56)
48 Decisions based on data-points Decisions are based on at least 4 data pointsIf all 4 scores fall above goal-line, responding to instruction (increase goal if continues for 4 more data points)If scores are hovering about the goal line, continue what you are doing.If all 4 scores are below goal-line, but parallel, decide to “wait” for 4 more points to see if student performance accelerates in level to reach original goal.If all 4 scores fall below goal-line, not responding to instruction, revise plan and implement different teaching strategy.Mark change on graph with vertical line.Derived from: Fuchs and Fuchs (2006) and Shapiro (2006)
50 Decisions based on trend lines Trend lines based on 6-8 data-pointsIf trend line is steeper than goal line, increase the goal.If trend line is flatter than goal line, revise instructionIf trend line equals goal line, make no change at this time.
61 What additional data is needed to modify interventions and meet Rebecca’s needs? PossibleHypothesesReviewInterviewObserveTestRebecca reads 41 WRC per minute in the Fall of 2nd grade while her peers read ____WRC per minute and with small group guided reading limited rate of improvement because …
62 Case Example: Rebecca Possible Hypotheses Review Interview Observe Rebecca reads 41 WRC per minute in the Fall of 2nd grade while her peers read80 WRC per minute and with small group guided reading limited rate of improvement because …PossibleHypothesesReviewInterviewObserveTestShe had insufficient opportunity to practiceInterview Teacher/ParentObserve during SSRCBMShe lacks the phonological skills to read the text fluently.Review her RecordsR-CBM scoresPhonological AssessmentInstructional materials are above her reading level.Review her Records/CBM scoresCBM survey level assessment
63 Case Example: Rebecca Predicted Hypotheses Review Interview Observe TestShe had insufficient opportunity to practiceInstructional Planning Form revealed Rebecca had 15 minutes of independent reading practice per dayObservation revealed that during SSR, Rebecca often played with items on desk.CBM probes indicated that she was not performing comparable to peersShe lacks the phonological skills to read the text fluently.Review of records indicated that she had done well in remedial phonics program the previous year and was graduated out of the program.IPF suggested that instructional time was given to phonological skills.CBM probes indicated that she was not performing comparable to peers; DIBELS assessments did not point to a phonological awareness or phonics problem.Her reading level is not instructionally appropriate.Teacher reports showed that she was at the 25th percentile in the Spring of 1st gradeTeacher felt that curriculum was appropriateSurvey level assessment revealed that Rebecca had mastered 1st grade text
64 Case Example: RebeccaRebecca reads 41 WRC per minute in the Fall of 2nd grade while her peers read 80 WRC per minute and is improving at a limited rate (.5 ROI) because …She had insufficient opportunity to practice.
65 What was the impact of progress monitoring assessments for Rebecca and her teachers?
67 3 - 2 - 1 3 things you learned today 3 things you learned today2 things you still have questions about1 statement about how you feel about PM now
68 We will have tough choices to make – we’ll decide based on what’s best for our kids
69 If you’re not hopelessly confused, you’re out of touch If you’re not hopelessly confused, you’re out of touch! If you are hopelessly confused, then you only have one choice — try stuff Tom Peters, Embracing Chaos, 1993
70 We will make mistakes – and fix them along the way
71 Thank You!Questions: Laura Hauerwas Kristen Matthes
72 ReferencesDeno, S., Lembke, E. and Anderson, A. (2005) Progress Monitoring Study Group Content Module available atFuchs and Fuchs (April, 2006) Progress Monitoring: Identifying LD and Improving Student Outcomes. Presentation at National SEA Conference on SLD Determination. Kansas City, MO.Hawkins, and Tilly, D. (Sept. 2005). Response to Intervention On the Ground: Diagnosing the Learning Enabled. Pacific Northwest Institute on Special Education and the Law.Shapiro, E. (2006). Advanced Issues in Monitoring Student Progress in Reading. Presentation for Pennsylvania Training and Technical Assistance Network on on RTI.Tilly (Sept, 2005) Problem Identification and Analysis. Presentation at the Principal Lead Problem Solving Initiative, Highland Park, IL.