PROGRESS MONITORING FOR DATA-BASED DECISIONS June 27, 2007 FSSM Summer Institute

2 Progress Monitoring: Learning Goals for Today Determine student’s current level of performance (Baseline data) CBM Identify learning goal (norms) Implement Research-Based Interventions Continue to measure and monitor students’ performance on a regular basis Graph the results Compare expected progress to actual rate Adjust instruction based on the data

3 What IS Progress Monitoring Find the in your handout packet. Work with your TEAM to complete this. Be ready to share a specific example of Progress Monitoring in your school. Handout PM 1

4 Progress Monitoring is a scientifically-based practice that is used to assess students’ academic performance and evaluate the effectiveness of instruction – for an individual student or an entire class.” National Center on Progress Monitoring

5 A Scientific Base Supports One Form of Progress Monitoring: Curriculum-Based Measurement CBM

6 Curriculum-Based Measurement (CBM)... Result of years of research Used in schools across the country Uses short “probes” for frequent PM Demonstrates strong reliability and validity Is sensitive to small gains in progress Helps teachers plan instruction

7 Uses of CBM Benchmark All Students – 3 times a year – F, W, S Strategic Monitor Monthly check up for students with moderate skill deficits who are receiving supplemental instruction in small group (3-5) Progress Monitor Monitor once or twice weekly to measure student’s response to more intensive interventions (individual or group of 2). Graph the data for on- going decision making.

School Calendar Year ( ): Benchmarking (Tier 1) 2-weeks during: September 1 to October 15 January 1 to February 1 May 1 to June 1

9 For Teachers: Classroom Report Benchmark Data AIMSweb.com

10 10th %ile 90th %ile 75th %ile 50th %ile 25th %ile Student is above the 90 %ile and is well above average. Student is above the 90 %ile and is well above average. Target ____________________________________ Box and Whisker Charts AIMSweb

11 AIMS web / Harcourt w/ permission 6/07

12 Decision Making Rules Students who score below the 25 th percentile on general outcome benchmark screening receive targeted intervention, and progress is monitored on a monthly basis. Progress for those below the 10 th percentile is monitored at least weekly, and they receive intensive intervention. Eight data points over at least 4 weeks are required to determine a trend line. Change or modify the intervention when the data points are below the aim line (goal line) for 4 consecutive data points, or when the trend line is not on course to meet the goal line.

13 AIMSweb/Harcourt w/ permission 6/07

14 AIMSweb/Harcourt w/ permission 6/07

15 AIMSweb/Harcourt w/ permission 6/07

16 How Does Student Progress Monitoring Work? Determine student’s current level of performance (Baseline data) Identify learning goal (local norms or accepted research standards) Use research based intervention(s) targeting the problem Continue to measure performance on a regular basis (CBM probes at same level) Graph the results Compare expected progress to actual rate Adjust instruction based on the data

17 Calculating Slope: First draw a Trend Line – Tukey Method Step 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.) Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.” Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement. PM 2

18 Step 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.) Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.” Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement.

19 Step 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.) Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.” Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement. X X

20 Step 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.) Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.” Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement. X X

21 Baseline Data & Goal TOM Second grade student, fall assessment, oral reading fluency, second grade probes Baseline Data: day 1 =11, day 2 = 13, day 3 =12 Plot these points on your graph PM 3

22 Baseline Data & Goal Line Baseline Data: 11, 13, 12 Find the median, or middle number when numbers are rank ordered: 11, 12, 13 Median = 12 words read correctly per min. Average Peer = ? Use norms table, Handout PM 4 Rate of Improvement = ? Words per wk. Use norms table, Handout PM 4

23 Baseline Data & Goal Baseline Data: 11, 13, 12 Median = 12 words read correctly per min. Average Peer = 55 wrc per minute Average Rate of Improvement = 1.1 words per week. Set a realistic but ambitious goal, such as: Goal = 2 words per minute more per week

24 Baseline Data & Goal Goal = 2 words per minute more per week Select # wks for monitoring: Weeks =12 Calculate: 2 wds./wk. increase, times 12 wks. = 24 words total increase Add the expected 24 word increase to baseline (12). Goal= 36 wrc in 12 weeks Mark the goal point (36) on the last data collection day, Thursday, at the end of 12 weeks of intervention. Connect baseline data median(12) to the goal point (36) = Goal Line (Aim Line)

25 GOAL LINE

26 Intervention Data Points 9/16/ /18/ /23/ /25/ /30/ /2/ /7/ /9/ Enter this data on your graph and draw a trend line.

27

28 Data-Based Decision Making Were there 4 consecutive data points below the goal line? Are there 8 data points for a trend line? Is the trend line on course to meet the goal line? Is this intervention effective? Decision?

29 Intervention #2 Data Points 10/14/ /16/ /21/ /23/ /28/ /30/ /4/ /6/ Enter this data on your graph and draw a trend line.

30 Intervention 1 Intervention 2

31 Data-Based Decision Making Draw a vertical line to show each intervention change. Label “Intervention 1”, “Intervention 2”, etc. Keep records of specific intervention protocols and fidelity of implementation. What is your decision based on Intervention 2 data?

32 Why is Fidelity Important? To demonstrate that measurable changes in behavior are related to systematic & controlled changes in the environment (intervention) Without objective & documented evidence that the intervention was implemented as planned, we can’t conclude that inadequate response to intervention was due to a poor intervention or insufficient intensity (ie: inadequate response may be due to poor instruction.) Likewise, success can’t be attributed to the intervention if we don’t know how it was implemented

33 3 rd Grade Winter R-CBM – Mia - Baseline Tues. 34 WRC Wed. 40 Thurs. 36 Enter the data. Mark MEDIAN with X PM 5

34 Graphing: Make Mia’s Goal Line Set a Reasonable but Ambitious Goal What is the average Rate of Improvement (ROI) for third grade? – norms – PM 4 Would a gain of 15 words per week be ambitious? Reasonable? Would a gain of 2 words per week be ambitions? Reasonable? Calculate Mia’s goal at the end of 12 weeks. Graph the GOAL LINE (Aim Line)

35 Mia’s Problem Statement “When given a third grade oral reading fluency probe, Mia reads 36 words correctly in one minute. Her average third grade peer reads 98 words correctly in one minute.” Intervention: Repeated Readings 2:30 – 2:50 M/W/F Implemented by: Classroom teacher Progress Monitor: T/Th R-CBM (3 rd )

36 Mia’s Progress Monitoring Data Mia’s R-CBM scores (Tues./Thurs.) Week 1: 34, 36 Week 2: 42, 36 Week 3: 36, 38 Week 4: 38, 38 Graph the data. PM 5

37 Make a Trend Line Use Mia’s data: Weeks 1-4 (Don’t include the baseline data) Use the Tukey Method to draw a trend line. Compare Mia’s Trend Line & Goal Line. What’s your decision about the intervention?

38 To Monitor Student Progress Determine student’s current level (Baseline) Identify learning Goal (local/research norms) Research based Intervention(s) target problem Implement with Fidelity Continue to measure performance on a regular basis (CBM probes at same level) Graph the results Compare expected progress to actual rate Adjust instruction based on the data

39 Where Do We Go From Here? RtI / SLD Eligibility Determination? Students who score below the 25 th percentile on general outcome benchmark screening receive targeted intervention, and progress is monitored on a monthly basis. Progress for those below the 10 th percentile is monitored at least weekly, and they receive intensive intervention. Eight data points over at least 4 weeks are required to determine a trend line. Change or modify the intervention when the data points are below the aim line (goal line) for 4 consecutive data points or when the trend line is not on course to meet the goal line. At least 2 intervention changes (ie: 3 interventions) are required before students may be referred for evaluation due to suspected disability.

40 Special Education Regulations Apply at Time of Referral Letter to parent informing of referral Evaluation decisions made by the Evaluation Team at the Review of Existing Data/Evaluation Plan Meeting. Parents are given written notice and must sign consent for the evaluation. “Specialized Instruction” is listed under “other” in the area of functioning (Ex: “Specialized Instruction, 225 minutes per week, special ed resource room.”) Specialized instruction must be very focused and targeted at the problem as operationally defined in the problem statement.

41 CBM is used to collect data in all academic areas, administered frequently (2X/wk) The Evaluation must be comprehensive and address all areas of concern. Consider norm referenced achievement assessment in addition to CBM (if not indicated in the student’s records). Level of performance must be far below & rate of improvement far below average peers. Use cognitive assessment if broad & pervasive concerns with student’s functioning across areas. However, do not calculate a discrepancy between IQ and norm referenced achievement scores. This information is not relevant to determining eligibility or interventions. See Handouts PM 7 & PM 8 for more specific procedues.

42 Progress Monitoring & IEP Goals Formula for Good Goals Using CBM Given a (specific grade level and subject area) probe, STUDENT will (increase/decrease/maintain) his/her ability to (state skill addressed) by (observable behavior) from (insert baseline) to (insert goal) for (insert monitoring period)

43 References Deno, S. (2003). Developments in Curriculum- Based Measurement. Journal of Special Education (37) (3), Fuchs, L. & Fuchs, D. (2002). Curriculum-based measurement: Describing competence, enhancing outcomes, evaluating treatment effects, and identifying treatment nonresponders. Peabody Journal of Education, 77, Hosp, M. & Hosp, J. (2003). Curriculum-based measurement for reading, math, and spelling: How to do it and why. Preventing School Failure, 48 (1),