Educational Consultation Services, LLC CBM, Charting, Progress Monitoring, and Data Based Instructional Decision Making in a RTI Model Tom Jenkins, Ed.D. Educational Consultation Services, LLC Wilmington, NC

What you can expect today CBM, CBA, CBE Instruction on progress monitoring Examples and hands-on activity Final thoughts and conclusions

What is CBM? Curriculum-based measurement Data collection tools derived directly from the curriculum that student is expected to learn

CBM CBM is believed to reduce the gap between assessment and instruction Aides teachers in generating superior student achievement Improved communication Higher level of sensitivity Enhancement of the database Administration time is shorter More cost effective

CBM The simplicity of CBM allows for quick and easy peer referencing Normative data can be collected This allows for comparison of a student’s performance to his/her actual peer group More representative geographically, culturally, ethnically, and has been exposed to similar instructional environment

CBM CBM has been shown to posses high levels of reliability 42 one-minute CBM type assessments in reading, math, and written expression for grade K-5 were found to have reliability coefficients between .90-.99 with just three one-minute administrations (Jenkins, 2002)

CBM Discriminant Validity Several studies have demonstrated the ability of CBM to differentiate between students receiving special education services, students receiving Chapter 1 services, and students not receiving any of those services (Deno, Marston, Shinn, and Tindal, 1983; Marston and Deno, 1982; Shinn and Marston, 1985; and Shinn, Tindal, Spira, and Marston, 1987).

CBM Procedures Remember different modalities Given the nature of reading probes they must be administered individually Math and written expression may be administered in a group setting

CBM Procedures Scoring One minute administration time except for written expression Number ID is scored corrects for minute; however, math computations are scored digits correct per minute Written expression is scored according to correct sequences

Example Administration example Work in pairs Each person gets a chance to adminster

CBM Procedures Written expression examples

Progress Monitoring Basic Principle #8 of PSM Progress monitoring an essential aspect of the intervention phase Basic Principle #9 of PSM Decision making in regards to the effectiveness of an intervention is based on analysis of progress monitoring data in relation of goal

Progress Monitoring Essential for four reasons There is no guarantee that interventions will be successful, thus the intervention must be “tested” to evaluate effectiveness Increased emphasis of specific outcomes for students, data base must be generated to guide intervention decision making Pre/post testing has be shown to be unreliable (small amount of data) and provides too little data to allow for instructional decision making – progress monitoring allows for evaluation of level of performance and rate of learning Research has shown that progress monitoring is associated with improved educational outcomes

Progress Monitoring Definition Frequent and repeated data collection (no less than 2-3 times a week) and analysis of student performance Data is collected during intervention and provides basis for intervention effectiveness

Progress Monitoring Essential components that must be in place for successful progress monitoring A well-defined behavior A measurement strategy Identification of student’s current level of performance (baseline) Intervention Goal Graph Decision-making plan

Well Defined Behavior Target behavior, observable, measurable, and specific Focus on enabling skills Skills that are prerequisite skills for more complex skills Deficiencies in enabling skills often adversely affects performance on global assessments

Well Defined Behavior Enabling skills for reading Phonemic awareness Alphabetic understanding Fluency Sight words Comprehension

Well Defined Behavior Enabling skills for math Number sense Facts Computation Applications Problem solving Enabling skills for written expression Mechanics Expression

Well Defined Behavior Enabling skills for behavior Social skills Work completion Compliance Problem solving skills

Measurement Strategy Frequent and repeated collection of data Time and cost efficient Sensitive to changes over short periods of time

Current Level of Performance (Baseline) Gathered prior to intervention Repeatable Provides comparison for progress data Helps set goal Median score – why?

Intervention Match intervention to problem Humans tend to employ interventions with which they are comfortable instead of intervention that the student needs Intervention should be developed with the expectation that it will be altered in some way as a result of the progress monitoring data No intervention works all of the time for every student

Goal Standard against which progress can be compared Allows for aimline to be established Possible goals Norms Percentile cutoff Realistic growth rates Ambitious growth rates Minimum celeration Local growth rates

Computing Growth Rates Winter Norm minus Fall Norm, divided by number of weeks between norming projects Gives you a growth expectancy for each week of school year Allows for obtaining student’s baseline then monitoring progress while comparing to growth expectancy

Computing Growth Rates Example – first grade, ID words in sentences Winter Norm minus Fall Norm, divided by number of weeks between norming projects 67.9119-35.2694 / 10 = 3.2643 words per week

Growth Rates based on research by Doug and Lynn Fuchs Realistic Growth Rates Gr 1 2 words/week Gr 2 1.5 wrds/week Gr 3 1 words/week Gr 4 .9 words/week Gr 5 .5 words/week Ambitious Growth Rates Gr 1 3 words/week Gr 2 2 words/week Gr 3 1.5 wrds/week Gr 4 1.1 wrds/week Gr 5 .8 words/week

Goal 1.25 minimum celeration, Precision Teaching

Graph Provides a visual representation of a large amount of data A visual representation of student’s acquisition of skills and allows for easier analysis of progress Semi-log chart Equal interval charts can misrepresent data, depending on how axis is quantified Equal interval charts assumes equal amounts of progress between all data points Precision Teaching

Decision Making Plan Facilitates interpretation of data Should include Rule for raising performance goal – 6/4 consecutive data points above the aimline Rule for altering the intervention due to lack of progress – 3/4 consecutive data points below the aimline Deno/Allison

Decision Making Plan If the decision is to adjust an intervention, small changes or refinements are recommended before major changes However, changes should be substantial enough that it has a possibility to result in improved student performance If making an adjustment, do not make two at the same time. It may result in the team being unable to determine what caused increased student performance

Decision Making Plan As student performance approaches goal, team must decide Raise the goal Begin work on another target behavior Discontinue PSM process If in level four consider Discontinuation of EC services

Charting Activity A means of interpreting large amounts of data Allows for relating performance trends to desired performance Research has shown the charting also facilitates positive student outcomes (Fuchs, 1989)

Charting Activity Definition Visual depiction of the student’s performance data, relative to the goal and aimline Includes baseline data, goal, aimline, and progress monitoring data

Charting Activity Procedures Write the prediction/goal statement Depict baseline data collection phase on the chart, indicate the median score, identify with a heavy dark line Depict the goal at the end of the anticipated intervention phase Depict the aimline by connecting the baseline median with the goal, this gives you expected rate of progress

Charting Activity Activity Plot the baseline data Monday – 14 Tuesday – 10 Wednesday – 6 Thursday – 5 Friday – 10 Plot the median and signify with a heavy dark line

Charting Activity Plot the goal at the end of eight weeks 40 Indicate the aimline

Charting Activity Performance trends should be analyzed periodically Trend above aimline – raise the goal Trend below aimline – adjust intervention If changes are made to the intervention, indicate change on the graph with a squiggle line Describe the changes on the back of the chart This allows for understanding of specific instructional adjustments that were successful/unsuccessful

Charting Activity Plot the first two weeks progress monitoring data Week One Tuesday – 16 Thursday – 14 Week Two Monday – 10 Wednesday – 18 Friday – 16 Make an informed decision regarding the effectiveness of the intervention

Charting Activity Remember Rule for raising performance goal – 6/4 consecutive data points above the aimline Rule for altering the intervention due to lack of progress – 3/4 consecutive data points below the aimline Deno/Allison

Charting Activity Plot progress monitoring data for next two weeks Week Three Tuesday – 14 Thursday - 10 Week Four Monday – 15 Wednesday – 16 Friday – 16 Make an informed decision regarding the effectiveness of the intervention

Charting Activity Phase two or three of the intervention? Plot the progress monitoring of the next two weeks Week Five Tuesday – 24 Thursday - 26 Week Six Monday – 26 Wednesday – 24 Friday – 26 Make an informed decision regarding the effectiveness of the intervention

Charting Activity Phase two or three of the intervention? Plot the progress monitoring of the next two weeks Week Seven Tuesday – 28 Thursday - 30 Week Eight Monday – 32 Wednesday – 36 Friday – 38 Make an informed decision regarding the effectiveness of the intervention Final decision?

Charting Activity More information than you wanted to know Trend lines Line that you draw through a series of data points that represents the student’s actual rate of progress If trend line slope is flatter than aimline slope, then adjust intervention If trend line slope is steeper than aimline slope, then adjust goal If slopes are the same, make no change

Charting Activity Trend line procedures Draw vertical line in middle of graph, half data points on one side, half on the other – if odd number of data points, put line through middle point Draw a vertical line in first half of data to separate data half and half Draw a vertical line in second half of data to separate data half and half

Charting Activity Trend line procedures Draw a horizontal line through median of first half of data – form an intersection with vertical line Draw a horizontal line through median of second half of data – form an intersection with vertical line Connect the two intersections Make your decision, is it the same decision?

Thoughts and Conclusions What if we took it a step further? What if we used the intervention/progress monitoring/charting process to move students towards a CBM reading fluency score that we knew predicted success on EOG’s and EOC’s?

Thoughts and Conclusions Espin et al, 2002 found correlation between CBM reading fluency scores and Minnesota grade 8 state standards test scores to be .78 Grade 5 correlation between CBM reading fluency scores and Minnesota Comprehensive Assessments - .77

Thoughts and Conclusions Similar results were found in Minnesota for grades 3 and 5 Similar results have been found in different states

Thoughts and Conclusions Similar results have been found in Oregon and Washington

Thoughts and Conclusions What does all this mean? Implement a school-wide model of progress monitoring Screen all students in the fall Identify students with CBM fluency scores that make them “at risk” on the standards based testing Intervene and monitor progress of the students “at risk” – move them towards a CBM reading fluency score that predicts a higher likelihood of success on the standards based testing

Thoughts and Conclusions Goals can be set based on local norming data – find CBM reading fluency for those that did meet standard Growth rates – realistic or ambitious Fluency benchmarks set by research (Deno) 1st grade – 60 wpm 2nd grade – 90 wpm 3rd grade – 120 wpm 4th grade – 130 wpm 5th grade – 140 wpm 6th grade – 150 wpm

Thoughts and Conclusions How can I progress monitor with all the other things that I have to do? School improvement occurs when focus is placed on student improvement outcomes, priorities are going to have to be changed so that progress monitoring is seen as just as important as instruction

Thoughts and Conclusions We have standards based tests to measure student progress, why do more? These tests are usually administered once a year and show student success in meeting a criterion. Progress monitoring enables learning, shows if a student is making progress towards success on standards based tests, and allows for informed instructional decision making that helps the student reach the goal of success on standards based tests.

Questions? Dr. Tom Jenkins, Director Educational Consultation Services, LLC Wilmington, NC (910) 367-7209 Fanofstel@aol.com www.educationalconsultationservices.com