Assessment and Tier III RtI Foundations Training June-September 2010 Erin Banks and Amy Roberts, School Psychologists

2 What is the Responsiveness to Instruction Model? A student with academic delays is given one or more research-validated interventions. The student's academic progress is monitored frequently to see if those interventions are sufficient to help the student to catch up with his or her peers. If the student fails to show significantly improved academic skills despite several well-designed and implemented interventions, this failure to 'respond to intervention' can be viewed as evidence of an underlying Learning Disability.

3 Why RtI? One advantage of RtI in the diagnosis of educational disabilities is that it allows schools to intervene early to meet the needs of struggling learners. Another advantage is that RtI maps those specific instructional strategies found to benefit a particular student. This information can be very helpful to both teachers and parents.

What is Curriculum Based Measurement? Curriculum Based Measurement is a form of Curriculum-Based Assessment (CBA). CBM is a method of monitoring student educational progress through direct assessment of academic skills CBMs measure basic skills in reading, mathematics, spelling, written expression and readiness skills –Teacher gives the student brief, timed samples, or “probes,” made up of academic material that the student is expected to learn Adapted from

CBMs Because CBM probes are quick to administer and simple to score, they can be given frequently to provide continuous progress data. The results are charted and provide for timely evaluation based on hard data Early literacy skills (phonics and phonological awareness) are downward extensions of CBM (General Outcome Measures - GOM).

Using Curriculum Based Measures as General Outcome Measures It’s about using General Outcome Measures (GOMs) for formative assessment/evaluation to: Inform teachingInform teachingAND ensure accountability. ensure accountability. It’s different from, but related to, summative high- stakes testing/evaluation, which: Doesn’t inform teaching. Doesn’t inform teaching. Mostly used for accountability/motivation. Mostly used for accountability/motivation. Adapted from

Formative Evaluations Formative Evaluation: Process of assessing student achievement during instruction to determine whether an instructional program is effective for individual students. When students are progressing, continue using your instructional programs. When tests show that students are not progressing, you can change your instructional programs in meaningful ways. Adapted from

Summative Assessments Summative Assessment: come at the end of a program of study Tells you what information the child has mastered for the year without ability to change instructionTells you what information the child has mastered for the year without ability to change instruction Big tests that include a wide range of information covered over a long period of timeBig tests that include a wide range of information covered over a long period of time High-stakes tests (EOGs)High-stakes tests (EOGs) Adapted from RtI in the Classroom

Universal (school-wide) screening using CBMs allows us to add systematic Formative Evaluation to current practice. For Teachers (and Students) For Teachers (and Students) Early Identification of At Risk Students Early Identification of At Risk Students Instructional Planning Instructional Planning Monitoring Student Progress Monitoring Student Progress For Parents For Parents Opportunities for Communication/Involvement Opportunities for Communication/Involvement Accountability Accountability For Administrators For Administrators Resource Allocation/Planning and Support Resource Allocation/Planning and Support Accountability Accountability Using Curriculum Based Measurement as a General Outcome Measure Adapted from

Using Curriculum Based Measurement as a General Outcome Measure: Research Curriculum-Based Measurement (CBM) was developed more than 20 years ago by Stanley Deno at the University of Minnesota through a federal contract to develop a reliable and valid measurement system for evaluating basic skills growth. Curriculum-Based Measurement (CBM) was developed more than 20 years ago by Stanley Deno at the University of Minnesota through a federal contract to develop a reliable and valid measurement system for evaluating basic skills growth. CBM is supported by more than 25 years of school- based research by the US Department of Education. CBM is supported by more than 25 years of school- based research by the US Department of Education. Supporting documentation can be found in 100s of articles, book chapters, and books in the professional literature describing the use of CBM to make a variety of important educational decisions. Supporting documentation can be found in 100s of articles, book chapters, and books in the professional literature describing the use of CBM to make a variety of important educational decisions. Adapted from

Summary of Research Validating Curriculum Based Measurement Reliable and valid indicator of student achievement Simple, efficient, and of short duration to facilitate frequent administration by teachers Provides assessment information that helps teachers plan better instruction Sensitive to the improvement of students’ achievement over time Easily understood by teachers and parents Improves achievement when used to monitor progress Adapted from

Curriculum Based Measurement: Advantages Direct measure of student performance Helps target specific areas of instructional need for students Quick to administer Provides visual representation (reports) of individual student progress and how classes are acquiring essential reading skills Sensitive to even small improvements in performance Adapted from

Advantages continued… Capable of having many forms Monitoring frequently enables staff to see trends in individual and group performance—and compare those trends with targets set for their students. Correlates strongly with “best practices” for instruction and assessment, and research-supported methods for assessment and intervention.

Curriculum Based Measurement: Things to Remember Standardized tests to be given, scored, and interpreted in a standard way. Standardized tests to be given, scored, and interpreted in a standard way. Researched with respect to psychometric properties to ensure accurate measures of learning. Researched with respect to psychometric properties to ensure accurate measures of learning. Are sensitive to improvement in brief intervals of time. Are sensitive to improvement in brief intervals of time. Tell us how students earned their scores (qualitative information). Tell us how students earned their scores (qualitative information). Designed to be as short as possible to ensure “do- ability.” Designed to be as short as possible to ensure “do- ability.” Are linked to decision making for promoting positive achievement and problem-solving. Are linked to decision making for promoting positive achievement and problem-solving. Adapted from

15 Curriculum Based Measurement CBM has been shown to posses high levels of reliability –Reliability - the extent to which the measurements of a test remain consistent over repeated tests of the same subject under identical conditions 42 one-minute CBM type assessments in reading, math, and written expression for grade K-5 were found to have reliability coefficients between with just three one-minute administrations (Jenkins, 2002)

16 Curriculum Based Measurement Discriminant Validity - Does it appear to measure what it’s supposed to measure? And Doesn’t associate with constructs that shouldn’t be related. Several studies have demonstrated the ability of CBM to differentiate between students receiving special education services, students receiving Chapter 1 services, and students not receiving any of those services (Deno, Marston, Shinn, and Tindal, 1983; Marston and Deno, 1982; Shinn and Marston, 1985; and Shinn, Tindal, Spira, and Marston, 1987).

17 Other types of CBMs - AIMSweb Similar reading probes to DIBELS that include reading fluency and early literacy Adds a reading comprehension piece (MAZE) Also has math computation, math concepts and applications, early numeracy and writing components

AIMSweb: Tests of Early Numeracy Number Identification Name single digit and double digit numbers One minute timed task Administer to: –Kindergarten – fall –Kindergarten – winter –Kindergarten – spring –1 st Grade – fall –1 st Grade – winter 18

AIMSweb: Tests of Early Numeracy Quantity Discrimination 19 Identify greater number One minute timed task Administer to: –Kindergarten – fall –Kindergarten – winter –Kindergarten – spring –1 st Grade – fall –1 st Grade – winter –1 st Grade - spring

AIMSweb: Tests of Early Numeracy Missing Number 20 Identify missing number One minute timed task Administer to: –Kindergarten – fall –Kindergarten – winter –Kindergarten – spring –1 st Grade – fall –1 st Grade – winter –1 st Grade - spring

AIMSweb: MAZE (Reading Comprehension) Student reads passage silently for 3 minutes Every 7 th word is replaced with three choices Student circles correct choice Can be group administered

AIMSweb: Data Management22

23

Other types of CBMs: Website has many CBM probes available for free You can create multiple forms of early literacy and numeracy probes Many national norms available for comparison 24

CBM Videos AIMSweb training videos –Letter Sound Fluency –Nonsense Word Fluency –Phoneme Segmentation Fluency –Quantity Discrimination

Tier III

What has happened to get us to this point? Student has not made sufficient progress with Tier I (4-6 weeks with pre and post test) and Tier II interventions (4-6 weeks with progress monitoring every 2 weeks) Now they go to RTI assistance team to receive more intensive interventions

Requirements of Tier III 8 week minimum time frame with 2 interventions (4 weeks each) Intervention 3-4 times a week for at least 30 minutes (in addition to the core) Progress monitoring 2 times a week Data check after 4 weeks

1st Meeting 1st Step - Parent invitation needs to be sent home PRIOR to the meeting Well defined target behavior: –Enabling skills for written expression: mechanics and expression –Enabling skills for math: number sense, facts, computation, applications, and problem solving –Enabling skills for behavior: social skills, work completion, compliance, and problem solving skills Define the problem by examining effects of environment, curriculum, instruction, and learner

Environmen t Environment: things in the child’s environment that is interfering with learning –Attendance issues, family life, transience –Issues in the classroom

Curriculum Curriculum: –Consider if curriculum appropriate for student. Consider sequence of objectives, teaching methods, and practice materials provided. –What does the core look like in the classroom? –What extra services is the child receiving? –Does the student need more systematic instruction? –Has he been exposed to numerous curriculums due to transience?

Instruction Instruction: –Manner in which teacher uses curriculum – consider instructional techniques, presentation style, questioning, feedback techniques. –Does he benefit from small group, 1:1 instruction?

Learner Learner: Student skill – necessary prerequisite skills needed to be successful; i.e. Lacks fluency for comprehension or lacks phonemic awareness which affects fluency and comprehension Student process – capacity to learn and problem solving techniques; i.e. Has difficulty with working memory or processing

RIOT How do you investigate why the problem is occurring? –RIOT: R - Review information: records, cumulative folders, etc. I - Interview others: parent, teacher, student O - Observe: student in specific class or core T - Test: basic skill levels –Assign who will be in charge of the investigation. Set the next meeting time.

Meet Bubba

Bubba’s Baseline Measuring Tool: Dog-A-Rama Scale Monday = 0 pounds lost Wednesday =.5 pounds lost Friday = 0 pounds lost Median Score = 0 pounds lost (baseline) (No intervention has started yet!)

Bubba’s Weight Loss Goal Goal Statement: Bubba will lose 25 pounds by 8 weeks, as measured by the Dog-O-Rama scale. Hypothesis: The gap between Bubba’s current and desired weight will be decreased if he begins a High Protein Dog Food diet.

Bubba’s Intervention Intervention: –Bubba will eat 1/2 cup of High Protein Dog Food at 7:30 AM and a 1/2 cup of High Protein Dog Food at 5:30 PM. –Stranger Test (does this make sense to someone off the street?)

Bubba’s Progress Monitoring Measuring Tool: Bubba’s progress will be measured by weighing him 2 times a week using the Dog-A-Rama scale to determine his pounds lost per week. Decision-Making Rule: If 5 or more data points are below the aimline, change intervention; if 5 or more data points are above the aimline; continue intervention.

Review Bubba’s Intervention Baseline = 0 pounds Data Points Below Aimline: 5 Data Points Above Aimline: 3 Is he making progress toward his goal? Should the vet change his intervention? YES!

Intervention Change Second Intervention: Bubba will participate in an exercise program consisting of walking 30 minutes, 3 times a week. Evaluation Tool: Continue to measure progress using the Dog- A-Rama scale, weighing Bubba 2 times a week.

Bubba’s Weight Loss Review Goal = 25 pounds Data Points Above Aimline: 1 Data Points Below Aimline: 8 Did Bubba make progress? Did he meet his goal? What should we do next?

Update on Bubba Bubba is losing weight! May choose to combine diet & exercise for most effective intervention

How Does Bubba Relate to the RtI Process? Obtaining Baseline: Gathered prior to intervention Provides comparison for progress data Helps set goals Give CBM 3 times Put the scores in numerical order and take the median score to get rid of outliers Get a new baseline if needed due to new information obtained from RIOT-ing

Review Baseline Team meets again within 2 weeks to document results Based on these results, set goal based on the data Baseline may be gathered between Tier II and Tier III to expedite process

Goal Setting Short-term goals: where do you want them to be at the end of 8 weeks (intervention term)? Long-term goals: where do you want them to be long-term, by the end of the year? Short-term goals are graphed

Goal Setting May use: – Grade-Level norms – RSS District norms – AIMSweb national aggregate norms –Rate of Improvement/Growth Rate Make short-term goal realistic based on baseline data but also high enough to get them to where they need to be (grade level) (at 10%ile, goal is 25%ile) Setting goals for students should put them on a TRAJECTORY to be on-grade level. Ex. By the end of 8 weeks, Johnny will increase his reading fluency to 100 correct words per minute on grade level reading passages

Reading Fluency Growth Rates (Doug and Lynn Fuchs) Realistic Growth Rates 1 - 2words/week words/week 3 -1 word/week words/week words/week Ambitious Growth Rates words/week words/week words/week words/week words/week

Goal Setting - Tier III paperwork Choose your goal Document the goal (where you want the student to be) Document their baseline (current level of performance) Calculate the difference Document method used for goal setting Ex. AIMSWeb Aggregate winter norms at 25th percentile (short-term goal)

Intervention Match intervention to problem. Intervention should be developed with the expectation that it will be altered in some way as a result of the progress monitoring data –Increase time –Increase frequency –Increase intensity before changing intervention completely (unless child has flatlined) No intervention works all of the time for every student

Intervention Plan What intervention will be used? (Use the stranger test!) –Ex. Florida Center for Reading Research (FCRR) P.001 Arrangements: where, who, how much, and with what? –Ex. 4 times a week in hallway alcove Persons responsible: who will deliver the intervention and who will progress monitor? –Ex. Mrs. Teacher will provide intervention and Amy Roberts will progress monitor

Intervention Plan… What will be used to measure progress and how often will it be measured? –Ex. AIMSweb reading CBM will be used to progress monitor 2x a week.

Intervention will be changed when… The majority of progress monitoring data points are below the child’s goal/aimline (5 out of 8) After 4 weeks there will be a data check to see what the data is telling us.

How to Create a Graph for Progress Monitoring Set up the graph –Write the student’s name on the graph –Label the vertical axis with what you are assessing (i.e. words correct per minute) –Label the horizontal axis with dates of assessment –Title the graph Establish baseline –Administer probe 3 times –Select the median (the middle number) –Mark the baseline by placing a dot on the vertical axis

Graphing Continued… Set the goal/target –Determine the expected rate of progress (e.g. 2 words per week, 110 words by end of 8 weeks) and plot on graph Draw the aimline –Using a red colored pencil and a ruler, connect the baseline data point with the target data point Measure and plot student performance –Administer probes or assessments according to plan –Place the data point on the graph –Connect the data point to the previous point

Graphing Analyze student performance –Determine whether the student is making adequate progress If 5 out of 8 data points are above aimline, raise the goal If 5 out of 8 data points are below aimline, the student is not making adequate progress and need a change in intervention If the data points are along the aimline, the student is progressing right on target and continue intervention and progress monitoring –If a change in intervention is required, draw a squiggle line from the top of the graph to the bottom to indicate intervention change

Last Page After 8 weeks, meet back and document child’s level before the intervention and their level after the intervention and if they achieved their goal Check decision: –No longer an area of concern –Continue intervention plan –Modify intervention plan –Move to Tier IV

Requirements to go to Tier IV a.Two interventions with data indicating performance below the 15 th %ile based on Rowan Salisbury norms. b.Performance that is more than two times below the grade mean (50 th %ile). For example, if the child has an ORF of 25 and the 50 th %ile is 100, the child’s performance is more than 2xs below the grade mean or 50 th %ile. c. The intervention is showing mild success but it is so intensive that it cannot be maintained without special education assistance. d.Growth rate indicating that of insufficient progress (their aimline and trend line will never cross).

That’s All Folks!