Progress Monitoring in Reading: Why, What, and How RtII Leadership Team RtII Champion Training 12/10/13 Today our session is on Progress Monitoring in Reading. The Why, What and How

Progress Monitoring The RTI Center defines progress monitoring as repeated measurement of academic performance to inform instruction of individual students in general and special education in grades K-8. It is conducted at least monthly to: (a) estimate rates of improvement, (b) identify students who are not demonstrating adequate progress, and/or (c) compare the efficacy of different forms of instruction to design more effective, individualized instruction. Have participant read the introduction. 2

Overview Why, What, How to Progress Monitor Why do it? What do we mean by progress monitoring? How do you do it? Using Progress Monitoring Data These will be the questions that will be answered today in this session. 3

Is this student making progress? Discussion could be added here for the class or small groups to look at and discuss what they think this graph means… is the student making progress? How do we know? What does the vertical line (intervention change/phase line) mean…. Is there a difference in the data before and after the change? Why do you think the teacher changed what was happening with Adam? 4

Assessment: One of the Key Components in RTI Curriculum and Instruction Assessment Curriculum and Instruction is one component, systems is another and then assessment which progress monitoring falls under. School Wide Organization & Problem Solving Systems (Teams, Process, etc) Adapted from Logan City School District, 2002 5

Assessment and Response to Intervention (RTI) A core feature of RTI is identifying a measurement system Screen large numbers of students Identify students in need of additional intervention Monitor students of concern more frequently 1 to 4x per month Typically weekly Diagnostic testing used for instructional planning to help target interventions as needed 2 primary measurement needs; screening and progress monitoring. For today we will focus on progress monitoring. The District has been working to identify a Universal screener. However, finances has slowed us a down a bit. What screeners do you use at your school. Dibels and Aimsweb is an example. Diagnostic testing expertise is also needed, but typically teachers are more familiar with tools and the process for assessment for instructional planning/diagnostic testing as the term is being used here, than with screening and progress monitoring. Screening - Assessments that are administered to determine which children are at risk for reading difficulty and who will need additional intervention. Diagnosis - Assessments that help teachers plan instruction by providing in-depth information about students’ skills and instructional needs. 6 6

Screening Data and Progress Monitoring can be linked The goal is to have a cohesive system. If possible, use the same measures for both screening and progress monitoring (e.g, CBM). Screen ALL students 3x per year (F, W, S) Strategic Support and Monitoring Students at Some Risk What do we mean by cohesive system? Ideally the progress monitoring data should be linked to other types of data and data collection in the school. For example if the school does universal screening(benchmarking) it is very nice if those same measures can be used for progress monitoring. Example Dibels, Aimsweb and Star. What do we mean by CBM General Outcome Measures? See General Outcome Measures CBM module. GOMs can be used to screen all students in the grade level 3x per year and they can also be used to do weekly progress monitoring with students who are at-risk. Note: Other types of information (diagnostic information, high stakes tests, etc) are not good for progress monitoring but obviously also have their own purpose and place in the school’s decision making process Intensive Support & Monitoring for Students at Extreme Risk 7 7

Why Monitor Progress? When teachers USE progress monitoring Students learn more! Teachers design better instructional programs Teacher decision making improves Students become more aware of their performance Safer & Fleishman, 2005 So why do we progress monitor, take a look at these reasons. Call on a participant to read. What are some other reasons progress monitoring is important to use with our students. 8

Why Monitor Progress? “In God we trust… All others must have data.” Dr. Stan Deno Main point- as professional educators, this is an expectation. Dr. Stan Deno is one of the original CBM researchers and a long-time advocate of data-based decision making. Our District has been using data for years to drive instruction. However, more than ever we have to use the data to to instruct our students. Our current leaders focus is on data. 9

Why Monitor Progress? We do NOT KNOW ahead of time whether an intervention will be successful for an individual student Do they ASSUME in the hospital that your heart is working just fine after your bypass surgery? After all… the surgery works well for MOST patients….. Main point- as professionals we use data to make sure a student is benefiting Also makes point that the more worried we are about a student (and the more resources being pumped into instruction) the more we should be sure to monitor progress closely. As an educator you also have to know what adjustments to make, like we saw in the graph earlier. A simple change in the time the intervention was being taught changed the outcomes positively for the student. 10

Why Progress Monitor Frequently? To change what you are doing with a student if it is not working (formative assessment) so you are effective and efficient with your time and instruction To help make decisions about instructional goals, materials, levels, and groups To aid in communication with parents To document progress for special education students as required for periodic and annual reviews Main Point: Frequent Progress Monitoring has many uses, but the MOST important one is the first reason Formative Assessment means you gather information over time and change what you are doing in response to the data Formative assessment The goal of formative assessment is to monitor student learning to provide ongoing feedback that can be used by instructors to improve their teaching and by students to improve their learning. More specifically, formative assessments: Help identify students strengths and weaknesses and target areas that need work Help recognize where students are struggling and address problems immediately Formative assessments are generally low stakes, which means that they have low or no point value. There are many important tools for formative assessment. Such tools can be observation, interaction, and listening. Writing sample- example writes d backwards give student a few words the end and begin with d to see the pattern. Work with the child by giving remedies and continue to monitor change. DRAFT May 27, 2009 Credit: based on slide by Dr. Kim Gibbons, SCRED 11

Change (and Keep Changing) if Instruction isn’t Working… This graph demonstrates that just implementing an intervention does not mean that the student will get better. It gets at the point of how valuable frequent progress monitoring is and shows that if the first couple of interventions were kept in place the rest of the school year, the student probably would not be reading at his/her current level. Credit: SCRED 12

You Can Also Use Monitoring Data to Celebrate Success! Main Point- Data also help us demonstrate when a student is making progress and can also lead to sharing “what worked” with fellow staff. Note: This is a graph of “real” data on phonemic awareness for a K student (weekly phoneme segmentation fluency data)- but not all students make growth that is this dramatic!) This graph was generated in the AIMSweb web-based software program.

What is Progress Monitoring? Standardized measures Reliable Valid Tied to important educational outcomes Given frequently (e.g., weekly) Simple, brief, efficient, and cheap  Sensitive to growth over short periods of time Main point are the 3 bullets- all 3 need to be in place for the type of progress monitoring we are talking about in this module What makes a measure reliable for progress monitoring purposes? Interrater and Test-Retest and Alternate Form consistency (if you have two people score it, if you give it twice within a short period of time or if you give two different probes that are supposed to be the same difficulty you get very similar or identical scores – correlations of .80 or higher, .90+ is gold standard.) What makes a measure valid for progress monitoring purposes? The measure has content and construct validity (measures the domain of interest and has evidence it measures it as well or better than other established measures) and has evidence it is useful specifically for progress monitoring (evidence it is sensitive to interventions and when educators use it to monitor progress and make decisions, students do better) What do we mean by “tied to important educational outcomes”? The progress monitoring measures are indicators of really important outcomes (e.g., becoming an overall really good reader!, passing the state reading “test”) See General Outcome Measure (CBM) ppt. You also should be able to use progress monitoring to aid in RTI decisions about intervention effectiveness and to monitor progress in terms of IEP goals. 14

Short Term (Mastery) and Long Term Progress Monitoring Mastery Monitoring Test subskill mastery and individual lesson effectiveness Ex: Q&A, worksheets following directions unit tests, “hot” reads accuracy, skills “checks” CBA Long Term General Outcome Measures Test retention, generalization and progress toward overall general outcome (reading) Ex: CBM, DIBELS Main Point- Schools need to use long term measurement for high quality progress monitoring When it comes to progress monitoring… can use lots of things to monitor progress. For RTI what you need is at least ONE method of progress monitoring that is a GOM (General Outcome Measure)…. A Term used to describe the “CBM family”… see the General Outcome Measure CBM module for more information Both types of progress monitoring (short term and long term) are very important for the professional teacher. Mastery monitoring includes getting 10 of 10 math facts correct, being able to pick out the key parts of a story for comprehension, completing an in-class writing assignment with all the key writing components, passing the unit test on the Native Americans of NW Minnesota, etc. Mastery monitoring is also needed and can provide good data about whether a student is able to demonstrate understanding about the concepts and skills presented in that lesson or unit. This is very useful information for lesson planning, scaffolding for students, differentiating instruction, etc. However, this type of progress monitoring does not give us good information about whether our students are retaining and generalizing those skills over time to make progress toward long-term important key educational outcomes like becoming successful overall readers. You can lose the forest for the trees… Long term monitoring of student progress allows the teacher to make decisions about whether the intervention is working over time- is the student becoming a better reader, writer, problem solver etc… See next slide for another example of the difference between the two and why monitoring progress in general outcome measures 15

Both Mastery Monitoring and Long Term Progress Monitoring are Important Sometimes mastering subskills doesn't’t generalize to the general outcome or students don’t retain the information over time For example: Melissa is very good at decoding letters and reading individual words, but is not generalizing these skills to reading text with automaticity and comprehension. Adam was really good at using his comprehension strategies and using those when they were working on these skills in class (showed mastery), but when they moved on to another unit he quit using the strategies. Main point- mastery monitoring is great, but you need general outcome measures too.

How Often Do I Need to Monitor Progress? Informally we do this all the time! For small instructional adjustments (repeat the lesson, how much help to give, etc.) In a standardized way to make sure we are “on track” with this student?- depends on level of concern For students in reading who are behind already, monitor progress toward generalized outcome 1 to 4x per month, ideally weekly The facilitator might need to reiterate the difference between GOMs and Mastery Monitoring here so that the students keep in mind that looking at grades from class work does not constitute progress monitoring. 17

How Do You Collect Frequent Progress Monitoring Data? Which students? What measures? What materials? How often? Who collects the data? Where? When? Organizer slide 18

Which students? Students of concern Below target Getting “extra” intervention or help Tier 2 or Tier 3 services Note: it is ok to start “practicing progress monitoring” by just picking a handful of students and collecting data. 19

What measures? www.aimsweb.com dibels.uoregon.edu www.renlearning.com Web-based sources for information on measures as well as access to materials, web-based data management, etc. www.aimsweb.com dibels.uoregon.edu www.renlearning.com If time and internet connect allow- could go online to show students what is available or give them an assignment to do this. 20

What materials? When possible, students are monitored using grade level materials E.g., student reads a different grade level passage or “probe” each week If this is not possible due to frustration “test down” and use the highest grade level of materials possible Periodically “check” how the student is doing on grade level materials and move into grade level materials as soon as possible Instructional level may be different again!!! (Note: Test “down” grade levels with 3 grade level passages at each grade and compare to the grade level targets of the materials. This is referred to as a survey level assessment or SLA) In Math… SLA may look very different….

How Often Do We Monitor Progress? Depends on the sensitivity of the measure and the level of concern we have about the student, but 1 to 4x per month typically For CBM Oral Reading Fluency Weekly with 1 passage (this is most common) Every 3 weeks with 3 passages Typically schools do 1 passage a week and collect data weekly However, some recent research (Jenkins, J, Graff, J., Miglioretti, D. 2009 Exceptional Children 75(2), 151-163) indicated 3 passages every 3 weeks may be an option 22

Who does it? When? Where? Anyone trained in the procedures can collect progress monitoring data Classroom teachers, special education teachers, School support personnel Be creative but careful When and Where? At a time and place that will provide valid information Use common sense Main point- this will depend on resources and physical space, but as long as people are correctly trained and the data are collected in a reasonable place/time it should be fine…. 23

Sharing the Data Just having progress monitoring data is not enough. You need to USE it. Scheduled graph review dates Grade level meetings Problem solving meetings Without scheduled times and support for looking at data, it will not happen. 24

Using Progress Monitoring Data: Is this intervention working? Module 2 has lots of graphs, activities, etc… 25

Avoid Common Mistakes Don’t use the same passage/probe every week!  Have an organized system in place Progress monitoring schedule for students Preprinted passages/probes in a binder An easy way to graph and look at the data Scheduled time to share/look at the data 26

Quiz 1.) What are at least four reasons why teachers should monitor progress? 2) What is the difference between mastery monitoring and frequency progress monitoring toward a general outcome? 3) How often should you monitor progress for students receiving extra help ? Answers: 1.) What are four reasons why teachers should monitor progress? -students learn more -teachers design better instructional programs -teacher decision making improves -students become more aware of their performance -we do not know ahead of time whether an intervention will be successful for an individual student -to change what you are doing with a student if it is not working so you are effective and efficient with your time and instruction -to help make decisions about instructional goals, materials, levels, and groups -to aid in communication with parents -to document progress for special education students as required for periodic and annual reviews -to celebrate success! 2) mastery monitoring can tell you whether a student is mastering or learning subskills or individual lessons over the short term, but does not test for retention and generalization of skills. General outcome progress monitoring tests regularly for progress toward an important more general outcome such as evidence the student is improving in overall reading skill over a long period of time. 3.) How often should you monitor progress for students receiving extra help? 1 to 4x per month, ideally weekly

Quiz (Cont’d) 4.) When possible, students are monitored using… A.) above grade level materials. B.) grade level materials. C.) below grade level materials. 5.) Who can collect progress monitoring data? Answers (cont’d) 4.) When possible, students are monitored using… B.) grade level materials. 5.) Who can collect progress monitoring data? anyone trained in the procedures, e.g. classroom teachers, special education teachers, Title teachers, aids, related services staff, adult volunteers

Quiz (Cont’d) 6.) To have integrity, your data collection must include what? A.) trained data collectors B.) integrity checks/refreshers C.) well chosen measures and materials D.) all of the above Answers (cont’d) 6.) To have integrity, your data must have what? D.) all of the above

Quiz (Cont’d) 7.) True or False? The most important use of frequent progress monitoring is to aid in communication with parents. Answers (Cont’d) 7.) True or False? The most important use of frequent progress monitoring is to aid in communication with parents. -False- the most important reason is to change what you are doing with a student if what you are doing is not working so you are effective and efficient with your time and instruction

Progress Monitoring in Reading: Why, What, and How Adapted from Minnesota State RTI Center Research Institute on Progress Monitoring http://progressmonitoring.net/