Measuring the ‘Intervention Footprint’: Issues of Planning, Documentation, & Follow-Through Jim Wright www.interventioncentral.org

‘Elbow Group’ Activity: Defining ‘Interventions’ In your group, define the term ‘intervention’. Come up with guidelines for judging when teacher activities should be considered ‘interventions’.

RTI Interventions: A Definition (Wright, 2007) “Interventions are specific strategies adopted to help students to make progress toward academic or behavioral goals.” Source: Wright, J. (2007). The RTI toolkit: A practical guide for schools. Port Chester, NY: National Professional Resources, Inc.

Matching Interventions to Appropriate Tier Levels Simple interventions can stand alone as Tier I (classroom) strategies. Simple interventions can also be used in higher Tiers as part of a larger intervention package.

Elements of an Effective Intervention Plan (Grimes & Kurns, 2003) “Intervention design and implementation. Interventions are designed based on [a thorough] analysis, the defined problem, parent input, and professional judgments about the potential effectiveness of interventions. The interventions are described in an intervention plan that includes goals and strategies; a progress monitoring plan; a decision-making plan for summarizing and analyzing progress monitoring data; and responsible parties. Interventions are implemented as developed and modified on the basis of objective data and with the agreement of the responsible parties.” Source: Grimes, J. & Kurns, S. (2003). An intervention-based system for addressing NCLB and IDEA expectations: A multiple tiered model to ensure every child learns. Retrieved on September 23, 2007, from http://www.nrcld.org/symposium2003/grimes/grimes2.html

Key Steps to Academic Intervention Planning Check for student motivation Apply the Instructional Hierarchy Define student academic problems in specific terms Package interventions as teacher-friendly scripts Verify that the intensity of an intervention appropriately matches the current RTI Tier of the student Measure intervention ‘follow-through’ Assemble an ‘Intervention Bank’ to have research-based ideas at hand when needed

Writing Quality ‘Problem Identification’ Statements

Writing Quality ‘Problem Identification’ Statements A frequent problem at RTI Team meetings is that teacher referral concerns are written in vague terms. If the referral concern is not written in explicit, observable, measurable terms, it will be very difficult to write clear goals for improvement or select appropriate interventions. Use this ‘test’ for evaluating the quality of a problem-identification (‘teacher-concern’) statement: Can a third party enter a classroom with the problem definition in hand and know when they see the behavior and when they don’t?

Writing Quality ‘Problem-Identification’ Statements: Template Format for Writing RTI Team Teacher Concerns Conditions when the behavior is observed or absent Description of behavior in concrete, measurable, observable terms During large-group instruction   The student calls out comments that do not relate to the content being taught. When reading aloud The student decodes at a rate much slower than classmates. When sent from the classroom with a pass to perform an errand or take a bathroom break The student often wanders the building instead of returning promptly to class.

Writing Quality ‘Teacher Referral Concern’ Statements: Examples Needs Work: The student is disruptive. Better: During independent seatwork , the student is out of her seat frequently and talking with other students. Needs Work: The student doesn’t do his math. Better: When math homework is assigned, the student turns in math homework only about 20 percent of the time. Assignments turned in are often not fully completed.

Judging the Intensity of Interventions: Tier I, II, or III?

Why Attempt to Judge the ‘Intensity’ of Interventions? Judging the intensity of interventions in advance ensures that intervention plans match the RTI Tier in which they are being used.

Avoiding the ‘Intervention Trap’ When planning Tier II (individualized) interventions, RTI Teams should take care to ensure that those plans are feasible and maintainable in general-education settings. If a Tier II intervention is so ambitious as to resemble a Special Education (Tier III) program, the team may find that the student responds well to the plan but would still lack information about whether the student requires more support than general education can offer. And the plan may not be maintainable!

Intervention Intensity Rating Form (pp. 109-111)

Sample Intervention Intensity Rating Form Items 2.On a per-pupil basis, the cost to purchase or effort needed to create intervention materials Intervention materials not needed or do not entail significant expense or effort Intervention materials required but can be obtained at a modest cost or with reasonable effort Intervention materials per pupil are costly or require substantial effort to create 4. Amount of preparation required for each session of the intervention Little or no preparation is needed Some preparation is needed (up to 15 minutes per session) Substantial preparation is needed (more than 15 minutes per session) 9. Potential of the intervention to distract other students or disrupt their learning Intervention can be implemented with little or no distraction of other students or disruption to their learning Intervention is likely to result in mild distraction of other students or disruption to their learning Intervention is likely to result in significant distraction of other students or disruption to their learning

Intervention Intensity Rating Form Guidelines for Interpreting Results If 7 or more of your ratings on this 10-item form fall under any single Tier, it is likely that the intervention has a level of intensity matching that Tier as well. An intervention with 8 checks under the Tier II column, for example, should be considered a Tier II intervention. If you have a mixed pattern of ratings—with no single column containing 7 or more checks—count up the number of checks in each column. The intervention should be considered equivalent in intensity to the highest column that contains 3 or more checks. (Tier I is the lowest column. Tier III is the highest.) An intervention with more than 3 checks under the Tier III column, for example, would be considered a Tier III intervention.

Evaluating ‘Intervention Follow-Through’ (Treatment Integrity)

Treatment Integrity Activity In your ‘elbow groups’, discuss the following question: How does your school measure the quality of intervention follow-through in classrooms?

What Consultant Factors Can Increase Teacher ‘Intervention Follow-Through’ (DiGennaro et al., 2007) Study contrasted two conditions of teacher support: for behavioral concerns (with four special education teachers participating): Condition 1: Goal Setting and Student Performance Feedback: Teachers set goals for student improvement and received daily written and graphed feedback about student performance. Condition 2: Teacher Performance Feedback and Direct Rehearsal With Meeting Cancellation: Teachers received daily feedback about their own performance in implementing the intervention, as well as student performance feedback. If teachers did not implement the intervention with 100 % integrity, they met with the consultant to ‘practice’ the missed steps. If they carried out the intervention with full integrity, they were able to skip the consultant meeting. Source: DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A comparison of performance feedback procedures on teachers’ treatment implementation integrity and students’ inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis, 40, 447-461.

What Consultant Factors Can Increase Teacher ‘Intervention Follow-Through’ (Cont.) (DiGennaro et al., 2007) Study outcome: Teachers had the highest rates of intervention integrity under Condition 2: Teacher Performance Feedback and Direct Rehearsal With Meeting Cancellation. However, two of four participating teachers rated elements of Teacher Performance Feedback and Direct Rehearsal With Meeting Cancellation condition as unacceptable. The study concluded that “allowing teachers to practice a skill and then avoid meeting with a consultant once skill acquisition in the natural setting is observed appears to be an effective means to promote treatment integrity.” p. 458 Source: DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A comparison of performance feedback procedures on teachers’ treatment implementation integrity and students’ inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis, 40, 447-461.

Why Monitor Intervention Follow-Through? If the RTI Team does not monitor the quality of the intervention follow-through, it will not know how to explain a student’s failure to ‘respond to intervention’. Do qualities within the student explain the lack of academic or behavioral progress? Did problems with implementing the intervention prevent the student from making progress?

What Are Potential Barriers to Assessing Intervention Follow-Through? Direct observation of interventions is the ‘gold standard’ for evaluating the quality of their implementation. However: Teachers being observed may feel that they are being evaluated for global job performance Non-administrative staff may be uncomfortable observing a fellow educator to evaluate intervention follow-through It can be difficult for staff to find time to observe and evaluate interventions as they are being carried out

Supplemental Ideas to Collect Information About Classroom Implementation of Interventions Assign a ‘case manager’ from the RTI Intervention Team to check in with the teacher within a week of the initial meeting to see how the intervention is going. Have the teacher use a data tool to collect information about the student’s response to intervention (e.g., Daily Behavior Report Card) or about the implementation of the intervention itself (e.g. Teacher Intervention Evaluation Log) Include a scripted question at the RTI Intervention Team Follow-Up Meeting that explicitly asks the referring teacher or instructional team to provide details about the implementation of the intervention. Leave a notebook in the classroom for the teacher to jot down any questions or concerns about the intervention. Assign an RTI Team member to stop by the classroom periodically to check the notebook and respond to any concerns noted.

Intervention Script Builder: pp. 107-108

Teacher Intervention Evaluation Log: p.112

END