RTI Module 3 Data Analysis and Team Problem Solving Process Marika Ginsburg- Block, Ph.D. University of Delaware marika@udel.edu Todd Gravois, Ph.D. University of Maryland tgravois@verizon.net And The Delaware Department of Education

Agenda: 8:00-8:30 Registration/ Networking 8:30-9:00 Welcome/ Homework Review 9:00- 10:15 Problem Solving as a Core Skill RTI 10:30- 11:30 Demonstration of Effective Problem Solving at Tier 1 11:30-12:00 Role Alike Activity 12:00-12:45 Working LUNCH- with District Teams 12:45- 1:30 Data Analysis & Decision Making within Problem Solving 1:30- 2:30 Case Study with Application of Problem Solving & Analysis 2:30- 3:15 District Team Activity 3:15- 3:30 Homework Assignment 3:30- 3:50 RTI Framework Debriefing 3:50- 4:00 Evaluation

Purpose of Training Modules RTI is not a program; RTI is a process State provides a framework and process Through regulations Through professional development Through technical assistance District/School develops unique implementation RTI implementation must take into consideration unique characteristics of local culture RTI should build on existing systems/initiatives District Leadership Diversity in contribution/input/skills Distribution in workload Everyone is knowledgeable and supportive Share framework and provide guidance for school implementation

RTI Framework Where have we been? Where are we going? Established District Leadership Team Overview of RTI Needs Assessment of District and School Level Implementation Tier 1 Framework Assessment Framework Assessments (December) Data Management (December) Where are we going? Team Problem Solving (February) Data Analysis (February) Interventions (April) SLD Determination (May) Secondary Focus (2008-2009)

Three -Tier Model of School Supports Decisions for those students not benefiting from interventions Quality Classroom Instruction Quality/ Individualized/ Documented/ Interventions Intensive Quality Instruction Decisions for those students not benefiting from the core curriculum 5

Let’s Be Clear….. Individualized problem solving and decision making is critical … but not synonymous with how programming is delivered. Individualized does not equate to individual instruction or individual interventions.

Three major considerations for problem solving and data analysis: Where in the RTI process should problem solving occur? What constitutes effective problem solving? How does data analysis relate to effective problem solving

Where should problem solving and data analysis focus? There will be times when problem solving and data analysis will be helpful in making decisions to identify students needing more assistance. Decisions for those students not benefiting from interventions Quality Classroom Instruction Quality/ Individualized/ Documented/ Interventions Intensive Quality Instruction Decisions for those students not benefiting from the core curriculum 8

Decisions for those students not benefiting from the core curriculum However, there will be times when problem solving and data analysis will be critical in determining the quality of instruction being delivered. Decisions for those students not benefiting from the core curriculum Decisions for those students not benefiting from interventions Quality Classroom Instruction Quality/ Individualized/ Documented/ Interventions Intensive Quality Instruction 9

“Fundamentally, RTI assumes that producing changes in student responding over time is important, which establishes the instructional task for educators. School-based intervention teams must be able to ensure that instruction is adapted correctly to students’ level of skill proficiency.” Daly, et al., 2007

Individualized/ Documented/ Quality Classroom Instruction There will be times when problem solving and data analysis will be critical in determining the quality of instruction being delivered. Intensive Quality Instruction Quality/ Individualized/ Documented/ Interventions Same as slide 4. LD Quality Classroom Instruction 11 11

Individualized/ Documented/ Quality Classroom Instruction Intensive Quality Instruction Quality/ Individualized/ Documented/ Interventions Same as slide 4. LD Quality Classroom Instruction 12 12

Individualized/ Documented/ Quality Classroom Instruction So, what is “quality” instruction? Intensive Quality Instruction Quality/ Individualized/ Documented/ Interventions Quality Classroom Instruction How do we define quality instruction? 13 13

Student Instruction Task Multiple Influences on Learning: (Gravois, Gickling & Rosenfield, 1999) (50-60%) Prior Knowledge Student Match=Success Instruction Task (25-35%) (5-15%)

Influences on Learning: High Achievers (80-90%) Prior Knowledge Student Match=Success Instruction Task (5-10%) (5-10%) 15

Influences on Learning: Low Achievers (10-20%) Prior Knowledge Student Match=Success Instruction Task (40-45%) (40-45%) 16

Importance of Task Long raidans were forming when Matthew arrived. He tried to phindate the amount of time it would take to get to the cornvorster. Vort it would be too long, plast he would miss the game. He varaxated for a moment until the raidans became even longer. He decided that he would ordrul in the raidan opet see vort it would start moving more expedititiously. No sooner had he started fleedjuul, when it began mostulag quite hard. Matthew became disgusted, zipped up his ornaforger, then walked back to his car. He drove home in the mostul. By the time he put the car in the garage, the mostul was droim poet the faetos was out. Matthew was doubly disgusted now. Suddenly, he went inside to watch the game. He turned on the television set but nothing happened. Matthew said to himself, “what a lousy frol”.

Three major considerations for problem solving and data analysis: Where in the RTI process should problem solving occur? What constitutes effective problem solving? How does data analysis relate to effective problem solving

Activity 1: What is effective problem solving? Think-Write-Share Working individually first think about how you approach a problem. Quickly write the steps of problem solving you typically follow-- try to be specific Share with a partner from your school. Compare and contrast. What are the implications of varied approaches to problem solving?

Researched Characteristics of Effective Problem Solving Explicit and Observable Descriptions Collection of Baseline Data Specification of Measurable Goals Specification of Intervention Components Integrity of Intervention Implementation Graphing of Data to Monitor Progress Evaluation of Intervention by Comparing Graphed Data to Baseline/ Goals Donovon & Cross, 2002

Researched Characteristics of Effective Problem Solving (cont.) Problems are best defined as a discrepancy between a desired state and what is occurring. When students are experiencing problems, one key to finding effective instructional approaches is to conduct analysis of why we believe students are not proficient. This analysis must focus on instructionally relevant and changeable variables. A plan is developed that is Goal directed Based on an analysis of the problem Identifies specific What? When? and How? Specifies method for monitoring progress Documents the plan Fits the resources and values of people in the setting Progress is monitored frequently and repeatedly, changes are timely, plan is adjusted based upon data. NASDSE 2005

Researched Characteristics of Effective Problem Solving (cont.) Increased specificity and integrity of the process linked to greater goal attainment (Levinsohn & Rosenfield, 2000) Increased accuracy and integrity of documenting the problem solving process the more likely to achieve the goals established (Fudell, 1992; LaFleur & Rosenfield, 2005)

Researched Characteristics of Effective Problem Solving (cont.) “It is important to ensure that all factors (e.g., curriculum, effective instruction, school and classroom environment) have been examined prior to assuming that a student factor (or disability) are responsible for student performance.” NASDSE 2005

Problem Solving Process Problem Identification & Problem Analysis Strategy/ Intervention Design Strategy/ Intervention Implementation Strategy/ Intervention Evaluation Follow-up and Re-design

Problem Solving Process Problem Identification & Problem Analysis Specific and observable concerns Assessment of instructional conditions Data Analysis Principles Prioritized/ Targeted Concerns Baseline Data (direct assessment of concern) Specific and measurable goal Charting and graphing of data

Problem Solving Process Problem Identification & Problem Analysis Strategy/ Intervention Design Academic: Conducted under instructionally matched conditions Effective instructional practices (modeling, repetition, corrective feedback, incentives for improvement) Plan for progress monitoring Behavior: Application of researched behavior principles Contingency management

Problem Solving Process Problem Identification & Problem Analysis Strategy/ Intervention Design Strategy/ Intervention Implementation Implementation integrity must be considered Note: Less than two-thirds of teachers implemented the planned strategy past 2 weeks.

Problem Solving Process Problem Identification & Problem Analysis Strategy/ Intervention Design Strategy/ Intervention Implementation Strategy/ Intervention Evaluation Charting and graphing of data (at least weekly) Continued comparison of data with baseline and goals

Problem Solving Process Problem Identification & Problem Analysis Strategy/ Intervention Design Strategy/ Intervention Implementation Strategy/ Intervention Evaluation Follow-up and Re-design - Recognition that refinement and “tweaking” are necessary parts of effective problem solving

Activity 2: Problem Solving Process Using the criteria for effective problem solving, listen to the case presentation. Note those components of effective problem solving that were addressed and which were not addressed. For any aspects not addressed, note how might those tasks be accomplished. Examining Problem Solving

Afternoon Agenda What’s still to come? Cases taken across the three-tiered model District team planning time How to evaluate problem solving in your district?

Activity 3: Role Alike Reflection Specific to problem solving and data analysis, what skills/ expertise does your training/ position bring? What skills/ expertise do you think will be required for effective problem solving and data analysis? What role do we play currently? How do we envision our role in how RTI will be implemented in our district? How will we be involved in maintaining integrity/ fidelity to quality instruction?

Lunch time! During your working lunch, please: Follow up with your district team Share what you learned from the Role Alike Activity

Three major considerations for problem solving and data analysis: Where in the RTI process should problem solving occur? What constitutes effective problem solving? How does data analysis relate to effective problem solving?

Agenda: 8:00-8:30 Registration/ Networking 8:30-9:00 Welcome/ Homework Review 9:00- 10:15 Problem Solving as a Core Skill RTI 10:30- 11:30 Demonstration of Effective Problem Solving at Tier 1 11:30-12:00 Role Alike Activity 12:00-12:45 Working LUNCH- with District Teams 12:45- 1:30 Data Analysis & Decision Making within Problem Solving 1:30- 2:30 Case Study with Application of Problem Solving & Analysis 2:30- 3:15 District Team Activity 3:15- 3:30 Homework Assignment 3:30- 3:50 RTI Framework Debriefing 3:50- 4:00 Evaluation

How does Data Analysis Relate to Effective Problem Solving Problem Identification & Problem Analysis Specific and observable concerns Assessment of instructional conditions Data Analysis Principles Prioritized/ Targeted Concerns Baseline Data (direct assessment of concern) Specific and measurable goal Charting and graphing of data Strategy/ Intervention Design Academic: Conducted under instructionally matched conditions - Effective instructional practices (modeling, repetition, corrective feedback, incentives for improvement) - Plan for progress monitoring Behavior: Conducted under instructionally matched conditions - Application of researched behavior principles - Contingency management - Plan for progress monitoring Strategy/ Intervention Implementation Implementation integrity must be considered Note: Less than two-thirds of teachers implemented the planned strategy past 2 weeks. Strategy/ Intervention Evaluation Charting and graphing of data (at least weekly) Continued comparison of data with baseline and goals Follow-up and Re-design - Recognition that refinement and “tweaking” are necessary parts of effective problem solving

How does Data Analysis Relate to Effective Problem Solving Problem Identification & Problem Analysis Specific and observable concerns Assessment of instructional conditions Data Analysis Principles Prioritized/ Targeted Concerns Baseline Data (direct assessment of concern) Specific and measurable goal Charting and graphing of data Strategy/ Intervention Design Academic: Conducted under instructionally matched conditions - Effective instructional practices (modeling, repetition, corrective feedback, incentives for improvement) - Plan for progress monitoring Behavior: Conducted under instructionally matched conditions - Application of researched behavior principles - Contingency management - Plan for progress monitoring Strategy/ Intervention Implementation Implementation integrity must be considered Note: Less than two-thirds of teachers implemented the planned strategy past 2 weeks. Strategy/ Intervention Evaluation Charting and graphing of data (at least weekly) Continued comparison of data with baseline and goals Follow-up and Re-design - Recognition that refinement and “tweaking” are necessary parts of effective problem solving

Guiding Principles Data Analysis: Occurs at all Tiers and Stages of Problem Solving Must match the purpose Involves use of multiple sources of data Is only good as the measure being used Requires data tools and involves multiple techniques Involves thoughtful application of decision rules Depends upon the skills and expertise of the professional making decisions Results in hypotheses, not certainties (Sources: Heartland AEA (2007))

Activity 4: Case of Ms. Brown Applying Effective Problem Solving and Analysis: Case Study (Tiers 1 - 3)

Case of Ms. Brown: Background: Part 1 RTI Regulations Tier 1 screenings conducted 3x annually; 1st screening within 2 weeks of school Children at or below 25th%ile will receive Tier 2 services Children above 25th%ile but below benchmark shall receive differentiated, needs-based instruction; progress monitored every 2 weeks until on trajectory; after 6 weeks Tier 2 or continued monitoring Assessment Tools used in this case Oral Reading Fluency Nonsense Word Fluency Data analytic terms 25th %ile & benchmarks Level of performance & rate of improvement Dual discrepancy

Part 1- Small Group Discussion Questions: Data on page 4, Questions on page 5 (only work on Q. 1 &2) 1. What hypotheses do you have about the basis of the student difficulties illustrated here? 2. What data would you need to test each of your hypotheses (complete the chart that follows)?

Case of Ms. Brown: Background: Part 2 RTI Regulations Tier 2 delivered in small group setting, 3 times per week for 30 minutes for 6 weeks, weekly monitoring against benchmarks After 6 weeks IST reviews progress, including fidelity of implementation, pacing, appropriateness of instructional groupings Team may request additional assessments, modify Intervention and continue Tier 2 for 6 more weeks, determine Tier 3 intervention is required

Part 2- Small Group Discussion Questions: Data on pages 7-8, Questions on page 9 (only work on Q. 1 &2) 1. Based on the progress monitoring data provided, describe the effectiveness of Tier 2 instruction for each of these three students. Consider student level of performance on CBM measures (as compared to the benchmarks) and rate of improvement towards the benchmarks (i.e. their actual trendline compared to their aimline). 2. Based on the intervention fidelity calendar, describe the implementation fidelity of this Tier 2 instruction. Is there additional data that you’d like to see?

Case of Ms. Brown: Background: Part 3 RTI Regulations After total of 12 weeks of Tier 2 intervention and no progress or not on trajectory toward benchmarks child shall begin receiving Tier 3 intervention Tier 3 delivered in smaller group settings, at least 5x per week for 30 min, for 6 wks, weekly monitoring After 6 wks of Tier 3 and no progress IST shall refer for initial evaluation Review applicable regs, assessments used & terms for data analysis necessary for each of the 3 Parts of the case: Review & example for Part 1 Teams work on Part 1 Review & Example for Part 2 Teams work on Part 2 Review and Example for Part 3 Teams work on part 3

Case of Ms. Farmer: Background: Part 3 -continued RTI Regulations After 6 weeks and not on trajectory IST shall assure differentiated needs based instruction received, including review of fidelity, pacing, and groupings; Team may request additional assessments, modify intervention and continue Tier 3 for 6 more wks, or determine initial eval is required After 12 wks of Tier 3 and not on trajectory, refer for initial evaluation Review applicable regs, assessments used & terms for data analysis necessary for each of the 3 Parts of the case: Review & example for Part 1 Teams work on Part 1 Review & Example for Part 2 Teams work on Part 2 Review and Example for Part 3 Teams work on part 3

Part 3- Small Group Discussion Questions: No extra data, Questions on page 11 (only work on Q. 1 &2) 1. What steps were taken to identify the problem, analyze the problem, design a strategy to address the problem, evaluate the strategy, and follow-up? 2. How are these steps consistent with or different from the data-based components of problem solving that we’ve discussed today?

Activity 5: District Teams What training needs do you have in order to implement the Problem Solving Process district wide? Complete Activity 5: To What Degree Does My School/District have a Functional Problem Solving Team?

Homework Preview Observe school-based teams in action, interview staff Use the Homework tool provided to evaluate the status of your teams.

RTI Framework Where have we been? Where are we going? Established District Leadership Team Overview of RTI Needs Assessment of District and School Level Implementation Tier 1 Framework Assessment Framework Assessments (December) Data Management (December) Team Problem Solving (February) Data Analysis (February) Where are we going? Interventions (April) SLD Determination (May) Secondary Focus (2008-2009)

Session Evaluation Thank you!

Selected Bibliography Batsche, G., et al., (2005). Response to intervention: Policy considerations and implementation. Alexandris, VA: National Association of State Directors of Special Education. Costas, L., Rosenfield, S., & Gravois, T.A., (2003) . Impact of instructional consultation on teacher satisfaction and skill development. Poster session presented at the American Psychological Association Meeting, Toronto. Daly, E., Martens, B., Barnett, D., Witt, J., & Olson, S. (2007). Varying intervention delivery in response to intervention. School Psychology Review, 36. 562-581. Donovan, S. & Cross, C. (Eds.) (2002). Minority students in special education: Committee on minority representation in special education. Washington, DC: National Academy Press. Flugum, K. R. & Reschly, D. J. (1994). Prereferral interventions: Quality indices and outcomes. Journal of School Psychology, 32(1), 1-14. Gravois, Knotek & Babinski (2002). Educating practitioners as consultants: Development and Implementation of the IC-Team Consortium. Journal of Educational and Psychological Consultation, 13, 113-132. Gravois & Gickling (2008). Best practices in instructional assessment. In A. Thomas & J. Grimes (Eds.) Best practices in school psychology, Vol. V. Bethesda, MD: NASP Gravois & Rosenfield (2002). A multi-dimensional framework for evaluation of instructional consultation teams. Journal of Applied School Psychology, 19 (1) 5-29.

Selected Bibliography, Cont. Gravois & Rosenfield (2006). Impact of Instructional Consultation Teams on the Disproportionate Placement of Minority Students in Special Education. Journal of Remedial and Special Education. Knotek, S., Rosenfield, S, Babinski, L. & Gravois, T.A. (2003). The process of fostering consultee development during instructional consultation. Journal of Educational and Psychological Consultation. 14, 303-328 Levinsohn & Rosenfield (2005). Evaluating instructional consultation teams for student reading achievement and special education outcomes. Unpublished manuscript. Rosenfield (2002) Best practices in instructional consultation. In A. Thomas & J. Grimes (Eds. Best practices in school psychology IV. Bethesda, MD: NASP. Rosenfield (1987) Instructional consultation.Mahwah, NJ: Erlbaum. Rosenfield & Gravois (1996). Instructional Consultation Teams: Collaborating for change. New York: Guilford. Rosenfield & Gravois (1999). Working with teams in the school. In C. Reynolds and T. Gutkin (Eds) Handbook of school psychology. John Wiley: New York.

Additional Resources http://eff.csuchico.edu/download_center Education for the Future- California State Chico campus initiative; free downloads, case studies Data Tools For School Improvement (Bernhardt) http://www.centeroninstruction.org/files/rti_role_definitions.pdf New Roles in Response to Intervention: Creating Success for Schools and Children   http://www.centeroninstruction.org/files/ImplementationOfRtI.pdf PowerPoint by Doug Marston re. implementation of Problem Solving Model in Minneapolis Public Schools-- includes example of effects on referrals, time spent in assessment by school psychologists, examples of barriers & strategies for successful implementation, example of data management interface