1. RTI Module 3 Data Analysis and Team Problem Solving Process
Marika Ginsburg- Block, Ph.D.
University of Delaware
Todd Gravois, Ph.D.
University of Maryland
And
The Delaware Department of Education

2. Agenda:
8:00-8:30 Registration/ Networking
8:30-9:00 Welcome/ Homework Review
9:00- 10:15 Problem Solving as a Core Skill RTI
10:30- 11:30 Demonstration of Effective Problem Solving at Tier 1
11:30-12:00 Role Alike Activity
12:00-12:45 Working LUNCH- with District Teams
12:45- 1:30 Data Analysis & Decision Making within Problem Solving
1:30- 2:30 Case Study with Application of Problem Solving & Analysis
2:30- 3:15 District Team Activity
3:15- 3:30 Homework Assignment
3:30- 3:50 RTI Framework Debriefing
3:50- 4:00 Evaluation

3. Purpose of Training Modules
RTI is not a program; RTI is a process
State provides a framework and process
Through regulations
Through professional development
Through technical assistance
District/School develops unique implementation
RTI implementation must take into consideration unique characteristics of local culture
RTI should build on existing systems/initiatives
District Leadership
Diversity in contribution/input/skills
Distribution in workload
Everyone is knowledgeable and supportive
Share framework and provide guidance for school implementation

4. RTI Framework Where have we been? Where are we going?
Established District Leadership Team
Overview of RTI
Needs Assessment of District and School Level Implementation
Tier 1 Framework
Assessment Framework
Assessments (December)
Data Management (December)
Where are we going?
Team Problem Solving (February)
Data Analysis (February)
Interventions (April)
SLD Determination (May)
Secondary Focus ( )

5. Three -Tier Model of School Supports
Decisions for those students not benefiting from interventions
Quality Classroom Instruction
Quality/
Individualized/ Documented/
Interventions
Intensive
Quality
Instruction
Decisions for those students not benefiting from the core curriculum 5

6. Let’s Be Clear…..
Individualized problem solving and decision making is critical … but not synonymous with how programming is delivered.
Individualized does not equate to individual instruction or individual interventions.

7. Three major considerations for problem solving and data analysis:
Where in the RTI process should problem solving occur?
What constitutes effective problem solving?
How does data analysis relate to effective problem solving

8. Where should problem solving and data analysis focus?
There will be times when problem solving and data analysis will be helpful in making decisions to identify students needing more assistance.
Decisions for those students not benefiting from
interventions
Quality Classroom Instruction
Quality/
Individualized/ Documented/
Interventions
Intensive
Quality
Instruction
Decisions for those students not benefiting from the core curriculum 8

9. Decisions for those students not benefiting from the core curriculum
However, there will be times when problem solving and data analysis will be critical in determining the quality of instruction being delivered.
Decisions for those students not benefiting from the core curriculum
Decisions for those students not benefiting from interventions
Quality Classroom Instruction
Quality/
Individualized/ Documented/
Interventions
Intensive
Quality
Instruction 9

10. “Fundamentally, RTI assumes that producing changes in student responding over time is important, which establishes the instructional task for educators.
School-based intervention teams must be able to ensure that instruction is adapted correctly to students’ level of skill proficiency.”
Daly, et al., 2007

11. Individualized/ Documented/ Quality Classroom Instruction
There will be times when problem solving and data analysis will be critical in determining the quality of instruction being delivered.
Intensive
Quality
Instruction
Quality/
Individualized/ Documented/
Interventions
Same as slide 4. LD
Quality Classroom Instruction 11 11

12. Individualized/ Documented/ Quality Classroom Instruction
Intensive
Quality
Instruction
Quality/
Individualized/ Documented/
Interventions
Same as slide 4. LD
Quality Classroom Instruction 12 12

13. Individualized/ Documented/ Quality Classroom Instruction
So, what is “quality”
instruction?
Intensive
Quality
Instruction
Quality/
Individualized/ Documented/
Interventions
Quality Classroom Instruction
How do we define quality instruction? 13 13

14. Student Instruction Task
Multiple Influences on Learning: (Gravois, Gickling & Rosenfield, 1999)
(50-60%)
Prior
Knowledge
Student
Match=Success
Instruction
Task
(25-35%)
(5-15%)

15. Influences on Learning: High Achievers
(80-90%)
Prior
Knowledge
Student
Match=Success
Instruction
Task
(5-10%)
(5-10%) 15

16. Influences on Learning: Low Achievers
(10-20%)
Prior
Knowledge
Student
Match=Success
Instruction
Task
(40-45%)
(40-45%) 16

17. Importance of Task
Long raidans were forming when Matthew arrived. He tried to phindate the amount of time it would take to get to the cornvorster. Vort it would be too long, plast he would miss the game. He varaxated for a moment until the raidans became even longer. He decided that he would ordrul in the raidan opet see vort it would start moving more expedititiously.
No sooner had he started fleedjuul, when it began mostulag quite hard. Matthew became disgusted, zipped up his ornaforger, then walked back to his car. He drove home in the mostul. By the time he put the car in the garage, the mostul was droim poet the faetos was out. Matthew was doubly disgusted now. Suddenly, he went inside to watch the game. He turned on the television set but nothing happened. Matthew said to himself, “what a lousy frol”.

18. Three major considerations for problem solving and data analysis:
Where in the RTI process should problem solving occur?
What constitutes effective problem solving?
How does data analysis relate to effective problem solving

19. Activity 1: What is effective problem solving?
Think-Write-Share
Working individually first think about how you approach a problem.
Quickly write the steps of problem solving you typically follow-- try to be specific
Share with a partner from your school. Compare and contrast.
What are the implications of varied approaches to problem solving?

20. Researched Characteristics of Effective Problem Solving
Explicit and Observable Descriptions
Collection of Baseline Data
Specification of Measurable Goals
Specification of Intervention Components
Integrity of Intervention Implementation
Graphing of Data to Monitor Progress
Evaluation of Intervention by Comparing Graphed Data to Baseline/ Goals
Donovon & Cross, 2002

21. Researched Characteristics of Effective Problem Solving (cont.)
Problems are best defined as a discrepancy between a desired state and what is occurring.
When students are experiencing problems, one key to finding effective instructional approaches is to conduct analysis of why we believe students are not proficient. This analysis must focus on instructionally relevant and changeable variables.
A plan is developed that is
Goal directed
Based on an analysis of the problem
Identifies specific What? When? and How?
Specifies method for monitoring progress
Documents the plan
Fits the resources and values of people in the setting
Progress is monitored frequently and repeatedly, changes are timely, plan is adjusted based upon data.
NASDSE 2005

22. Researched Characteristics of Effective Problem Solving (cont.)
Increased specificity and integrity of the process linked to greater goal attainment (Levinsohn & Rosenfield, 2000)
Increased accuracy and integrity of documenting the problem solving process the more likely to achieve the goals established (Fudell, 1992; LaFleur & Rosenfield, 2005)

23. Researched Characteristics of Effective Problem Solving (cont.)
“It is important to ensure that all factors (e.g., curriculum, effective instruction, school and classroom environment) have been examined prior to assuming that a student factor (or disability) are responsible for student performance.”
NASDSE 2005

24. Problem Solving Process
Problem Identification & Problem Analysis
Strategy/ Intervention Design
Strategy/ Intervention Implementation
Strategy/ Intervention Evaluation
Follow-up and Re-design

25. Problem Solving Process
Problem Identification & Problem Analysis
Specific and observable concerns
Assessment of instructional conditions
Data Analysis Principles
Prioritized/ Targeted Concerns
Baseline Data (direct assessment of concern)
Specific and measurable goal
Charting and graphing of data

26. Problem Solving Process
Problem Identification & Problem Analysis
Strategy/ Intervention Design
Academic:
Conducted under instructionally matched conditions
Effective instructional practices (modeling, repetition, corrective feedback, incentives for improvement)
Plan for progress monitoring
Behavior:
Application of researched behavior principles
Contingency management

27. Problem Solving Process
Problem Identification & Problem Analysis
Strategy/ Intervention Design
Strategy/ Intervention Implementation
Implementation integrity must be considered
Note: Less than two-thirds of teachers implemented the planned strategy past 2 weeks.

28. Problem Solving Process
Problem Identification & Problem Analysis
Strategy/ Intervention Design
Strategy/ Intervention Implementation
Strategy/ Intervention Evaluation
Charting and graphing of data (at least weekly)
Continued comparison of data with baseline and goals

29. Problem Solving Process
Problem Identification & Problem Analysis
Strategy/ Intervention Design
Strategy/ Intervention Implementation
Strategy/ Intervention Evaluation
Follow-up and Re-design
- Recognition that refinement and “tweaking” are necessary parts of effective problem solving

30. Activity 2: Problem Solving Process
Using the criteria for effective problem solving, listen to the case presentation.
Note those components of effective problem solving that were addressed and which were not addressed.
For any aspects not addressed, note how might those tasks be accomplished.
Examining
Problem Solving

38. Afternoon Agenda What’s still to come?
Cases taken across the three-tiered model
District team planning time
How to evaluate problem solving in your district?

39. Activity 3: Role Alike Reflection
Specific to problem solving and data analysis, what skills/ expertise does your training/ position bring?
What skills/ expertise do you think will be required for effective problem solving and data analysis?
What role do we play currently? How do we envision our role in how RTI will be implemented in our district?
How will we be involved in maintaining integrity/ fidelity to quality instruction?

40. Lunch time! During your working lunch, please:
Follow up with your district team
Share what you learned from the Role Alike Activity

41. Three major considerations for problem solving and data analysis:
Where in the RTI process should problem solving occur?
What constitutes effective problem solving?
How does data analysis relate to effective problem solving?

42. Agenda:
8:00-8:30 Registration/ Networking
8:30-9:00 Welcome/ Homework Review
9:00- 10:15 Problem Solving as a Core Skill RTI
10:30- 11:30 Demonstration of Effective Problem Solving at Tier 1
11:30-12:00 Role Alike Activity
12:00-12:45 Working LUNCH- with District Teams
12:45- 1:30 Data Analysis & Decision Making within Problem Solving
1:30- 2:30 Case Study with Application of Problem Solving & Analysis
2:30- 3:15 District Team Activity
3:15- 3:30 Homework Assignment
3:30- 3:50 RTI Framework Debriefing
3:50- 4:00 Evaluation

43. How does Data Analysis Relate to Effective
Problem Solving
Problem Identification & Problem Analysis
Specific and observable concerns
Assessment of instructional conditions
Data Analysis Principles
Prioritized/ Targeted Concerns
Baseline Data (direct assessment of concern)
Specific and measurable goal
Charting and graphing of data
Strategy/ Intervention Design
Academic:
Conducted under instructionally matched conditions - Effective instructional practices (modeling, repetition, corrective feedback, incentives for improvement) - Plan for progress monitoring
Behavior:
Conducted under instructionally matched conditions - Application of researched behavior principles - Contingency management - Plan for progress monitoring
Strategy/ Intervention Implementation
Implementation integrity must be considered
Note: Less than two-thirds of teachers implemented the planned strategy past 2 weeks.
Strategy/ Intervention Evaluation
Charting and graphing of data (at least weekly)
Continued comparison of data with baseline and goals
Follow-up and Re-design
- Recognition that refinement and “tweaking” are necessary parts of effective problem solving

44. How does Data Analysis Relate to Effective Problem Solving
Problem Identification & Problem Analysis
Specific and observable concerns
Assessment of instructional conditions
Data Analysis Principles
Prioritized/ Targeted Concerns
Baseline Data (direct assessment of concern)
Specific and measurable goal
Charting and graphing of data
Strategy/ Intervention Design
Academic:
Conducted under instructionally matched conditions - Effective instructional practices (modeling, repetition, corrective feedback, incentives for improvement) - Plan for progress monitoring
Behavior:
Conducted under instructionally matched conditions - Application of researched behavior principles - Contingency management - Plan for progress monitoring
Strategy/ Intervention Implementation
Implementation integrity must be considered
Note: Less than two-thirds of teachers implemented the planned strategy past 2 weeks.
Strategy/ Intervention Evaluation
Charting and graphing of data (at least weekly)
Continued comparison of data with baseline and goals
Follow-up and Re-design
- Recognition that refinement and “tweaking” are necessary parts of effective problem solving

45. Guiding Principles Data Analysis:
Occurs at all Tiers and Stages of Problem Solving
Must match the purpose
Involves use of multiple sources of data
Is only good as the measure being used
Requires data tools and involves multiple techniques
Involves thoughtful application of decision rules
Depends upon the skills and expertise of the professional making decisions
Results in hypotheses, not certainties
(Sources: Heartland AEA (2007))

46. Activity 4: Case of Ms. Brown
Applying Effective Problem Solving and Analysis: Case Study (Tiers 1 - 3)

47. Case of Ms. Brown: Background: Part 1
RTI Regulations
Tier 1 screenings conducted 3x annually; 1st screening within 2 weeks of school
Children at or below 25th%ile will receive Tier 2 services
Children above 25th%ile but below benchmark shall receive differentiated, needs-based instruction; progress monitored every 2 weeks until on trajectory; after 6 weeks Tier 2 or continued monitoring
Assessment Tools used in this case
Oral Reading Fluency
Nonsense Word Fluency
Data analytic terms
25th %ile & benchmarks
Level of performance & rate of improvement
Dual discrepancy

48. Part 1- Small Group Discussion Questions:
Data on page 4, Questions on page 5 (only work on Q. 1 &2)
1. What hypotheses do you have about the basis of the student difficulties illustrated here?
2. What data would you need to test each of your hypotheses (complete the chart that follows)?

49. Case of Ms. Brown: Background: Part 2
RTI Regulations
Tier 2 delivered in small group setting, 3 times per week for 30 minutes for 6 weeks, weekly monitoring against benchmarks
After 6 weeks IST reviews progress, including fidelity of implementation, pacing, appropriateness of instructional groupings
Team may request additional assessments, modify Intervention and continue Tier 2 for 6 more weeks, determine Tier 3 intervention is required

50. Part 2- Small Group Discussion Questions:
Data on pages 7-8, Questions on page 9 (only work on Q. 1 &2)
1. Based on the progress monitoring data provided, describe the effectiveness of Tier 2 instruction for each of these three students. Consider student level of performance on CBM measures (as compared to the benchmarks) and rate of improvement towards the benchmarks (i.e. their actual trendline compared to their aimline).
2. Based on the intervention fidelity calendar, describe the implementation fidelity of this Tier 2 instruction. Is there additional data that you’d like to see?

51. Case of Ms. Brown: Background: Part 3
RTI Regulations
After total of 12 weeks of Tier 2 intervention and no progress or not on trajectory toward benchmarks child shall begin receiving Tier 3 intervention
Tier 3 delivered in smaller group settings, at least 5x per week for 30 min, for 6 wks, weekly monitoring
After 6 wks of Tier 3 and no progress IST shall refer for initial evaluation
Review applicable regs, assessments used & terms for data analysis necessary for each of the 3 Parts of the case:
Review & example for Part 1
Teams work on Part 1
Review & Example for Part 2
Teams work on Part 2
Review and Example for Part 3
Teams work on part 3

52. Case of Ms. Farmer: Background: Part 3 -continued
RTI Regulations
After 6 weeks and not on trajectory IST shall assure differentiated needs based instruction received, including review of fidelity, pacing, and groupings; Team may request additional assessments, modify intervention and continue Tier 3 for 6 more wks, or determine initial eval is required
After 12 wks of Tier 3 and not on trajectory, refer for initial evaluation
Review applicable regs, assessments used & terms for data analysis necessary for each of the 3 Parts of the case:
Review & example for Part 1
Teams work on Part 1
Review & Example for Part 2
Teams work on Part 2
Review and Example for Part 3
Teams work on part 3

53. Part 3- Small Group Discussion Questions:
No extra data, Questions on page 11 (only work on Q. 1 &2)
1. What steps were taken to identify the problem, analyze the problem, design a strategy to address the problem, evaluate the strategy, and follow-up?
2. How are these steps consistent with or different from the data-based components of problem solving that we’ve discussed today?

54. Activity 5: District Teams
What training needs do you have in order to implement the Problem Solving Process district wide?
Complete Activity 5: To What Degree Does My School/District have a Functional Problem Solving Team?

55. Homework Preview Observe school-based teams in action, interview staff
Use the Homework tool provided to evaluate the status of your teams.

56. RTI Framework Where have we been? Where are we going?
Established District Leadership Team
Overview of RTI
Needs Assessment of District and School Level Implementation
Tier 1 Framework
Assessment Framework
Assessments (December)
Data Management (December)
Team Problem Solving (February)
Data Analysis (February)
Where are we going?
Interventions (April)
SLD Determination (May)
Secondary Focus ( )

57. Session Evaluation
Thank you!

58. Selected Bibliography
Batsche, G., et al., (2005). Response to intervention: Policy considerations and implementation. Alexandris, VA: National Association of State Directors of Special Education.
Costas, L., Rosenfield, S., & Gravois, T.A., (2003) . Impact of instructional consultation on teacher satisfaction and skill development. Poster session presented at the American Psychological Association Meeting, Toronto.
Daly, E., Martens, B., Barnett, D., Witt, J., & Olson, S. (2007). Varying intervention delivery in response to intervention. School Psychology Review,
Donovan, S. & Cross, C. (Eds.) (2002). Minority students in special education: Committee on minority representation in special education. Washington, DC: National Academy Press.
Flugum, K. R. & Reschly, D. J. (1994). Prereferral interventions: Quality indices and outcomes. Journal of School Psychology, 32(1), 1-14.
Gravois, Knotek & Babinski (2002). Educating practitioners as consultants: Development and Implementation of the IC-Team Consortium. Journal of Educational and Psychological Consultation, 13,
Gravois & Gickling (2008). Best practices in instructional assessment. In A. Thomas & J. Grimes (Eds.) Best practices in school psychology, Vol. V. Bethesda, MD: NASP
Gravois & Rosenfield (2002). A multi-dimensional framework for evaluation of instructional consultation teams. Journal of Applied School Psychology, 19 (1) 5-29.

59. Selected Bibliography, Cont.
Gravois & Rosenfield (2006). Impact of Instructional Consultation Teams on the Disproportionate Placement of Minority Students in Special Education. Journal of Remedial and Special Education.
Knotek, S., Rosenfield, S, Babinski, L. & Gravois, T.A. (2003). The process of fostering consultee development during instructional consultation. Journal of Educational and Psychological Consultation. 14,
Levinsohn & Rosenfield (2005). Evaluating instructional consultation teams for student reading achievement and special education outcomes. Unpublished manuscript. Rosenfield (2002) Best practices in instructional consultation. In A. Thomas & J. Grimes (Eds. Best practices in school psychology IV. Bethesda, MD: NASP.
Rosenfield (1987) Instructional consultation.Mahwah, NJ: Erlbaum.
Rosenfield & Gravois (1996). Instructional Consultation Teams: Collaborating for change. New York: Guilford.
Rosenfield & Gravois (1999). Working with teams in the school. In C. Reynolds and T. Gutkin (Eds) Handbook of school psychology. John Wiley: New York.

60. Additional Resources
Education for the Future- California State Chico campus initiative; free downloads, case studies Data Tools For School Improvement (Bernhardt) New Roles in Response to Intervention: Creating Success for Schools and Children   PowerPoint by Doug Marston re. implementation of Problem Solving Model in Minneapolis Public Schools-- includes example of effects on referrals, time spent in assessment by school psychologists, examples of barriers & strategies for successful implementation, example of data management interface