1. RtI and Roles Revisited: Opportunities for School Psychologists
Ann Casey, Ph.D.
&
Holly Windram, Ph.D.

2. Objectives for today
Understand the links between your current skills and roles and those you will need in RtI functioning schools
• Understand that consultation and assessment continue to be highly valued skills, but may look different
• Learn of tools and resources that you can use to help your school in the implementation process

3. Fundamental Assumptions
All the students are all our responsibility
All students can make progress when given the amount and kind of support needed
Teaching to the middle doesn’t meet all students needs.
Therefore, we must use our resources in new, different and collaborative ways to ensure each student is as successful as possible! 3

4. What is RtI? RtI is the practice of:
Providing high quality instruction/intervention matched to student needs
Using learning rate over time and level of performance to
Make important educational decisions.
NASDSE, 2005 4

5. 3 Components & 3 Tiered System
RtI is a process comprised of 3 main components:
Evidenced Based Instructional Practices
System of Universal Screening and Progress Monitoring
Problem Solving as a decision making system to determine who gets what interventions, when and by whom [all of which are the systems that support RtI]
EBP - Reading, Behavior, math t
By the end of today, you should leave with a good idea of what each of these 3 components is 5

6. This is a fully integrated system - of both academics and behavior
RtI is just a name - but the name has been helpful in unifying seemingly disparate systems. In both an academic and behavior focus we still rely on measurement, EBP, and problem solving 6

7. Why a Pyramid?
An aerial view- emphasizing that all students need a strong foundation in core instruction and that all students are part of the same educational system.
Tier 1
Tier 2
Tier 3 7

8. What is not RtI It’s not a prescription
It’s not a program, curriculum, strategy, or intervention - some would say this is a paradigm shift
• RtI is a model that requires your school structure & resources be used in ways that improve individual student needs.
RtI requires 3 components, and a tiered instructional framework, but it’s going to look different in your school than how it looks in mine. 8

9. Why does my school need a 3 tiered system?
Are you satisfied with how all your students are achieving?
Many schools have had few options for struggling students & have not been ideal methods in preventing failure
In fact, special education has really been a ‘wait to fail model’
Title I is a funding source but how that has been transformed into prevention services for students varies from school to school. 9

10. RtI as an Organizing System for Accelerating Achievement
While RtI was conceived with low achieving students in mind, there is no reason the system can not be used for high achieving students as well
First and foremost, Response to Intervention is a school improvement model
Discuss Loring example of 120 minute reading block: 60 minutes of core instruction that is heterogenously grouped, and 60 minutes of targeted instruction for all students, including those above benchmark. 10

11. What happens when you implement an RtI system?
St. Croix River Education District data
A small cooperative in east central MN comprised of 5 rural districts 11

12. 12

13. 13

14. 14

15. CHANGE
It’s about change. What direction will SPs go? How will my roles and responsibilities change? How will I be asked to adjust my own views of problem-solving for kids from . . Test to see what’s going within the child to assess the environment to make instructional changes?

16. There are some challenges . . .

17. Whole Group Discussion
When you think about RtI - what are your biggest challenges right now?

18. True or False?
People resist change
Judy Elliott

19. People resist change when they experience loss (or the fear of loss)

20. More work. Work I don’t know how to do.
And then our minds start to spin this . . .
Kid comes from a bad home, poor community, no parent involvement, kid is unmotivated, kid has mental health issues, chemical use . . .
BUT

21. “It’s a whole lot easier to look out a window than look in the mirror
What’s the only thing we have control over at school?

22. Leadership for implementing
Roles for . . .
School Psychologists
Leadership for implementing
RtI framework
Did you know that School Psychologist can, in fact, function as leaders for RtI implementation? Why?

23. #1: We love data.
SPs can walk the talk when it comes to knowing and understanding data, how to use data to make decisions, etc. Many staff fear data. SPs don’t. In fact, we’re data geeks most of the time.

24. #2: We like problems
We love being problem-solvers. In fact, that what’s keeps the job so interesting.

25. #3: We love solutions
Solutions that that are research-based; provide a good instructional match to what the child needs; that are highly effective to drive outcomes for kids.

26. #4: We like (most) people
This is where I’ll build in consulting.
We like kids. We like families. We like teachers. We like consulting and working with these folks to use DATA to Solve Problems.

27. Leaders for implementing
School Psychologists
Leaders for implementing
a RtI framework

28. SCRED School Psychologists: Tier 1
Facilitate building level RtI
Collect & interpret screening data
Consult with general ed teachers

29. Tier 2 Facilitate problem-solving teams
Influence Standard Treatment Protocol
Ensure implementation integrity
Ensure research-based instruction
Facilitate regular data reviews

30. Tier 3 Facilitate intensive problem-solving
Ensure implementation integrity
Ensure research-based instruction
In-depth Problem Analysis
Facilitate regular data reviews

31. Chisago Lakes High School
Facilitator of Problem Solving Process
Facilitator of using data for decision-making
Leadership on research-based instruction
Coaching & support for regular ed. staff
Guidance for a systems viewpoint
Data collection and integrity checks
Advanced knowledge of problem-solving model, academic and social/behavior interventions, and using data for decision-making.
On-going support and coaching for regular ed. staff on problem-solving team, interventions, teaching RtI classes, etc. to foster staff buy-in.
Ability to see the big picture of school structure and assist with problem solving the systematic issues.
Committed follow up with all aspects of problem solving, especially data collection and integrity checks.

32. Minneapolis Public Schools
School Psychologists in MPS do not have formal roles as previously described - yet they are the ‘keepers’ of the problem solving process
Ensuring problems are well defined and analyzed
That data are collected
And that interventions are changed if progress is not made rather than immediately going to special ed. evaluation.
That the system is working - helping people analyze and disaggregate data across the school so that resources are allocated appropriately

33. What is your role in RtI?
There are a number of important roles for psychologists. But the key to well functioning RtI systems is collaboration among and between various roles groups.
We have strengths that we need to capitalize on for the benefit of the system.

34. Evidence Based Practices
The implementation of a tiered instructional delivery system rests upon the use of EBP at all levels or tiers
How can school psychologists be of assistance in this endeavor?
Understanding of importance of controlled studies
Knowledge of various types of educational journals
Skepticism is taught and valued - show me the evidence that something is effective
Discuss skills - many educational professionals do not have as much emphasis on what constitutes research. We have to be careful here - yes we have different knowledge (not superior, but different) and other roles have other emphases and when all these talents are used together we can create something really good for kids..

36. Do School Psychs Need to Know all the Research on Reading, Math, and Behavior?
No - but we must be willing to ask the tough questions when someone else suggests a particular approach, curriculum, or strategy be used:
Is there an evidence base for the effectiveness of this approach?
Honestly, if you are going to useful to your team, you will make it a point to put some of your professional reading time into reading about the evidence for effective practices in these areas. We can not be expert in all things, but you can start with an area of interest to you, or one of necessity for your school.

37. Assessment
Most School Psychs think of themselves as having expertise in this area
In RtI we shift this focus to the system as a whole, rather than focusing primarily on individuals
We need reliable and valid data for universal screening and progress monitoring.
When you are doing less individual formal assessment, you will have time for other roles such as helping teams view and analyze data to make good instructional decisions

38. Data Usage and Data Analysis
This is an important skill in making RtI work - and initially, many educators may need assistance with how to use data to make good decisions for kids
Graphing behavior or skills
View trends in data across grades, years
Analyzing multiple sources of data across individuals or small groups
Includes things such as
comparing current skills to expected skills - norms or benchmarks. How big is the gap? Some people need assistance with understanding common metrics such as percentiles (e.g th percentile are the average range - and the idea the percentile ranks are not of equal size/interval)
2. Graphing behavior or skills and applying decision rules to the data to know when to make changes
Looking at trends in data across grades, across years, and summarizing patterns
Looking at multiple sets of data across an individual or small groups of students and making summary statements of what is known/not known
Some psychs actually become the managers of the data system for their school -

39. Consultation
Problem solving which is a key component in RtI has it’s roots in consultation
Many of you have had training in behavioral, instructional, or collaborative consultation
While these skills will continue to be very useful for students needing tier 3 interventions - one of the shifts school psychs will need to make is using these skills to focus on the system rather than individuals if we are serious about all students making progress
Your consultation skills will be incredibly valuable in RtI
Break at 10:00

40. Consultation/Collaboration
School Psychologists as educational translators
We need to help staff see the connections!
This is not more on the plate - RtI is the plate.
What’s that universal language - Esperanto
Well we need an esperanto in Education because we don’t communicate well across disciplines

41. Consensus building Not a one time event
Goals and consensus for achieving those goals needs to revisited often to keep focus and momentum going
There is a role for you in this area

42. Think and Share: Whole Group
Integrity is _______________.
Holly: “Integrity is doing the right thing even if no one is looking.”

43. What’s in a word or phrase?
Fidelity • Treatment Integrity
Adherence • Intervention Integrity
“Intervention Integrity is the degree to which a planned intervention is implemented as designed”
Greshm, Gansle, Noell, Cohen, & Rosenblum, 1993
The degree to which an interventionist is committed to implement a specific treatment and actively demonstrates intervention-related behaviors (Meichenbaum & Turk, 1987)
The degree to which a planned intervention is implemented as designed (Moncher & Prinz, 1991)

44. Ensuring Intervention Integrity is Important
Key Questions:
Is it being done?
Is is being done well?
What the reason for progress or no progress?
Without checking on implementation integrity, teams cannot be sure that interventions are being applied as designed (DuPaul & Stoner, 1994)
If the intervention is not applied as designed, progress (or lack thereof) cannot be attributed to the specific plan (Kaufman & Flicek, 1995)
Is it being done?
Is is being done well? (this means is it being as designed)
What the reason for progress or no progress? Is it the intervention was not well-matched?

45. Why School Psychologists?
Skilled at problem-solving
Trained observers of human behavior
Know the right questions to ask
For example, Is it being done? We can go observe or interview and find out. What if it’s not being done well? We know how to look at multiple data sources and come up with some possible hypotheses. That’s what ensuring integrity is all about. Do your teams randomly pick interventions for kids based on what’s convienent or available rather than what’s a good instructional match? You’re in a position to suggest that best practice would be to see if we could creatively come up with some other options that are a better match.

46. BREAK

47. Problem-Solving Steps
Identification
2. Problem
Analysis
5. Plan
Evaluation
3. Plan
Development
Ann
4. Plan
Implementation

48. Problem-Solving Process
In an RtI model, we use a 5 step problem solving process to determine who gets Tier 2 or 3 support based on data - not on a referral process
Afternoon break is before this section 48

49. What is problem-solving?
A decision making process
1. Problem Identification
2. Problem Analysis
5. Plan Evaluation
Revise
Modify
Intensify
With Expanding Support
3. Plan Development
4. Plan Implementation

50. 2 Levels of Problem Solving
Grade level teams - for tier 2
Building team - for tier 3
Presumably you already are a member of a bdlg. Problem solving team - they have many names - TAT, SAT, PST, Child Study, etc. This has been the one kid at a time approach.
And if GLTeaming isn’t put into place in your RTI model, one of the first things that will happen is that the building team will be swamped with ‘referrals’ - this is a word that will go away in RtI - as the data tell us who needs more support - no one person no longer has the responsibility to

51. Collaborative Problem Solving
Grade level teaching teams meet together on a regular basis to review student data and student progress toward important goals
Students not making adequate progress receive additional targeted or intensive support
When people ask me - can I visit somewhere where they are doing RtI well - they don’t necessarily know what they want to see - but this is what they need to see! 51

52. Steps of Problem-Solving
IDENTIFICATION
2. Problem
Analysis
5. Plan
Evaluation
3. Plan
Development
Holly
4. Plan
Implementation

53. Step 1: Problem Identification
Question: What is the discrepancy between what is expected and what is occurring?
List problem behaviors and prioritize
Collect baseline data on the primary area of concern (target student and peer comparison):
Record Review
Interview
Observation
Testing
State discrepancy between target student performance and expected/peer performance.

54. Problem Identification Key Points
Collect & analyze school-wide data on top referral concerns
Define “expected” (e.g. local norms / national norms / criterion)
Group v. Individual interventions
Prioritize one concern
Concern is stated measurably
Multiple data sources certify the problem (RIOT)
Avoid problem glorification
“Who, what, when, where, how” plan for tasks
Collect & analyze regular school-wide data on top referral concerns
A decision must be made about how to define “expected” (local norms / national norms / criterion)
When possible work with groups of students
One concern must be prioritized
Concern needs to be stated measurably
Data from a variety of sources should certify this problem (teach RIOT)
Avoidance of problem glorification
Logistics plan for completing the tasks

55. Tools to... Review (R) Interview (I) Observe (O) Test (T)
Cumulative folder
Permanent products
Interview (I)
Brief Problem Identification Interview
Instructional Planning Form (IPF)
Observe (O)
Washington Classroom Observation
On-task
Frequency/Duration/Latency
Correct/Errors
Momentary Time Sampling: e.g., DENO, BASC, Social Play
Test (T)
GOM / Early Literacy
MAP
Other norm or criterion referenced assessments
This will be Data Collection Tools 101. We won’t go indepth except later on the MTS.
Whole Group: What information might you look for in the record review?
One way to gather existing information about the student is to review the cumulative folder for:
Health/Medical records
Attendance
Educational history
Number of schools attended
SE evaluations
Special instruction, e.g., small group, Title 1, etc.
Onset & duration of problem
Past interventions Interview referring teacher, parent, and/or student to obtain information on instruction, curriculum, environment, and learner.
Guidelines:
Should be no more than minutes
Best when occurs prior to problem-solving meeting
Observe: Helps to create a precise definition of the target behavior you want to increase/decrease/change
Test: Reliable and valid.

56. Basic Problem Identification Interview:
What strengths have you identified regarding this student?
List your current concerns for this student.
Which one concern would you like to work on first?
How could this concern be defined as an observable, measurable problem?
Do you have any other data regarding this problem?
If this problem was magically fixed tomorrow, what would look different for this student?

57. Instructional Planning Form (IPF)
Purpose: Examine current educational program and classroom environment
Activity (e.g., Focus or Skill, Teaching Strategy)
Materials
Arrangements
Time
Motivational Strategies
(Later) Focus on these alterable variables to develop hypotheses and interventions.
HANDOUT

58. Goals of Problem Identification
Establish a positive working relationship among team members
Define the problem in observable, measurable terms.
Identify the conditions under which the problem exists across setting.
Provide a strength of the behavior across settings (e.g., how often, severe).
PRODUCT: Discrepancy Statement

59. Discrepancy Statements
Discrepancy statement: A clear and measurable statement of the student’s performance and same-age peer performance.
When given the Picture Naming IGDI Measurement tool, Sally is able to identify 5 pictures correctly whereas same-age peers are able to correctly identify 18.
When observed in circle time, Billy is on-task 30% of the time compared to classmates who are on-task 88% of the time.
When observed during the art activity for 10 minutes, given 8 opportunities, Tina makes 7 following instructions errors compared to peers who make 2 following instructions errors.
The end result of Prob ID is to have a discrepancy statement about the problem.

60. One Minute Activity: Discrepancy Statement
Jimmy is observed for 15 minutes during circle time.
Jimmy
Same-age peers
Off task 9%
Out of Place 0%
Noise
40% 3%
Physical Contact
Total time academically engaged
51%
88%
When observed for 15 minutes during circle time, Jimmy is academically engaged 51% of the time compared to same-age peers who are engaged 88% of the time.
Noise
What discrepancy statements could you make?

61. Activity: Three Statements in One Minute
Tim is reading 10 words correct on 1st grade level CBM ORF probes. Tim is in first grade. The target benchmark is 52 wrc in the spring.
Holly blurts out 18 times in a 20 minute observation during circle time. Other kids blurt out 3 times.
Minnie has been referred to the Principals office 4 times this month. National data* show that students grades K-6 are referred .35 times per month.
PAIRED ACTIVITY - ASSIGN A # TO EACH TABLE.
*PBIS SWIS database.

62. Steps of Problem-Solving
Identification
2. PROBLEM
ANALYSIS
5. Plan
Evaluation
3. Plan
Development
4. Plan
Implementation

63. Step 2: Problem Analysis
Question: Why is the problem occurring?
Collect additional RIOT data to
Differentiate between a skill and performance problem (e.g., can’t do v. won’t do).
Determine situations in which the problem behavior is most likely and least likely to occur.
Generate hypotheses
Narrow down to the most validated and alterable hypothesis.
Ann will talk about this step more in-depth.

64. Steps of Problem-Solving
Identification
2. Problem
Analysis
5. Plan
Evaluation
3. PLAN
DEVELOPMENT
4. Plan
Implementation

65. Complete thorough problem analysis to
Make the link
Complete thorough problem analysis to
make the link between
identified problems
intervention plans
In order to answer the questions of plan development, need to have completed a thorough problem analysis/functional behavior assessment.
Need to make educated link between identified problems and intervention plan designs.
Intervention plans will fail before they begin if we don’t do a good job of this!

66. Step 3: Plan Development
Question: What is the goal?
A. Write the goal, a measurable statement of expected outcomes.
Question: What is the intervention plan to address the goal?
B. Define logistics (e.g., what strategies/procedures will be used, when and how often the intervention will occur, who will implement the intervention and where it will be implemented, and when it will begin).
Question: How will progress be monitored?
C. Define logistics (e.g., what materials are used, when and how often data will be collected, where data will be collected, and who is responsible).
D. Decide on decision-making rules for plan evaluation.

67. Plan Development Key Points
Determine rate of growth to reduce the discrepancy:
Oral Reading Fluency: 2 words per week or to spring target
Written Expression: 1/2 CWS per week or to spring target
Math Facts: 1/2 fact per week or to spring target
Behavior: 10% improvement per week
Research-based intervention plan
Technically adequate progress monitoring tool
Role of master schedule in planning interventions
Determine rate of growth for goal that would result in a reduction of the discrepancy of student performance
Research based intervention plan
Master list of interventions used in district
Knowing the difference between an intervention and a modification
Role of master schedule in planning interventions
Technically adequate progress monitoring tool for outcomes measurement (typically not mastery monitoring)

68. A. Write the Goal Specify desired behavior
Specify measurement condition
Specify criterion for success
In (number) weeks, when (condition) occurs, (learner) will (behavior) to a (criterion).
This goal is the same as an IEP goal would be, except shorter term

69. Could you graph this goal?
Check:
Is the behavior to be measured defined?
Are the measurement conditions clear?
Is the criterion for success specified?

70. Activity Three Goal Statements in 1 Minute
Tim is reading 10 words correct on 1st grade level CBM ORF probes. Tim is in first grade. The target benchmark is 52 wrc in the spring.
Holly blurts out 18 times in a 20 minute observation during circle time. Other kids blurt out 3 times.
Minnie has been referred to the Principals office 4 times this month. National data* show that students grades K-6 are referred .35 times per month.
*PBIS SWIS database.

71. Discuss & Document…

72. Remember . . .
Objective of an intervention for behavior is not just to define and eliminate undesirable behaviors but to teach and reinforce effective replacement behaviors.

73. Remember
There should be a direct relationship between the severity of the problem and the amount of resources being used. N E D S
RESOURCES

74. What is it?
Accommodation?
Modification?
Intervention?

75. C. Define Progress Monitoring Logistics
Progress monitoring objectively measures
intervention effectiveness so we can:
make data-based decisions,
know if an intervention is successful,
increase emphasis on student outcomes,
improve student outcomes,
and set clear expectations.

76. Discuss and Document . . .

77. Steps of Problem-Solving
Identification
2. Problem
Analysis
5. Plan
Evaluation
3. Plan
Development
4. PLAN
IMPLEMENTATION

78. Response to “Failure to Implement” Intervention?
“If we are going to implement interventions, and use data from these interventions to determine a child’s eligibility for special education services, we must be confident that the student’s outcome (either positive or negative) is a response to the intervention and not a response to someone’s failure to implement the intervention.” Cochrane & Castle, 2006
If the intervention is not applied as designed, progress (or lack thereof) cannot be attributed to the specific plan
(Kaufman & Flicek, 1995)
(Quote given by Kerry Bollman)
Response to “Failure to Implement” Intervention?

79. Step 4: Plan Implementation
Question: How will implementation integrity be ensured?
Select an intervention with high probability of success
Communicate a clear plan to interventionists
Provide specific training and support to interventionists.
Directly observe intervention in action.
Make adjustments to the plan if needed.
Collect and graph data on the goal.

80. Ensuring Intervention Integrity Rarely Happens
. . . fewer than 15% evaluated and reported data concerning intervention fidelity. (Gresham, Gansle, Noell, Cohen & Rosenblum, 1993)
Pre-referral interventions in applied settings are often absent of treatment integrity measures (Telzrow, McNamara, & Hollinger, 2000)
Researchers examined a decade of published studies pertaining to school based behavioral interventions, and reported that fewer than 15% evaluated and reported data concerning intervention fidelity. (Gresham, Gansle, Noell, Cohen & Rosenblum, 1993)

81. Without checking on implementation integrity, teams cannot be sure that interventions are being applied as designed
(DuPaul & Stoner, 1994)
Bollman slide

82. If the intervention is not applied as designed, progress (or lack thereof) cannot be attributed to the specific plan
(Kaufman & Flicek, 1995)
Bollman slide

83. Student behavior change is correlated with intervention treatment integrity
(Noell et.al., 2005)
Why? Mostly it’s because the adults stay consistent and do a better job of implementation.

84. SCRED LD FAQs (Kerry Bollman)
“We serve our children best by being circumspect and not allowing our confidence in our teaching skills translate into a sense of infallibility.”
SCRED LD FAQs (Kerry Bollman)

85. Directly Observe the Intervention in Action
Do not assume that there is a correspondence between talking and doing or that talk produces change in behavior! (Witt, 1997)
Participant report alone should not be used to evaluate treatment integrity
Wickstrom, et al. (1998) reported significant discrepancies between levels of treatment fidelity reported by teachers (54%) and revealed by direct observation (4%)
Objective data collected by outside sources are most useful (Moncher & Prinz, 1991)

86. Wickstrom, et al. (1998) reported significant discrepancies between levels of treatment fidelity reported by teachers (54%) and as revealed by direct observation (4%).
Bollman

87. Provide Specific Training and Support Interventionists
Initial training for interventionists (Hirallel & Martens, 1998)
Trainer explains the procedure to the interventionist
Trainer demonstrates the procedure
Interventionist practices the procedure with the trainer as mock student
Trainer provides specific feedback
Repeat steps as necessary
Application in the instructional setting

88. Plan Implementation Key Points
Intervention Scripts & Training
Specifics of intervention are well understood by interventionist
Interventionists like them!
Training: Modeling, practice, and feedback with adults prior to use with students
Integrity checks
Was a direct observation done?
Do data support that plan was implemented as designed?
Intervention Scripts & Training
Increases likelihood that specifics of intervention will be well understood by those performing the intervention
Are preferred by interventionists rather than global intervention descriptions
Training should include modeling, practice, and feedback with adults prior to use with students
Integrity checks
It is impossible to evaluate the success of a plan if the team is not certain that the plan was implemented as designed
Did the program happen as planned at least once (formal observation)
Do you have data to indicate that the student participated fully in the intervention (attendance, time logged in, accuracy of participation in intervention)

89. Integrity doesn’t stop there . . .
Complete ongoing assessment of implementation through:
Participant Reports
Observation
Review of Permanent Product(s)

90. A Note About Participant Reports
Self-report feedback from interventionists and students is useful and good.
BUT
Participant report alone should not be used to evaluate treatment integrity

91. Making Adjustments to Plans
Consider a plan change when the current plan:
Is not acceptable to the interventionist
Is not feasible to implement.
Is not perceived as being effective.
Is highly disruptive to the classroom ecology.

92. Changes? Team decision Document!
Interventionists should not make unilateral decisions regarding changes to an intervention plan

93. Integrity of Progress Monitoring
Collect and graph data on intervention goal
Monitor the progress monitoring plan
Provide support to those collecting the data.
If the data are not being collected designed then
Give additional support OR
Change the progress monitoring plan

94. Steps of Problem-Solving
Identification
2. Problem
Analysis
5. PLAN
EVALUATION
3. Plan
Development
4. Plan
Implementation

95. Step 5: Plan Evaluation Question: Is the intervention plan effective?
Use data to determine student progress
Evaluate intervention acceptability
Determine as a team what to do next.

96. Plan Evaluation Key Points
What is adequate response to intervention?
SCRED Practice:
CBM target scores for fall, winter, spring of each grade as linked to state test
Inherent growth rate within these target scores is defined as our desired rate of growth
Calculated confidence interval (Standard error of slope) around the target growth rate
Students with growth rates below the bottom of the confidence interval are considered to not have adequate response to the intervention
Grade
Minimum Growth
Expected Growth
Maximum Growth 2
1.03
1.31
1.59 3
0.75

97. Steps of Problem-Solving
Identification
2. PROBLEM
ANALYSIS
5. Plan
Evaluation
3. Plan
Development
4. Plan
Implementation

98. Problem Identification

99. Problem Identification
Question: What is the discrepancy between what is expected and what is occurring?
Academic v. Behavior:
“Can’t read because won’t sit still or won’t sit still because they can’t read?”
Performance v. Skill
Can’t do v. Won’t do

100. Tools to... Review (R) Interview (I) Observe (O) Test (T)
Cumulative folder
Permanent products
Interview (I)
Brief Problem Identification Interview
Instructional Planning Form (IPF)
Observe (O)
Washington Classroom Observation
On-task
Frequency/Duration/Latency
Correct/Errors
Momentary Time Sampling: e.g., DENO, BASC, Social Play
Test (T)
GOM / Early Literacy
MAP
Other norm or criterion referenced assessments
This will be Data Collection Tools 101. We won’t go indepth except later on the MTS.
Whole Group: What information might you look for in the record review?
One way to gather existing information about the student is to review the cumulative folder for:
Health/Medical records
Attendance
Educational history
Number of schools attended
SE evaluations
Special instruction, e.g., small group, Title 1, etc.
Onset & duration of problem
Past interventions Interview referring teacher, parent, and/or student to obtain information on instruction, curriculum, environment, and learner.
Guidelines:
Should be no more than minutes
Best when occurs prior to problem-solving meeting
Observe: Helps to create a precise definition of the target behavior you want to increase/decrease/change
Test: Reliable and valid.

101. Basic Problem Identification Interview:
What strengths have you identified regarding this student?
List your current concerns for this student.
Which one concern would you like to work on first?
How could this concern be defined as an observable, measurable problem?
Do you have any other data regarding this problem?
If this problem was magically fixed tomorrow, what would look different for this student?

102. Instructional Planning Form (IPF)
(Handout)
Purpose: Examine current educational program and classroom environment
Activity (e.g., Focus or Skill, Teaching Strategy)
Materials
Arrangements
Time
Motivational Strategies
(Later) Focus on these alterable variables to develop hypotheses and interventions.
Go through handout instructions.

103. Review: Goals of Problem Identification
Establish a positive working relationship among team members
Define the problem in observable, measurable terms.
Identify the conditions under which the problem exists: antecedent, situation, and consequent conditions across settings.
Provide a strength of the behavior across settings (e.g., how often, severe).

104. Problem Identification Outcome: Discrepancy Statement
An observable and measurable statement of the student’s performance compared to same-age peer performance.
The end result of Prob ID is to have a discrepancy statement about the problem.

105. Examples
When given the Picture Naming IGDI Measurement tool, Sally is able to identify 5 pictures correctly whereas same-age peers are able to correctly identify 18.
When observed in circle time, Billy is on-task 30% of the time compared to classmates who are on-task 88% of the time.
When observed during the art activity for 10 minutes, given 8 opportunities, Tina makes 7 following instructions errors compared to peers who make 2 following instructions errors.

106. One Minute Activity: Discrepancy Statement
Jimmy is observed for 15 minutes during circle time.
Jimmy
Same-age peers
Off task 9%
Out of Place 0%
Noise
40% 3%
Physical Contact
Total time academically engaged
51%
88%
When observed for 15 minutes during circle time, Jimmy is academically engaged 51% of the time compared to same-age peers who are engaged 88% of the time.
Noise
What discrepancy statements could you make?

107. Activity: Three Statements in One Minute
Tim is reading 10 words correct on 1st grade level CBM ORF probes. Tim is in first grade. The target benchmark is 52 wrc in the spring.
Holly blurts out 18 times in a 20 minute observation during circle time. Other kids blurt out 3 times.
Minnie has been referred to the Principals office 4 times this month. National data* show that students grades K-6 are referred .35 times per month.
PAIRED ACTIVITY - ASSIGN A # TO EACH GROUP.
*PBIS SWIS database.

108. Problem Analysis
Goal of problem analysis is to understand the variables that may be contributing to the problem, to develop potential hypotheses from this learning, and design a plan to solve the problem.
Ann

109. PA: Scientific Method Observe and identify problem
Develop relevant hypotheses
Design procedures to test hypothesis
Collect data
Analyze and synthesize data
Make a conclusion
Problem ID PA Pd PI PE

110. Hypotheses and Level of Inference
All hypotheses require some inference
What is known vs what is inferred
We want to rely more on low level vs high level inferences - more explicit behaviors and environmental events
These will lead to alterable variable, ones which we have control over
The instruction, curriculum, school environment

111. Shift in thinking
If you are used to giving tests that purportedly give you information about something within the student that is contributing to the problem, then we are asking you to suspend this for a moment.
While student characteristics are not unimportant, we want to investigate those lower level inferences first.
We consider student characteristics when they interact with alterable variables

112. Thinking Differently about Students’ Problems
Student problems can be defined and changed
Questions will drive assessments
Assessments will lead to instructional decisions
Enabled learning rather than discrepancy or diagnosis is the goal

113. Instead of diagnosing the learner, we begin by diagnosing the instruction. We identify flaws in the instruction and correct them, with the assumption that the learner’s problems were caused by flaws in the instruction (Engelmann & Carnine, 1982, p.18.

114. Asking the Right Question
What does this student need in order to fix the problem we have identified?

115. Problem Analysis Defined…
Problem Analysis is the process of gathering relevant information in the domains of the instruction, curriculum, environment, and the learner (ICEL) through the use of reviews, interviews, observations, and tests (RIOT) in order to evaluate the underlying causes of the problem.

116. Importance of Problem Analysis
Interventions are derived from results (data) generated during problem analysis.
Interventions linked to assessment information are more likely to be effective in meeting the desired level of performance.
Ineffective interventions can lead to resistance to intervention by further supporting problem behaviors and making it harder to meet desired outcomes.

117. Warning!!!
Teams: Do Not discuss hypotheses for why the behavior is occurring before you have defined the behavior you are considering.
Team note takers should have a written discrepancy statement on the Problem Identification Summary Form before hypothesis discussions begin.
This will help keep discussions more focused and efficient

118. Question: Why is the problem occurring?
• Based on what you know list possible causes for the student’s problem (hypotheses)
Consider all domains (Instruction, Curriculum, Environment, Learner)
Differentiate between skill problems and performance problems
Determine situations in which the problem behavior is most likely and least likely to occur
• For each hypothesis generated, list the data you have that supports or refutes the hypothesis
• Narrow down to the most validated and alterable hypothesis
• Collect any additional data you need to validate the hypothesis that the team considers to be the most likely
This might be an observation or informal assessment to examine skills - but you are looking for convergent data

119. Problem Analysis This phase can be a bit circular… Look at the data
you have
Collect any
additional data
you would need
to confirm or
refute your guess
Review and interview could provide your current data but if you need more, then you could observe and/or test.
Think about what
hypotheses seem
possible given the
data you have, and list data for each

120. Irrelevant and Unknown
A. Consider what you know about the target behavior that is relevant to determining why the problem is occurring and a possible solution
Relevant and Known
Relevant and Unknown
Irrelevant and Known
Irrelevant and Unknown
Curriculum
Instruction (e.g., arrangement, response format)
Allotted/Engaged Time
Classroom Environment
Motivational Strategies

121. PA: Summary
Interviewing is often a skill strength that many school psychologists have developed
These skills are directly transferable to the Problem Analysis process, and will be a valuable asset to teams in solving problems
This afternoon we will practice using a tool to help in this important part of the Problem Solving Proces

122. LUNCH!

123. Resources
See Handout

124. Leadership

125. Name the five wealthiest people in the world
Name the last five Heisman trophy winners
Name the last five winners of Miss America.
Name then people who have won a Noble or Pulizter prize.
Name the last half dozen Academy Award winners for best actor and actress.
Name the last decade’s worth of World Series winners.

126. None of us remember the headliners of yesterday.
None of us remember the headliners of yesterday. These are no second-rate achievers. They are the best in their fields. But the applause dies. Awards tarnish. Achievements are forgotten. Accolades and certificates are buried with their owners.

127. List a few teachers who aided your journey through school.
Name three friends who have helped you through a difficult time.
Name five people who have taught you something worthwhile.
Think of a few people who have made you feel appreciated and special.
Think of five people you enjoy spending time with.

128. They are the ones that care.
The Lesson:
The people who make a difference in your life are not the ones with the most credentials, the most money, or the most awards.
They are the ones that care.
You do not have to be the Administrator or blah, blah to make a legitimate, measurable difference in the lives of ALL students in your buildings and districts. You may be feeling more behind the scenes or feeling like it’s one student at a time - a slow process - but your efforts will pay off exponentially. Just think of why you dientified the people you did in quiz 2.