A collaborative project between the Florida Department of Education and the University of South Florida Y3D1 SBLT Tier 3 Problem Identification & Problem Analysis School Implementation Blueprints R

Advance Organizer Year 3 Training Plan & Rationale Importance of Established PS Process Tier 3 Problem Identification - Worksheet Tier 3 Problem Analysis – Worksheet

Training Outline Influences Skills to Reinforce –Problem Solving Process –Intervention Dev/Support –Intervention Integrity –Integrating the Tiers –Decision Rules –Scheduling New Skills –Tier 3 Data Sources –Characteristics of Tier 3 Interventions –Tier 3 Scheduling/Resource Mapping –Integrating the Tiers –Eligibility Other Indications from Data Review –Staff Involvement –Parent Involvement –School Implementation Plan –Using Data to Inform Implementation –Scaffolding –District-School Communication –Facilitation –Graphing/Technology Let’s look at the data which drove a few of these decisions

Problem Solving Process

Staff Involvement

Year 1 SBLT: Scaffolding Heavy Scaffolding

Year 1 SBLT: Scaffolding Lite Scaffolding

Year 2SBLT: Scaffolding Heavy Scaffolding

School Implementation Plan District-School Communication

Skills Training School Level Blueprint for Implementation Facilitated Through: Implications for Training Plan Problem Solving Process Scaffolding New Skills School Implementation Plan District-School Communication Staff Involvement

Review of Y3 Training Plan Instructional Template –Case Study Format –Four Steps of Problem Solving at Tier III –Worksheets –Skills Exercises

Problem Analysis Intervention Design Response to Instruction/ Intervention Problem Identification Problem Solving Process Accomplished through ongoing instructional decision making in response to student performance data after the delivery of instruction based upon verified hypotheses of why an identified problem is occurring. Goal is Student Achievement –At Tiers I, II, and III

Problem Solving Process Identify the Problem Design Intervention Monitor Progress Analyze the Problem Implement Intervention Evaluate Intervention Effectiveness 

TIER 3 PROBLEM IDENTIFICATION Identifying Students in Need of Individualized, Intensive Instruction

16 Academic SystemsBehavioral Systems Tier III: Intensive Interventions ( Few Students) Provided to students who need Individual Intervention Tier II: Strategic Interventions (Some Students) Provided to students who need more support in addition to the core curriculum Tier II: Targeted Group Interventions (Some Students) Provided to students who need more support in addition to school-wide positive behavior program Tier I: Core Curriculum Provided to All students Tier I: Universal Interventions Provided to all students; all settings Three Tiered Model of School Supports Tier III: Comprehensive and Intensive Interventions ( Few Students) Provided to students who need Individualized Interventions From the Florida Problem Solving/Response to Intervention Project

Tier 3: Intensive, Individualized Instruction Services provided to students who are intensely behind and/or not sufficiently responsive to core and strategic (Tier 2) instruction. Required by small percentage of students (approximately 5%)

Identifying A Tier 3 Problem Typically students come to our attention through one of the following: Periodic review of universal screening data Review of progress monitoring data for students receiving strategic instruction Teacher or parent referral to PS team

Digits correct per minute Math Computation target student peer group benchmark

Tier 3: Never Assume When a student is identified for intensive instruction, data should indicate: Core and strategic instruction has been provided with fidelity Access to effective core instruction (80% of students successful) Access to effective strategic instruction (70-80% successful)

Problem Identification Data Required: Replacement behavior or target skill Current/Observed level of performance Expected level of performance Peer level of performance GAP analysis

Review Replacement Behavior or Target Skill Replacement behaviors/target skills should: be specific, measurable and observable indicate what you WANT the student to be able to do attract reinforcement

Standards for Expected Level of Performance Sources include, but are not limited to: School, district, state or national norms Benchmark standards District standards and/or benchmarks Teacher expectations Direct peer comparison Criteria for next environment (Kurns & Upah, 2007, p. 44)

“Which one do I use?” Choose the standard specific to the target behavior. When available, it may be appropriate to use more than one. For example, if both benchmark and local norms are used we know how level of current performance compares to research standards and peer performance. (Kurns & Upah, 2007, p. 44)

Performance Discrepancy The “gap” or magnitude of the discrepancy is the size of the difference between expectation and current/observed performance

Reviewing Gap Analysis Expected Level Gap = Current/Observed Level Rule of thumb: 2.0 = significant gap

Reviewing Gap Analysis Target Student’s Observed/Current Level of Performance: –40 WCPM Expected Level of Performance –92 WCPM Peer Level of Performance –98 WCPM GAP Analysis: –Expected Level/Target Student92/40= 2+X differenceSIGNIFICANT GAP –Expected Level/Peer92/98= <1 X differenceNO SIGNIFICANT GAP Is this an individual student problem?

Individual Student Problem

Considering Tier 3 Is Tier 2 Effective? At least 70% - 80% of students should benefit from Tier 2 Decision Making Options: – Integrity + Rate of Response –Unit of analysis is the group of students % benefiting from Tier 2 supports

Rose Struggling in target skill area. Tier One data indicate that approximately 15% of the student’s in Rose’s class demonstrate this discrepancy. Core effectiveness: 85% Rose is receiving 20 min. of strategic instruction (Tier 2) five times per week along with two other classmates.

Is Tier 2 instruction effective for 70-80%? benchmark

Are Tier 3 services appropriate? Yes The Tier 2 Rose has been receiving is effective for approximately 70% of the intervention group. Intervention integrity is very good.

Jarrett Not proficient in target skill area. Tier One data indicate that approximately 20% of the student’s in Jarrett’s class demonstrate this discrepancy. Core effectiveness: 80% Jarrett is receiving 20 min. of strategic instruction five times per week specific to the target skill area.

Is Tier 2 instruction effective for 70-80%? benchmark

Are Tier 3 services appropriate? No Intervention integrity is good. Yet, progress monitoring data indicate that the Tier 2 intervention Jarrett is being provided is not effective for % of the intervention group.

Stephanie Below benchmark in target skill area. Tier One data indicate that approximately 20% of Stephanie’s class demonstrate this discrepancy. Core effectiveness: 80% Stephanie is receiving 30 min. of strategic instruction 3 times per week along with a small group of classmates.

Is Tier 2 instruction effective for 70-80%? benchmark

Are Tier 3 services appropriate? No The Tier 2 Stephanie has been receiving is effective for approximately 70% of the intervention group. However, Tier 2 intervention integrity has been jeopardized due to Stephanie’s excessive absences.

Randy Review case study

Randy What is the replacement behavior or target skill? (measurable, observable, reportable) What is the student’s current level of performance? (Be sure to include data that directly assesses the target skill you want the student to perform). What is the benchmark/expected level of performance? What is the peer level of performance? Gap Analysis Benchmark & Student Benchmark & Peer Peer & Student What percentage of students in the classroom demonstrate this discrepancy? At what tier will this problem be addressed (circle one)? Tier I, Tier II, Tier III

Randy Do we have enough information to complete Problem Identification? If yes, go to Problem Analysis If no, what information is still needed? When will we meet again?

Your Student As a team, complete worksheet for Problem Identification

TIER 3 PROBLEM ANALYSIS Identifying Students in Need of Individualized, Intensive Instruction

How Does it Fit Together? Addl. Diagnostic Assessment Instruction Results Monitoring Individual Diagnostic Individualized Intensive As necessary All Students at a grade level ODRs Monthly Bx Screening Bench- Mark Assessment Annual Testing Behavior Academics None Continue With Core Instruction Grades Classroom Assessments Yearly Assessments Group Diagnostic Small Group Differen- tiated By Skill Approximately monthly Step 1 Step 2Step 3Step 4 Supplemental 1-5% 5-10% 80-90% Core Intensive

Problem Analysis is the process of gathering information in the domains of instruction, curriculum, environment, and learner (ICEL) through the use of reviews, interviews, observations, and tests (RIOT) in order to evaluate underlying causes of a problem and to validate hypotheses. Problem Analysis:

Tier 3 Problem Analysis Diagnostic process… Student individually assessed Determine most probable cause of problem Match intervention to student need Content area experts are necessary!

Role of assessment in problem analysis Question-driven assessment process: What do we know/need to know about problem? What are some possible causes? What are some predictions about solutions?

Role of assessment (con’t) What data are needed to support/refute hypotheses? What intervention best matches student need?

Assessment: Ongoing process—not an “event” Functional—relevant and directly related Purposeful—data not collected until assessment questions developed Formative—will inform instruction

Why is the problem occurring? PROBLEM ANALYSIS Step 1: Gather assessment information Step 2: Develop hypotheses Step 3: Validate hypotheses Step 4: Link to intervention

To answer these questions: 1)WHY the difference btw. expected/observed 2)WHAT do we need to teach (curriculum) and HOW do we need to teach (instruction)? 3)WHICH intervention will have the highest probability of being successful? Step 1: Gather information

Determining What Data to Collect Known Information Unknown Information Educationally Relevant & Alterable Less Educationally Relevant & Unalterable Conduct Assessments to Gather this Information (Behavior observations, specific skill assessments) These are assessment questions Disregarded or Low Priority (Height, eye color) Don’t Go Here! (Cognitive processing?) Gather this Existing Information ( Classroom screening data, ODRs)

Gather information that… is educationally relevant is alterable will directly impact student gains in the classroom environment

Assessment domains Gather information from: Instruction Curriculum Environment Learner Is there something we could change about I, C, E that would enable student to learn?

Assessment methods Gather information through: Review of existing records Interview teacher, parent, student, etc. Observe in classroom or other appropriate setting Test learner on particular skill/concept

Step 2: Develop hypotheses Developing assumed causes… After initial assessments, focus of PA becomes hypothesis generation— process of making informed statements about why a problem occurs

Develop hypotheses Hypotheses… Are developed to determine reasons for why the replacement behavior is not occurring Should be based on research relevant to the target skills Focus on alterable variables Should be specific, observable, and measurable Should lead to intervention

Develop hypotheses Hypotheses… Must consider both SKILL and PERFORMANCE deficits: –Skill Deficit Student does not have the skills to perform the task –Student lacks fluency skill for grade level –Student lacks private speech for self control –Performance Deficit Student does perform existing skill or performs at lower level –Student reads slowly because of fear of ridicule by peers for mistakes –Peers reinforce bad choices more than teacher reinforces good choices

General format: “______________________(the problem) Is occurring because _____________________(hypothesis)”

Format for behavior: “The problem is occurring because when this occurs____________(trigger), the student_____________(behavior) in order to/because_______________ (hypothesized function/reason). (Heartland AEA 11)

Prediction statement: Inference from hypothesis What would be expected if some other action took place Comprised of specific actions that are reasonable and feasible If/then wording Used to develop assessment questions to confirm/disconfirm hypotheses

Prediction statement: “If_____________(specific action) would occur, then______________(problem) would be reduced.

Hypotheses Validation Why do Problem Solving Teams need to Validate a Hypothesis? If the hypothesis is inaccurate and the wrong intervention is implemented valuable time could be wasted on an intervention that was not an appropriate instructional match for the student.

Step 3: Validate hypotheses Formulate assessment questions to gather information or data that will support/refute hypothesis Collect additional data as needed Validate before intervention is implemented

DOMAINS R Review I Interview O Observe T Test I Instruction C Curriculum E Environment L Learner Domains for Assessment

Format for Hypothesis Validation Validated?: Mary is noncompliant because she does not have the skills to complete the work successfully. If we reduce the academic demand or improve her skills, Mary will become more compliant. HypothesisPrediction Assessment Question(s): Is task difficulty appropriate for Mary’s skill level? Where are the answers?: Review Learner records for evidence of skills; Review Curriculum to understand expectation. Answers: Review of records and review of curriculum indicates that Mary has the skills to complete the requested tasks. No

Format for Hypothesis Validation Validated?: a. Mary is not being positively reinforced for compliant behavior. b. Mary is being reinforced for noncompliant behavior If Mary is positively reinforced for compliant behavior / not reinforced for noncompliant behavior, her compliance will increase. HypothesisPrediction Assessment Question(s): Is Mary being positively reinforced for compliant behavior? Is Mary being reinforced for noncompliant behavior? Where are the answers?: Observe the Environment in the situations where Mary displays noncompliance and compliance. Answers: Observations indicate that Mary is not being consistently reinforced for compliance in large group settings outside of the homeroom, but is being consistently reinforced within the homeroom where she displays compliant behavior. She is also avoiding assignments through noncompliance. Yes

Step 4: Link to intervention From validated hypotheses, team selects hypothesis which seems most likely to lead to effective intervention Keep in mind… skills of interventionist feasibility of implementation

Problem Analysis Practice Joe has difficulty keeping his hands to himself in the lunchroom. Examples include when he touches other students, grabs students’ plates and food, and pushes trays of food onto the floor. “Joe” example from Heartland AEA 11, “Polishing our Practice”

Teamwork Identify the specific assessment questions you would ask regarding Joe’s lunch room problem –Consider each of the ICEL domains How would you collect the info (RIOT x ICEL)?

77 Problem Analysis: Joe’s Assessment Questions What are lunch room rules? Have they been taught? Is he having problems with specific students? What has been tried before and how did he respond? Does it happen with certain lunchroom attendants? What does he gain or avoid? What is the adult:kid ratio? Has Joe had scientifically- based instruction in reading? Does Joe have limited English proficiency? Does Joe have health or medical concerns?

Assumed Causes Joe touches other people’s food and pushes trays of food to the floor when he does not get to sit by two specific people in order to gain the preferred seating arrangement. He has trouble using verbal communication to let people know where he wants to sit.

Prediction statements Predictions: If there was a seating arrangement for Joe to sit by his preferred lunch partners, then Joe’s inappropriate use of hands would decrease. If Joe had a way to communicate his wants for lunch partners, his inappropriate use of hands would decrease.

Validate Hypothesis Information from interview of lunchroom supervisors indicates that Joe has fewer incidences of inappropriate use of hands on days when he is sitting by preferred lunch partners. Information from teacher indicates that when Joe can communicate his wants and needs, he displays fewer inappropriate physical behaviors.

Linking Problem Analysis to Intervention Design Recommendations for intervention design based on problem analysis: Provide a seating arrangement for lunch where Joe is seated next to at least one of his preferred lunch partners Continue to work on teaching Joe to verbally communicate his wants and needs.

Randy Review case study

Randy Hypothesis 1: The problem is occurring because: the curriculum being delivered to him does not address reading fluency and accuracy. Prediction Statement 1: If ____ would occur, the problem would be reduced Relevant Data Validated Yes/No

Randy Hypothesis 2: The problem is occurring because: Randy does not have adequate decoding skills to read accurately and fluently. Prediction Statement 2: If ____ would occur, the problem would be reduced Relevant Data Validated Yes/No

Randy Hypothesis 3: The problem is occurring because: Randy does not self-monitor while reading. Prediction Statement 3: If ____ would occur, the problem would be reduced Relevant Data Validated Yes/No

Randy Hypothesis 4: The problem is occurring because: Randy does not have adequate grade level sight words. Prediction Statement 4: If ____ would occur, the problem would be reduced Relevant Data Validated Yes/No

Randy Do we have enough information to complete Problem Analysis? If yes, go to Intervention Development If no, what information is still needed? When will we meet again?

Your Student Complete worksheet for Problem Analysis