Basic FBA to BSP Module 7: Evaluation Using FBA to Develop Function-Based Support for Students with Mild to Moderate Problem Behavior Module 7: Evaluation

Adapt BSP and implementation as needed based on on-going monitoring The Basic FBA to BSP Process 1. Define the Problem Behavior 2. Conduct assessment for behavior support planning a. Functional Behavioral Assessment • Defining behavior in observable & measureable terms • Ask staff and student about where, when, & why behavior occurs • See the behavior during specified routines • Hypothesize a final summary of where, when, & why behavior occurs 3. Design an individualized behavior support plan (BSP) • Ensure technical adequacy • Ensure contextual fit 4. Ensure Fidelity of Implementation 5. Monitor Plan Impact on Student Behavior Adapt BSP and implementation as needed based on on-going monitoring Adapted from Horner, Albin, Todd, Newton & Sprague, 2011

Basic FBA to BSP Training Series Module 1- Teaching Basic Principles Module 2- FBA: Practice Interviewing Module 3- FBA: Practice Observing Module 4- Critical Features of BSP Module 5- Building BSP from FBA Module 6- Implementation & Evaluation Module 7- Evaluation

Basic vs. Complex FBA/BSP Focus of this training series Basic Complex For: Students with mild to moderate problem behaviors (behaviors that are NOT dangerous or occurring in many settings) Students with moderate to severe behavioral problems; may be dangerous and/or occurring in many settings What: Relatively Simple and Efficient process for behavior support planning based on “practical” FBA data Time-intensive process that involves emergency planning, family-centered planning, and collaboration with outside agencies Developed by whom: Team of school-based professionals (e.g., PBS team members whose job responsibilities include FBA and behavior support planning) School-based team including professionals trained to develop and implement intensive interventions for students with severe problem behaviors (e.g., behavior specialist) 4 4 4

Objectives By the end of this module you will be able to: Define the necessary components of evaluation plans and provide examples of appropriate short- and long-term goals Describe data collection procedures that would be used to track implementation fidelity and student progress when provided with a sample BSP Describe the process for conducting and products that should result from a Plan Review Meeting

Review #1 The Basic FBA to BSP training series is designed to teach school staff to conduct assessment and develop supports for students with what type of behavior? Mild to Moderate, NON-DANGEROUS Behaviors

Review #2 What are the 4 steps of the Basic FBA Process (Hint: D.A.S.H)? #1: Define behavior in observable, measureable terms #2: Ask staff about When, Where, & Why the behavior occurs #3: See the behavior (direct observation) #4: Hypothesize a final summary of when, where and why behavior occurs

Review #3 Please list the 4 critical components of Behavior Support Plans: #1: Competing Behavior Pathway #2: Prevention, Teaching, and Consequence Strategies #3: Implementation Plan #4: Evaluation Plan

Leading a BSP Team

Evaluation Planning The team leader will ensure that the BSP includes an evaluation plan with: A short-term goal that is reasonable based on current performance Focused on decreasing problem behavior and increasing Replacement behavior A long-term goal focused on increasing desired behavior Specific activities/procedures that will be used to evaluate progress A specific date when the team will next meet to review progress

Evaluation Planning: How Will We Measure Progress?? Behavioral Goal (Use specific, observable, measurable descriptions of goal) The team identifies: Short-term goal Long-term goal Specific evaluation procedures Date to meet and evaluate the effectiveness of the plan What is the short-term behavioral goal? _________ Expected date What is the long-term behavioral goal? Evaluation Procedures Data to be Collected Procedures for Data Collection Person Responsible Timeline Is Plan Being Implemented? Is Plan Making a Difference? Plan date for review meeting (suggested within 2 weeks) ________________

Use Competing Behavior Pathway to Identify Goals Long-term goal Desired Behavior Typical Consequence Routine: Setting Event Antecedent Problem Behavior Maintaining Consequence Alternate Behavior Short-term goal 12

Evaluation Planning: Short- and Long-term Goals Short-term goal Focus on reducing problem behavior and increasing student’s use of the identified Replacement behavior & Use baseline data to develop a REASONABLE initial goal that student will be able to achieve Short term goal will CONTINUOUSLY be revised as student moves closer to achieving the long-term goal Long-term goal Focus on desired behavior & sustained reductions in problem behavior Begin by reinforcing approximations of desired behavior 13

Developing Goals Goals should include the following components: Condition Student Behavior Criterion Measurement Plan

Developing Goals Both short and long-term goals should: A. Be written in observable, measureable terms What specific behaviors will you increase/decrease? Increase use of Replacement Behavior Reduce Problem Behavior Increase APPROXIMATIONS of the desired behavior B. Include specific mastery criteria How will you know when the student has met the goal?

Measuring Reduced Problem Behavior via increased Positive Behavior Developing Goals Measuring Reduced Problem Behavior via increased Positive Behavior Short-term Goal During Reading, when asked to complete independent writing tasks, (Condition) Jonas (Student) will work in his seat quietly OR appropriately request a break (Behavior) at least 70% of the time (Criterion) as measured by ratings on a daily point card (Measurement). Notice that the goal is stated positively (what to do) rather than negatively… reductions of problem behavior can be measured by monitoring the expected behavior

Sample Short-Term Goal for Dexter Short-term: Dexter will: a) stay on task without leaving his seat or talking to peers about unrelated topics for at least 75% of independent work time, OR b) appropriately ask to work with a peer (or work independently) in Writing, c) complete at least 25% of his daily writing assignments for 4 out of 5 days across 2 consecutive weeks. Decrease Problem Behavior Increase use of Replacement Behavior Increase Approximations of Desired Behavior Mastery Criteria 17

Example Goals for Leroy During Writing class, Leroy is currently engaging in problem behavior (throwing materials and cursing) to escape difficult tasks in Math approximately 4 days per week. On average, he is completing only 25-30% of his work in class. Behavioral Goals ( Always include mastery criteria ) What is the short-term behavioral goal? During Writing, Leroy will ask appropriately for an easier task or for a “break” from difficult tasks without throwing materials or cursing at least 75% of the time as measured by a daily point card for 2 consecutive weeks. What is the long-term behavioral goal? Leroy will complete at least 80% of his assigned work in his math class with no more than 3 incidences of problem behavior (throwing materials, cursing) for 3 consecutive weeks. __5/1____ Expected date Short-term goal Increase Alt. Behavior & Reduce Problem Behavior + Approximation toward Desired Behavior Long-term goal Increase Desired Behavior & Reduce Problem Behavior

How will we measure student progress?

Is the Plan Making a Difference? FAQ: I see the student every day, why do I need to collect data? Answer: Data help us to Document what has occurred and the variables responsible Predict future performance Be accountable for our own behavior Determine when program modifications are needed

Balancing Accuracy and Feasibility The “quality” of the measurement system is irrelevant if no one uses it. How often will data need to be collected? How often does the behavior occur? How much time, effort will data collection methods require? Does this “fit” the context/setting? Are there forms that staff are already using (ex. point cards) that can be modified/used? Are implementers included when finalizing the measures to ensure feasibility?

Measures for Tracking Student Behavior Considerations: Does the measure capture the specific tasks/target behaviors of interest? Was it a “good” or “bad” day? vs. How many talk-outs occurred during Spanish class today? Is the measure sensitive enough to change? Are we tracking specific student behaviors?

Developing Data Forms Estimates vs. Real-Time Recording Consider using Scales or Ratings rather than Frequency Counts/Duration Measures Ex. 0, 1-3, 4-5, 5+ incidents/minutes (circle one)

Example Goals for Leroy During Writing class, Leroy is currently engaging in problem behavior (throwing materials and cursing) to escape difficult tasks in Math approximately 4 days per week. On average, he is completing only 25-30% of his work in class. Behavioral Goals ( Always include mastery criteria ) What is the short-term behavioral goal? During Writing, Leroy will ask appropriately for an easier task or for a “break” from difficult tasks without throwing materials or cursing at least 75% of the time as measured by a daily point card for 2 consecutive weeks. What is the long-term behavioral goal? Leroy will complete at least 80% of his assigned work in his math class with no more than 3 incidences of problem behavior (throwing materials, cursing) for 3 consecutive weeks. __5/1____ Expected date Short-term goal Increase Alt. Behavior & Reduce Problem Behavior + Approximation toward Desired Behavior Long-term goal Increase Desired Behavior & Reduce Problem Behavior

Measuring the Short-term Goal During Science, LeRoy will ask appropriately for an easier task or for a “break” from difficult tasks without making inappropriate comments or throwing materials at least 75% of the time as measured by a daily point card. Measuring the Short-term Goal LeRoy Feb. 4th Measure Decrease in Problem Behavior through focus on increasing Expected Behaviors Ask teacher for break / easier task Use Respectful Language Use materials appropriately

Example: Rating Feasibility During Science class, LeRoy often makes inappropriate comments, pushes materials off his desk and refuses to do his work to escape difficult tasks. To evaluate the effectiveness of Leroy’s BSP, the team members have suggested that his teacher: Have participants rate using hand signals (i.e., thumbs up, thumbs down, thumb in the middle/maybe) Indicate at the end of each class period if LeRoy had ( 0, 1-3, or 4 or more ) instances of problem behavior

LeRoy’s Weekly Data Chart During Science, LeRoy will ask appropriately for an easier task or for a “break” from difficult tasks without making inappropriate comments or throwing materials at least 75% of the time as measured by a daily point card. LeRoy’s Weekly Data Chart Date Staff Staff provided modified tasks & pretaught to ask for “break” or easy task Problem behavior (inappropriate comments, throwing materials) Replacement Behavior (Student asked for break or easier task)   yes no 0 1-3 4-6 7+ Please complete at the end of each Science block

Intermediate Goals: Approximations Leroy’s Short-Term Goal Leroy will ask appropriately for an easier task or for a “break” from difficult tasks without throwing materials or cursing at least 75% of the time as measured by a daily point card for 2 consecutive weeks. Leroy will ask appropriately for an easier task or for a “break” no more than 3 times during Math block with no more than 2 problem behavior incidents for 4 consecutive days . Leroy will ask appropriately to cross off up to 60% of difficult math problems and will have no more than 3 problem behavior incidents for 2 consecutive weeks. Leroy’s Long-Term Goal Leroy will complete at least 80% of his assigned work in his math class with no more than 3 incidences of problem behavior (throwing materials, cursing) for one month.

Over time focus shifts to Long Term goal Focus will shift to work completion Calculate percentage of problems completed on class work daily & graph Monday = 15/20 Tuesday = 11/12 Wednesday = 14/15 Thursday = 25/25 Friday = 11/14

Grant’s Weekly Data Chart When one of Grant’s teachers asks him to complete a folder task at his work station, Grant will often yell, tear up materials, and flop on the floor to avoid completing the task. Grant’s team has decided that during desk work, they will teach him how to use pictures to ask for a break or for an easier task. He will also have a visual reminder placed on his desk during this time. Grant’s Weekly Data Chart Date Staff Staff used visual cue (1st, then or schedule) Problem behavior (yell, destroy materials, flop) Replacement Behavior (Student used PECS book to communicate)   yes no 0 1-3 4-6 7+ Please complete at the end of each 20 min 1:1 work time block

Activity #2 (page 89) When asked to read aloud or answer questions during small group reading lessons, Charlie makes inappropriate comments (e.g., “This is so stupid”), puts her head down on the desk and refuses to comply with any of the teachers directions. The FBA shows that this problem behavior is maintained by adult attention, and the team has decided to teach Charlie to raise her hand and ask appropriately for teacher help/attention. Develop: a) a short-term goal, b) an intermediate goal, and c) a long-term goal for Charlie. Make sure that you include: observable, measureable descriptions of behavior mastery criteria

Evaluation Planning: How Will We MEASURE Progress? In addition to long- and short-term goals, the evaluation plan includes the specific data that will be collected to assess: Is the plan making an impact on student behavior?

Evaluating the BSP: Role of Plan Implementers Collect fidelity of implementation data at least 1 x per week Report any difficulties in implementing the plan to the team leader Collect data on student behavior at least 3 x per week to assess progress If problem behavior increases or escalates contact team leader immediately

Is the Plan Being Implemented? Considerations When Developing Measures of Fidelity: Does the measure capture the specific tasks/target behaviors of interest? Is the plan being implemented? Did I implement the plan? vs. Did I check in with student and provide specific praise when she entered class?

Manipulate Antecedent Example: Sheldon Desired Behavior Sitting appropriately and responding with group during circle time Consequence Feeling of success, adult praise for appropriate behavior Routine: Circle Time on Carpet Setting Event None Identified Antecedent Teacher teaching to whole class or praising another student Problem Behavior Making noises with mouth or hands, leaning on or grabbing others or materials, out of seat Consequence/Function Teacher reprimands, teacher takes to time-out Access Teacher / Adult Attention Replacement Behv Raise hand for adult attention or to answer a question Setting Events Manipulate Antecedent Teach Behavior Alter Consequences Eliminate/Neutralize Setting Events N/A Eliminate/Modify Antecedents - Sheldon will sit in front of teacher / tape outline on carpet marking his spot Weighted vest Frequent adult attention/ stickers for appropriate behavior (at least 1 x per min) Prompt Alt/Des Behavior - Pre-teach on the way to circle time (model raising hand, “safe body”) Teach Replacement Behavior Teach Sheldon to raise hand for attention Teach what it means to have a “listening ears” Teach Desired Behavior/Skills Teach appropriate sitting on carpet (use examples/non-examples; teach outside of circle time routine) Rewards Sheldon will earn stickers and teacher praise for sitting appropriately (5 stickers will = teacher recognition in front of class) Teacher will quickly respond to and praise hand raising Response to Problem At first sign of problem behavior, remind Sheldon to raise his hand using visual cue only (minimize adult attention to problem behavior) Time out (no longer than 3 min), praise and bring back after several seconds of approp behv

Example: Implementer Checklist Questions Yes (every day this week) Kind of (2-3 days this No (0 days This Notables * 1. Was the tape outline on the carpet marking Sheldon’s spot? 2. Was Sheldon given a weighted vest at the beginning of circle time? 3. Did I remind Sheldon what appropriate sitting/raising hand looks like at the beginning of circle time? 4. Did I provide stickers for hand raising, and staying in seat? 5. Did I minimize attention to problem behavior? 6. Did I provide frequent attention for appropriate behavior? 7. If Sheldon had to go to time-out, did I allow him to rejoin the group within 2 minutes? TOTALS

Manipulate Antecedent Desired Behavior Work with peers to complete group assignment Consequence Success, sense of accomplishment, peer attention Routine: Social Studies Setting Event Student gets to school late and misses breakfast Antecedent Asked to complete a project with a group (2-3 peers) Problem Behavior Whining and refusing to do work Consequence/Function Student is told to go complete her work in the office Avoids working with peers Replacement Behavior Ask appropriately if she may work on the assignment alone Setting Events Manipulate Antecedent Teach Behavior Alter Consequences Provide a.m. snack on days when Maizey arrives late and misses breakfast. When passing out assignments provide student with a choice of working with a group or completing the assignment alone Place a “reminder” card on student’s desk stating that she may ask to work alone at any point during the group task Explicitly teach Maizey: a) when it is okay to appropriately ask for a break, and b) what “appropriately asking to work alone” looks like using examples and non-examples Provide social skills training focused on how to work cooperatively with peers 3 x per week Rewards Student will be allowed to work alone when asks appropriately When Maizey works with peers for 15 min, she will be given the option to work alone Response to Problem At first sign of problem behavior, remind Maizey that she can ask to work alone. Student is told that she may work alone after she either a) asks appropriately, or b) completes one part of the task with peers

Evaluation Procedures for Maizey Data to be Collected Procedures for Data Collection Person Responsible Timeline (Implement as of 2/27) Is plan being implemented? -Verbal and visual prompts used - Pre-teaching “appropriate asking” -Rewards for app. behavior -Response to prob. behavior - Social Skills Training Miss Posey (school psych) to observe in classroom Collect daily-task checklists Check in with Miss Posey for progress in social groups Miss Posey Mrs. Ryan Mrs. Foster 1 x per week Collect 3 x per week Is plan making a difference? -# of instances of work refusal per week in social studies -Student use of hand-raising to request working alone - % group assignments completed Teacher notes # of occurrences per week in math Teacher notes daily on behavior report card Teacher grade book / permanent products Check in 3 x per week Daily for 2 weeks Weekly Plan Review Date 3/14/05 38

Activity #3 (page 91) Using the summary statement and strategies provided for Kelly on pages 91-92 in your guide, work with a partner to determine/describe: What SPECIFIC student behaviors will you collect data on?

Manipulate Antecedent Kelly Desired Behavior Stay in seat, sit quietly, complete assignments Consequence Teacher acknowledgement Routine: Math, Writing Setting Event Altercation/ reprimand on the bus on way to school Antecedent Asked to complete desk work independently Problem Behavior Talks out, makes noises, talks to/makes faces at peers, out of seat/walks around Consequence/Function Peers talk back / laugh or tell him to leave them alone Obtain Peer Attention Replacement Behavior Raise hand and ask to work with peer Setting Events Manipulate Antecedent Teach Behavior Alter Consequences Eliminate / Neutralize On days when an altercation on bus has occurred, Kelly will be given a job/task to do with peer before beginning work (ex. carrying books down to office) Eliminate / Modify Antecedents Provide modified assignments that are broken down into steps Give specific praise often in front of peers Have Kelly sit in front of teacher / away from distracting peers Prompt Replacement/Desired Behv At beginning of independent work, remind Kelly that she can earn time with peers for being on task (point sheet on desk) Teach Replacement Behavior Teach Kelly how and when to raise hand and ask to work with peer (use examples and non-examples) Teach Desired Behavior / Skills Explicitly teach what on-task behavior looks like (Teach whole class how to ignore problem behavior) Rewards Kelly will be allowed to work with a peer if she raises her hand and asks appropriately She will earn points for staying on task that can be exchanged to earn free-time with peers Response to Problem At first sign of problem behavior, redirect to alternative behavior Remind peers to ignore problem behavior and ask Kelly to work away from the group

Meeting to Review the Plan

Meeting to Review the Plan The plan is a WORKING DOCUMENT! Team members meet regularly to: a) Monitor progress b) Modify the plan as needed to: Make the plan more effective or Change mastery criteria and increase student independence

Review Meeting: Role of the Team Leader Guide team in reviewing DATA for each component of the plan to document: 1. Is each strategy being implemented as designed? 2. Is the plan resulting in change in student behavior? 3. Do data indicate that the plan needs to be modified and how? 4. What is the date of the next Review Meeting?

The Implementation Plan is used to record: 1. The extent to which the plan is being implemented and… 2. Team evaluation decisions made, based on the data presented at the meeting : 3/21/11 Add more multi-digit problems Monitor Monitor Completed/ Discontinue 7 8

Insert data graph – example of data for student outcomes

Barriers to Implementation Questions to ask if plan is not being implemented: Do implementers understand how and when to use strategies? Have implementers been provided with a way to measure implementation? Are strategies feasible in the natural setting? Are there ways that plan can be modified to make implementation more likely? *Note: If the plan is not being implemented with fidelity, we can not assess if the plan is working.

Barriers to Plan Effectiveness Questions to ask if progress is not being made: Is plan being implemented regularly and accurately? prevention, teaching & reinforcement strategies? Is student Consistently being rewarded for alt/des behavior? How often? Are reinforcers for alt/des behavior “powerful” enough? Are reinforcers functionally equivalent? Is the Problem Behavior still being Reinforced? Remember: we must minimize pay-off for problem behavior for new appropriate behavior to “compete”

The Plan is Working! Now What?? The student is making progress, but has not yet met the short term goal. Continue plan and progress monitoring The student has met the short-term goal!! Create a new short-term goal that more closely approximates the desired behavior and ASK: Can we expand the plan/implement in more contexts? Can we being fading antecedent prompts and modifications? Should we start fading and/or modifying rewards? Are there strategies that we can use to increase the student’s monitoring of his or her own behavior?

Key Points An EVALUATION PLAN for determining A) if the plan is being implemented B) if the plan is making a difference in student behavior C) when team members will meet again to discuss progress Both the Team Leader and Team Implementers collect Fidelity and Effectiveness data regularly. The Behavior Support Plan is a Work in Progress!!! Team members meet every to weeks to determine: Is the BSP being implemented as agreed on by the team? Is the student making progress? Do we need to modify the plan to: Improve effectiveness Increase student independence? When is the date of our next meeting? 49

Check #3 When developing a BSP evaluation plan, short-term goals should focus on increasing the student’s use of: The Replacement Behavior The long-term goal focuses on increasing: Desired Behavior/Skills

Check #2 Edgar’s team has met to review his progress since implementing the BSP. The data show that Edgar’s problem behavior has not decreased in the past 2 weeks. What is the first question that Edgar’s team should ask? Is the plan being implemented??

Check #3 Go to page 105 in your guidebook. In teams of 3, please select a sample scenario and use the forms provided in your guidebook (pages 106-108) to build a complete student BSP.

Example #1: Charlie At the end of “free-choice” time, when asked to transition back to her desk, Charlie verbally refuses, cries, and falls to the floor to avoid transitioning to a less preferred activity. This is most likely to occur on days when Charlie does not take her medicine before school. Charlie’s “tantrums” occur 3-4 times per week and can last up to 10 minutes.

Example #2: Garret During large-group instruction in Math, when students are asked to attend to the materials being projected on the screen at the front of the class. Garrett often turns around in his chair, gets out of his seat and walks around the room, and makes comments to or faces at peers. FBA data show that his behavior is mostly likely maintained by peer attention. Garrett is currently off-task approximately 85% of the time during large-group and he is turning in less than 50% of his math assignments.

Task Over the next two weeks… Please meet with team members at your school to develop a student plan based on FBA results. Your role as Team Leader will be to guide the team through the questions on pages 101 and 102 in your workbook and to ensure that the plan is: Complete (don’t forget about implementation and evaluation plans!) Function-Based Contextually Relevant 55 55

Comments/Questions about Module 7 At the bottom of page 109 please write any comments/questions you may have pertaining to this module. Thank you for your time & attention! 56 56