Presentation on theme: "Examining Strategies to Increase Behavior Support Plan Implementation: Contextual Fit, Training, Performance Feedback. Jeffery Barker, Mary Jo Brackett,"— Presentation transcript:
Examining Strategies to Increase Behavior Support Plan Implementation: Contextual Fit, Training, Performance Feedback. Jeffery Barker, Mary Jo Brackett, Sally Geary, Renee Hathorn, Stacy Henderson, Brian McKenzie, Colin Ryan, Alicia Teays, and Roland Wilson. Portland State University
Introduction Approximately 1% to 5% of students exhibit chronic and intense externalizing behavior as manifested by: aggression, property destruction and antisocial behavior (Sugai et al., 2000). These students account for 50% of referrals. When plan implementation deteriorates, treatments/interventions may be less effective
Introduction Cont. Federal law has attempted to address the difficulty of educating students who require more support The IDEA (1997)and IDEIA (2004) require use of function-based positive behavior support plans based on functional behavior assessments (Ingram, Lewis-Palmer,& Sugai, 2005).
Implementation Fidelity Professionals in education are experiencing cuts in funding and dwindling resources via time and adequate training. Function-based behavior plans must be feasible and implemented as intended The best developed plans make no difference if they are not implemented with integrity.
Barriers to Implementation Lack of training Lack of time Difficulty implementing behavior plans in their setting. – Bambara, Goh, Kern, & Grace 2012 Potential Solutions – Implementing a quality BSP requires an active team process. – The use of performance feedback shows promise in promoting implementation of PBSPs (Solomon, Klein, Politylo, 2012).
Reinforcement & Feedback Studies show that reinforcement for teachers can be a factor in implementation fidelity (Cossairt, Hall & Hopkins, 1973). Research indicates that frequent performance feedback results in a higher level of implementation fidelity (Jones, Wickstrom & Friman, 1997).
Research Questions How does implementation fidelity vary as a function of: a)Performance Feedback b)Contextual Fit to Implementation c)Training of Implementation? Do student outcomes appear to be related to implementation fidelity?
Methods Settings and Participants Measures Contextual Fit Training Performance Feedback – Reinforcement
Participants/Settings 9 Students – 7 students with IEP’s 4 in inclusive settings, 3 in self- contained settings – 2 students not identified 1 in transition/day treatment program 1 in general education environment Implementers – 3 general education teachers – 3 SPED teachers – 5 SPED assistants
Settings Schools – 4 elementary schools – 3 middle schools – 1 high school – 1 post secondary
Measures: Contextual Fit Purpose – Han and Weiss (2005) found that in order for a program or plan to be successful, it must be acceptable to the staff implementing it. – Assessment needed to determine the fit of the plan within the implementer's environment, resources, and values. Adapted from the “Self Assessment of Contextual Fit” survey developed by Horner, Salentine, & Albin (2003) – Chosen for it’s comprehensive assessment of contextual fit and for its ease of use – Major changes included: Condensing eight categories down to three Pairing/combining questions to fit into these categories Eliminating categories and questions due to redundancy.
Measures: Contextual Fit Cont. Category Example – “Knowledge of elements and Skills Needed to Implement the Behavior Support Plan.” Question Examples – “I am aware of the elements of this behavior support plan and the steps that I am expected to do to implement this plan.” Strongly Moderately Barely Barely Moderately Strongly Disagree Disagree Disagree Agree Agree Agree – “I have the skills needed to implement this behavior support plan.” Strongly Moderately Barely Barely Moderately Strongly Disagree Disagree Disagree Agree Agree Agree
Measures: Training Allen and Forman (1984), Sarakoff et al. (2004), and Digennaro-Reed et al. (2010) found that multiple strategies can be effective in teacher training. These include: -Didactic training-Video Modeling -Prompting-Modeling -Feedback-Role Playing -Reinforcement
Measures: Training Cont. Training was measured through self-report on a Likert-scale questionnaire designed by the research team to reflect the key components found in the literature review.
Measures: Training Cont. Give the implementer a copy of the BSP to look at on their own Verbal description provided to implementer Model/Demonstra te the intervention Have implementer practice doing the intervention Provide feedback to implementer while they practice implementation Name Date Rate the extent that you trained your intervention implementer to implement the targeted intervention: Implementer name: Intervention: The student was present when the implementer was provided the description or opportunity to model or practice the intervention?YN The description/model/practice occurred in the targeted setting where the intervention is supposed to occur? YN
Measurements: Performance Feedback Five aspects of feedback were measured. 1.Frequency Ruhl (2004) and Solomon (2012) found that high frequency of feedback increased implementation fidelity. Scoring: ☐ 1 Never ☐ 2 Once a Week ☐ 3 Twice Weekly ☐ 4 Daily ☐ 5 Hourly
Measurements: Performance Feedback Cont. 2. Mode of feedback delivery Mortenson (2004) found that this can be tailored to fit specific teaching environments. Scoring: ☐ 1 No Feedback ☐ 2 ☐ 3 Written Note ☐ 4 Call/Skype ☐ 5 In Person
Measurements: Performance Feedback Cont. 3. Time Lapse Codding et al. (2005) found that decreased time between observation and feedback delivery increased improvements in implementation fidelity. Scoring: ☐ 1 No Feedback Given ☐ 2 End of Week ☐ 3 End of Day ☐ 4 Immediate ☐ 5 Real Time
Measurements: Performance Feedback Cont. 4. Type of feedback Hagermoser Sanetti et al. (2007) found that combinations of types of feedback increased implementation fidelity. Scoring: ☐ 1 Descriptive feedback (vague in nature, no specific details noted, either strengths or needed improvements) ☐ 2 Descriptive feedback (specific in nature, strengths and/or improvements needed noted) ☐ 3 Numeric feedback (feedback scaled on numeric scale) ☐ 4 Graphic feedback (vague in nature, not explained or hard to understand meaning of graphs) ☐ 5 Graphic feedback (specific in nature, easy to understand, clearly shows successes or areas in need of improvement)
Measurements: Performance Feedback Cont. 5. Reinforcement – Studies have shown that both positive (Cossairt et al., 1973) and negative (DiGennaro et al., 2005) reinforcement increased implementation fidelity. – Scoring: ☐ 1 No Reinforcement Involved ☐ 2 Reinforcement Involved, but very limited (present 1-3 times, not rewarding to individual) ☐ 3 Reinforcement Involved, limited (present 4-6 times, marginal level of reward to individual) ☐ 4 Reinforcement Involved, moderate (present at most sessions, rewarding to individual) ☐ 5 Reinforcement Involved (present at all sessions where implementation fidelity met, highly rewarding to individual)
Measures: Student Outcomes OpportunityStaff scoreStaff initialsEH score P3 to P st 5 minutes P P6 to P st 5 minutes of P P7 to HR st 5 minutes of HR0 1 2 Each researcher created their own student outcomes measurement tool specifically designed to fit their individual case. These tools were used daily to collect student outcome data. Example: Random Acts of Koolness! Date:_________________________ 0 – Not Kool at all 1- Kool, but had to be reminded 2- Way Kool!
Measures: Implementation Fidelity DateUsed White Board Gave reminders and prompts Allowed BreaksChecked in and out Total Points = ________________ Points Possible = 24 Today _________________% Goal _________________% Each researcher created their own implementation fidelity measurement tool specifically designed to fit their individual case. These tools were used daily to collect implementation fidelity data. Example: Implementation Integrity Successes: __________________________________________________________________________________ Parent Signature______________________________________________________________________________
Procedures Each Researcher performed a Functional Behavioral Assessment to access their targeted student, which was instrumental in developing a Behavior Support Plan, and developed a plan to support Implementation Fidelity consisting of: – Contextual fit – Training – Performance Feedback and Reinforcement – Implemented the plan and collected data. – Compiled and reviewed data, making adjustments when necessary
Research Design Descriptive Case Study – Collected data Interviews, existing records, and observational studies – 9 Single case studies – Only 2 of the 9 includes baseline data Examine trends across case studies related to implementation fidelity x implementation supports
Individual Case Data & Mean (Impl. Supports, Fidelity & Outcomes) remaalcrshbmsgrwjMean Contextual Fit Training Performance Feedback Implementation Supports (% In Place) Implementation Fidelity (% of impl) Student Outcomes (% of points earned)
High Implementation Fidelity Mean Implementation fidelity was 92% across cases with low of 79% Paying attention to Implementation Supports seemed to have an impact: – Contextual Fit – Training – Performance Feedback & Reinforcement
Contextual Fit Researcher/Practitioners designed interventions that had high levels of Contextual Fit as rated by implementers: Feasibility: Educators were confident they could implement plan in environment Effective: Plan was appropriate in response to problem behavior and could result in positive behavior
Training Ratings on Training Provided scoring rubric ranged from “2 – Verbal Description provided to implementer” to “5 - Provide feedback to implementer while they practice implementation” – w/ Demonstration and Modeling: Implementation was 96% with fidelity – Even w/ only “Verbal Description without Modeling” implementation remained high Possibly due to high ratings on Contextual Fit and Performance Feedback
Performance Feedback Ranges conducted Hourly/Daily/Weekly/Biweekly – All levels showed consistent implementation fidelity suggesting that varying levels of Performance Feedback can be used to support implementation. – This is particularly true with interventions that have high contextual fit and effective training up front.
Reinforcement Implementers rated levels of reinforcement received during intervention 55% reported no reinforcement involved. Implementation fidelity averaged (91%). 33% reported moderate reinforcement involved. Implementation fidelity averaged (92%). – Implementers reported it had been a rewarding experience…. Simply having someone check in and provide verbal feedback
Implications for Practice When contextual fit is good and when training and regular feedback are given implementation fidelity is more likely to be achieved – Training- Verbal description of the intervention may be sufficient to support fidelity of the plan (when the plan has strong contextual fit & is paired w/ performance feedback) – Performance feedback is important, in some cases, as little as weekly feedback may be sufficient, if a plan has strong contextual fit
Limitations This was only a case study, not an experimental study – So we cannot isolate the specific impact that training, contextual fit or performance feedback had on implementation In most cases ratings of student outcomes and implementation fidelity were designed to be practical to a classroom situation, which could lead to some subjectivity and bias in ratings Case study data was constrained by time, no long term data.
Future Research Conduct an experimental study to determine what the precise impact of contextual fit, training & performance feedback has on implementation. What is the most effective and efficient frequency for providing performance feedback (weekly, etc.)? What would the function be of having a structured feedback loop in a behavior support plan? – Most efficient use of time How does overall class structure affect implementation fidelity – Peer experience, relationships
Conclusion As discussed in our Literature review, our research also found – Designing a Behavior Support Plan with Contextual Fit, Increases the likelihood of implementation fidelity – Performance feedback in combination with training also increases likelihood of implementation fidelity – Frequent performance feedback with individuals administering an intervention often results in a higher level of implementation fidelity – Higher implementation fidelity results in greater desired behavior success with student