Presentation on theme: "Examining Strategies to Increase Behavior Support Plan Implementation: Contextual Fit, Training, Performance Feedback. Jeffery Barker, Mary Jo Brackett,"— Presentation transcript:
1 Examining Strategies to Increase Behavior Support Plan Implementation: Contextual Fit, Training, Performance Feedback.Jeffery Barker, Mary Jo Brackett, Sally Geary, Renee Hathorn, Stacy Henderson, Brian McKenzie, Colin Ryan, Alicia Teays, and Roland Wilson.Portland State University
2 IntroductionApproximately 1% to 5% of students exhibit chronic and intense externalizing behavior as manifested by: aggression, property destruction and antisocial behavior (Sugai et al., 2000).These students account for 50% of referrals.When plan implementation deteriorates, treatments/interventions may be less effective
3 Introduction Cont.Federal law has attempted to address the difficulty of educating students who require more supportThe IDEA (1997)and IDEIA (2004) require use of function-based positive behavior support plans based on functional behavior assessments (Ingram, Lewis-Palmer,& Sugai, 2005).
4 Implementation Fidelity Professionals in education are experiencing cuts in funding and dwindling resources via time and adequate training.Function-based behavior plans must be feasible and implemented as intendedThe best developed plans make no difference if they are not implemented with integrity.
5 Barriers to Implementation Lack of trainingLack of timeDifficulty implementing behavior plans in their setting.Bambara, Goh, Kern, & Grace 2012Potential SolutionsImplementing a quality BSP requires an active team process.The use of performance feedback shows promise in promoting implementation of PBSPs (Solomon, Klein, Politylo, 2012).
6 Reinforcement & Feedback Studies show that reinforcement for teachers can be a factor in implementation fidelity (Cossairt, Hall & Hopkins, 1973).Research indicates that frequent performance feedback results in a higher level of implementation fidelity (Jones, Wickstrom & Friman, 1997).
7 Research QuestionsHow does implementation fidelity vary as a function of:Performance FeedbackContextual Fit to ImplementationTraining of Implementation?Do student outcomes appear to be related to implementation fidelity?
8 Methods Settings and Participants Measures Contextual Fit Training Performance FeedbackReinforcement
9 Participants/Settings 9 Students7 students with IEP’s4 in inclusive settings, 3 in self- contained settings2 students not identified1 in transition/day treatment program1 in general education environmentImplementers3 general education teachers3 SPED teachers5 SPED assistants
10 Settings Schools 4 elementary schools 3 middle schools 1 high school 1 post secondary
11 Measures: Contextual Fit PurposeHan and Weiss (2005) found that in order for a program or plan to be successful, it must be acceptable to the staff implementing it.Assessment needed to determine the fit of the plan within the implementer's environment, resources, and values.Adapted from the “Self Assessment of Contextual Fit” survey developed by Horner, Salentine, & Albin (2003)Chosen for it’s comprehensive assessment of contextual fit and for its ease of useMajor changes included:Condensing eight categories down to threePairing/combining questions to fit into these categoriesEliminating categories and questions due to redundancy.
12 Measures: Contextual Fit Cont. Category Example“Knowledge of elements and Skills Needed to Implement the Behavior Support Plan.”Question Examples“I am aware of the elements of this behavior support plan and the steps that I am expected to do to implement this plan.”1 2 3 4 5 6Strongly Moderately Barely Barely Moderately Strongly Disagree Disagree Disagree Agree Agree Agree“I have the skills needed to implement this behavior support plan.” 1 3 4 5 6
13 Measures: TrainingAllen and Forman (1984), Sarakoff et al. (2004), and Digennaro-Reed et al. (2010) found that multiple strategies can be effective in teacher training.These include:-Didactic training -Video Modeling-Prompting -Modeling-Feedback -Role Playing-Reinforcement
14 Measures: Training Cont. Training was measured through self-report on a Likert-scale questionnaire designed by the research team to reflect the key components found in the literature review.
15 Measures: Training Cont. Name DateRate the extent that you trained your intervention implementer to implement the targeted intervention:Implementer name:Intervention:The student was present when the implementer was provided the description or opportunity to model or practice the intervention? Y NThe description/model/practice occurred in the targeted setting where the intervention is supposed to occur? Y NGive the implementer a copy of the BSP to look at on their ownVerbal description provided to implementerModel/Demonstra te the interventionHave implementer practice doing the interventionProvide feedback to implementer while they practice implementation12345
16 Measurements: Performance Feedback Five aspects of feedback were measured.FrequencyRuhl (2004) and Solomon (2012) found that high frequency of feedback increased implementation fidelity.Scoring: ☐1 Never ☐2 Once a Week ☐3 Twice Weekly ☐4 Daily ☐5 Hourly
17 Measurements: Performance Feedback Cont. 2. Mode of feedback deliveryMortenson (2004) found that this can be tailored to fit specific teaching environments.Scoring: ☐1 No Feedback ☐2 ☐3 Written Note ☐4 Call/Skype ☐5 In Person
18 Measurements: Performance Feedback Cont. 3. Time LapseCodding et al. (2005) found that decreased time between observation and feedback delivery increased improvements in implementation fidelity.Scoring:☐1 No Feedback Given ☐2 End of Week ☐3 End of Day ☐4 Immediate ☐5 Real Time
19 Measurements: Performance Feedback Cont. 4. Type of feedbackHagermoser Sanetti et al. (2007) found that combinations of types of feedback increased implementation fidelity.Scoring:☐1 Descriptive feedback (vague in nature, no specific details noted, either strengths or needed improvements)☐2 Descriptive feedback (specific in nature, strengths and/or improvements needed noted)☐3 Numeric feedback (feedback scaled on numeric scale)☐4 Graphic feedback (vague in nature, not explained or hard to understand meaning of graphs)☐5 Graphic feedback (specific in nature, easy to understand, clearly shows successes or areas in need of improvement)
20 Measurements: Performance Feedback Cont. 5. ReinforcementStudies have shown that both positive (Cossairt et al., 1973) and negative (DiGennaro et al., 2005) reinforcement increased implementation fidelity.Scoring:☐1 No Reinforcement Involved ☐2 Reinforcement Involved, but very limited (present times, not rewarding to individual) ☐3 Reinforcement Involved, limited (present 4-6 times, marginal level of reward to individual) ☐4 Reinforcement Involved, moderate (present at most sessions, rewarding to individual) ☐5 Reinforcement Involved (present at all sessions where implementation fidelity met, highly rewarding to individual)
21 Measures: Student Outcomes Each researcher created their own student outcomes measurement tool specifically designed to fit their individual case.These tools were used daily to collect student outcome data.Example:Random Acts of Koolness! Date:_________________________0 – Not Kool at all Kool, but had to be reminded 2- Way Kool!OpportunityStaff scoreStaff initialsEH scoreP3 to P41st 5 minutes P4P6 to P71st 5 minutes of P7P7 to HR1st 5 minutes of HR
22 Measures: Implementation Fidelity Each researcher created their own implementation fidelity measurement tool specifically designed to fit their individual case.These tools were used daily to collect implementation fidelity data.Example: Implementation IntegritySuccesses: __________________________________________________________________________________Parent Signature______________________________________________________________________________DateUsed White BoardGave reminders and promptsAllowed BreaksChecked in and outTotal Points = ________________Points Possible =Today _________________%Goal _________________%
23 ProceduresEach Researcher performed a Functional Behavioral Assessment to access their targeted student, which was instrumental in developing a Behavior Support Plan, and developed a plan to support Implementation Fidelity consisting of:Contextual fitTrainingPerformance Feedback and ReinforcementImplemented the plan and collected data.Compiled and reviewed data, making adjustments when necessary
24 Research Design Descriptive Case Study Collected data Interviews, existing records, and observational studies9 Single case studiesOnly 2 of the 9 includes baseline dataExamine trends across case studies related to implementation fidelity x implementation supports
28 Individual Case Data & Mean (Impl. Supports, Fidelity & Outcomes) remaalcrshbmsgrwjMeanContextual Fit8081.696.610098.399.59394Training604067Performance Feedback726448689264.4Implementation Supports (% In Place)66.671.285.561.575.463.197.372.474Implementation Fidelity (% of impl)837999968792.592.3Student Outcomes (% of points earned)67.966.7857676.396.189.494.583.5
30 High Implementation Fidelity Mean Implementation fidelity was 92% across cases with low of 79%Paying attention to Implementation Supports seemed to have an impact:Contextual FitTrainingPerformance Feedback & Reinforcement
31 Contextual FitResearcher/Practitioners designed interventions that had high levels of Contextual Fit as rated by implementers:Feasibility: Educators were confident they could implement plan in environmentEffective: Plan was appropriate in response to problem behavior and could result in positive behavior
32 TrainingRatings on Training Provided scoring rubric ranged from “2 – Verbal Description provided to implementer” to “5 - Provide feedback to implementer while they practice implementation”w/ Demonstration and Modeling: Implementation was 96% with fidelityEven w/ only “Verbal Description without Modeling” implementation remained highPossibly due to high ratings on Contextual Fit and Performance Feedback
33 Performance Feedback Ranges conducted Hourly/Daily/Weekly/Biweekly All levels showed consistent implementation fidelity suggesting that varying levels of Performance Feedback can be used to support implementation.This is particularly true with interventions that have high contextual fit and effective training up front.
34 ReinforcementImplementers rated levels of reinforcement received during intervention55% reported no reinforcement involved. Implementation fidelity averaged (91%).33% reported moderate reinforcement involved. Implementation fidelity averaged (92%).Implementers reported it had been a rewarding experience…. Simply having someone check in and provide verbal feedback
35 Implications for Practice When contextual fit is good and when training and regular feedback are given implementation fidelity is more likely to be achievedTraining- Verbal description of the intervention may be sufficient to support fidelity of the plan (when the plan has strong contextual fit & is paired w/ performance feedback)Performance feedback is important, in some cases, as little as weekly feedback may be sufficient, if a plan has strong contextual fit
36 Limitations This was only a case study, not an experimental study So we cannot isolate the specific impact that training, contextual fit or performance feedback had on implementationIn most cases ratings of student outcomes and implementation fidelity were designed to be practical to a classroom situation, which could lead to some subjectivity and bias in ratingsCase study data was constrained by time, no long term data.
37 Future ResearchConduct an experimental study to determine what the precise impact of contextual fit, training & performance feedback has on implementation.What is the most effective and efficient frequency for providing performance feedback (weekly, etc.)?What would the function be of having a structured feedback loop in a behavior support plan?Most efficient use of timeHow does overall class structure affect implementation fidelityPeer experience, relationships
38 ConclusionAs discussed in our Literature review, our research also foundDesigning a Behavior Support Plan with Contextual Fit, Increases the likelihood of implementation fidelityPerformance feedback in combination with training also increases likelihood of implementation fidelityFrequent performance feedback with individuals administering an intervention often results in a higher level of implementation fidelityHigher implementation fidelity results in greater desired behavior success with student