Presentation on theme: "PENT Research Update September 2007. Research Phase One: Summary Forms with no training: produces 11% adequate plans Training on basic concepts: produces."— Presentation transcript:
Research Phase One: Summary Forms with no training: produces 11% adequate plans Training on basic concepts: produces 42% adequate plans One day training on BSP-QE: produces 65% adequate plans Follow-up training on BSP-QE: produces 75% adequate plans
Comparison of plan quality with no training, Six key concepts training, Round One training on BSP-QE, and Round Two training on BSP-QE 11% Adequate 89% Inadequate No Training 6 Concepts Training 42% Adequate 58% Inadequate 35% Inadequate 65% Adequate BSP-QE Training Round 1 25% Inadequate 75% Adequate BSP-QE Training Round 2
Phase Two: Current Research Exploring the connection between plan quality, plan implementation fidelity, and student outcomes. Behavior support Plan Quality Plan Implementation Fidelity Student Outcomes
Cadre Report: Relationship Between PBS Plan Quality and Student Outcomes Three ways of looking at this: 1. Decrease in problem behaviors Correlation =.43* The better the plan, the more likely the students problem behaviors will decrease. 2. Increase in general positive behaviors Correlation =.32* The higher quality the plan, the greater chance the student will increase his/her general positive behaviors. 3. Increase in child using a FERB Correlation =.24* The higher quality the plan, the more likely the team will witness an increase in FERBs.
Plan Implementation Fidelity: The Link between Plan Quality and Student Outcomes Results suggest that the better the plan in terms of substantive quality, the more likely the plan will be implemented as written. The better the plan is implemented as written, the greater the likelihood of improved student outcomes in the form of: Reductions in problem behaviors Increases in functionally-equivalent replacement behavior(s) Increase in general positive behavior Improved academic performance
Additional Research Findings: Cadre Characteristics Do Cadre member characteristics impact behavior plan quality? If so, what characteristics predict for better plans?
Cadre Characteristics and Plan Quality Two important findings: 1.The number of university courses taken in ABA significantly predicts the quality of behavior support plans. 2.The number of years working in education is negatively related to the quality of behavior support plans. That is, those cadre members more recently entering education produce, on average, better quality plans
Cadre Characteristics (cont…) These results indicate that individuals who (a) have taken courses in ABA, (b) develop the plan in the meeting, (c) have their BCBA, (d) serve individuals with low incidence disabilities, and (e) are a BICM are likely to develop plans of better quality than individuals who do not posses these characteristics. No Yes
Substantive Adequacy of Plans Plan ComponentMean Beh. Definition1.58 Predictors1.69 Environ. Support1.53 Environ. Changes1.78 Function1.45 Replacement Beh1.43 Teaching Strat.1.42 Reinforcement1.17 Reactive Strat.1.52 Beh. Goal/Objs1.04 Team Coord.1.56 Communication1.08 TOTAL SCORE16.84 Plan ComponentMean Beh. Definition1.84 Predictors1.76 Environ. Support1.80 Environ. Changes1.77 Function1.67 Replacement Beh1.66 Teaching Strat.1.63 Reinforcement1.49 Reactive Strat.1.64 Beh. Goal/Objs1.62 Team Coord.1.62 Communication1.54 TOTAL SCORE20.11 No Classes in ABA Classes in ABA
Results for 2007 Plans Plan ComponentBSP Scores Beh. Goal/Objs1.04 Communication1.08 Reinforcement1.17 Teaching Strat.1.42 Replacement Beh1.43 Behavior Function1.45 Reactive Strat.1.52 Environ. Support1.53 Team Coord.1.56 Beh. Definition1.59 Predictors1.69 Environ. Changes1.78 TOTAL SCORE17.28 Again, Behavioral Goals/Objs and Team Communication are the components with the lowest scores. The components with the highest overall ratings were Predictors of Problem Behavior and Environmental Changes, which is consistent with prior research. Of concern, is the low score obtained for the component Reinforcement Tactics. The average total plan score was 17.28, indicating that Cadre members, on average, wrote plans in the Adequate range. LEAST TO MOSTLEAST TO MOST
Comparison Scores: Cadre vs. Research Associates The largest discrepancies between Cadre Members self-ratings and Research Associates ratings were found for the following components: Team Communication Behavioral Goals/Objectives Reinforcement Tactics The Cadre Members were most accurate at rating their own plans on the following components: Environmental Changes Team Coordination Predictors of Problem Behavior Of Concern: Cadre Members rated their own plans as approaching or falling in the Superior range, whereas the Research Associates rated the Cadre Members plans as barely falling into the Good range. Targets for training Targets for fluency building
North/Central vs. South Comparison Plan Component North Cadre Members South Cadre Members Beh. Definition 1.631.54 Predictors1.771.62 Environ. Support 1.541.51 Environ. Changes 1.831.74 Function1.541.38 Replacement Beh 1.491.38 Teaching Strat. 1.501.35 Reinforcement1.201.14 Reactive Strat. 1.591.45 Beh. Goal/Objs 1.071.01 Team Coord. 1.611.52 Communication1.081.07 TOTAL SCORE 17.9316.70 Overall, Cadre members in North developed plans of better quality than Cadre members in the South. Cadre members in North obtained higher scores on ALL of the components. Both North and South Cadre members obtained the lowest scores on Behavior Goals/Objectives and Team Communication.
Returning vs. New Cadre Members Plan Component Returning Cadre Members New Cadre Members Beh. Definition 1.601.54 Predictors1.671.72 Environ. Support 1.561.44 Environ. Changes 1.791.75 Function1.441.49 Replacement Beh 1.361.61 Teaching Strat. 1.381.52 Reinforcement1.181.15 Reactive Strat. 1.511.52 Beh. Goal/Objs 1.051.02 Team Coord. 1.551.59 Communication1.091.03 TOTAL SCORE 17.2416.36 Results: Continuing to attend PENT Forums improves plans
Phase Three: Whats Next in Research? Forum 2007: Training in BSP Desk Reference: Do plans submitted for Forum 2008 reflect both individual and group improvement? Forum 2008: Cadre members receive back their critiqued 2007 plan and participate in targetted instruction; Will plans submitted in 2009 reflect specific critical feedback improves plan adequacy? Forum 2008: All plans evaluated as falling in the adequate range will be awarded silver star status
Take home messages The BSP form validly embodies the key concepts in behavior analysis Use this form and reference the research base The BSP QE II is a valid and reliable instrument for evaluating behavior plans Encourage all staff to periodically score their plans A training that teaches BSP QE II produces the best plans Use your Cadre Member for training on this instrument
Take home messages (cont) The BSP DESK Manual is available free on the PENT website Encourage TEAMS to use this manual in developing behavior plans PENT Forums are improving outcomes Keep your Cadre members coming to PENT Forums, and replace members as needed