1. School Based Interventions & Becoming a Consumer of Research
P E N T
Keynote PENT Forum 2005
School Based Interventions & Becoming a Consumer of Research

2. School-Based Interventions for Preventing Antisocial Behavior: What works, what doesn’t, what’s promising?
Jeffrey Sprague, Ph.D.
The University of Oregon Institute on Violence and Destructive Behavior
PENT Forum 2005

3. Background on the problem Evidence-based or research validated?
School-Based Interventions for Preventing Antisocial Behavior: What works, what doesn’t, what’s promising?
Background on the problem
Evidence-based or research validated?
Efficacy vs. Effectiveness
Foundation for effective school-based intervention
Effective interventions
Why do they work?
PENT Forum 2005

4. Challenging Behaviors
Exist in every school and community (always will)
Vary in intensity and frequency
Are associated w/ a variety of risk factors (no single pathway)
Present our greatest public health problem!
PENT Forum 2005

5. PENT Forum 2005

6. If you always do what you always did, you’ll always get what you always got
~Moms Mably
PENT Forum 2005

7. Common Response to School Problems: Apply Sanctions
Increase monitoring and Supervision
Restate rules & sanctions
Refer to office, suspend, or expel disruptive students
PENT Forum 2005

8. Punishment practices may “work” in the short term
Referrals, suspension and expulsion produces immediate, but short-lived relief
Punishment practices may “work” in the short term
Remove student
Relieve ourselves and others
Assign responsibility for change to student &/or others (family)
PENT Forum 2005

9. But….false sense of efficacy!
Punishment practices, when used alone, promote more antisocial behavior (Mayer,1991; Skiba&Peterson, 1999)!
Vandalism, aggression, truancy, dropout
Impairs child-adult relationships and attachment to schooling
Weakens academic outcomes and maintains antisocial trajectory
PENT Forum 2005

10. PENT Forum 2005

11. Where to Start: NCLB Principles of Effectiveness
Conduct Needs Assessment
Incidence of violence and illegal drug use
Include consultation and input from parents
Include analysis of risk and protective factors
Select objective performance measures (goals and objectives)
Select Evidence-based Practices
Rigorously and periodically evaluate the programs
PENT Forum 2005

12. Conduct Needs Assessment
Incidence of Violence and Illegal Drug Use
ODR’s
California Healthy Teens
Risk and Protective Factors
Communities that care survey (Hawkins and Catalano)
Oregon School Safety Survey
Behavioral and Emotional Rating Scale
Person-centered planning
Systems assessment (what do we have in place?)
SET ( )
Assessing Behavior Support in Schools Survey ( )
PENT Tools to develop, implement, and score a behavior plan ( )
PENT Forum 2005

13. Set objective performance measures (goals and objectives)
Include consultation and input from parents
Site council
PBS team
SAT/BIP/CST (behavior SWAT team)
PENT Forum 2005

14. So what about “evidence-based” practices?
Let’s buy a Second Step kit for the school counselor and that will fix all those tough kids
I bought Project Alert, the Virtues project, and Life Skills Training for my school, and the teachers would not use it!
FBA is too hard, can’t I just make my best guess?
There are so many programs to choose from, I can’t decide what to do!
PENT Forum 2005

15. Evidence-based or research validated?
Several school-based studies indicate efficacy (Can it work?)
Research funding
Enhanced or high fidelity independent variable
We are learning more about effectiveness (What does it take to work in typical settings?)
PENT Forum 2005

16. Efficacy and Effectiveness
Efficacy: can it work?
Highly controlled research
Expensive
Effectiveness
Larger, “real world” sample
Less controlled
Usually less “effective”
PENT Forum 2005

17. Kinds of Research Qualitative: Descriptive with words (mostly)
Descriptive: Tell about it with numbers
Quasi experimental
Single Subject: Focus on individual changes and the process
Quantitative: Large samples and statistical treatments – focusing on student outcome
PENT Forum 2005

18. What is “evidence-based” in schools?
Randomized controlled trial design
Quasi-experimental controlled design
Statistically significant positive effect
Positive effect sustained for at least one year post intervention
Positive effect replicated in one or more settings and/or populations
Opinions of respected authorities
Source: U.S. Department of Education and HHS
PENT Forum 2005

19. Select Evidence-based Practices
PENT Forum 2005

20. Levels of Evidence Level I-A Experimental design
Random assignment to intervention or control group
Statistically significant positive effect, and the effect is sustained for at least 1 year post-intervention
Evidence is stronger if the beneficial effect has been replicated in one or more settings
PENT Forum 2005

21. Levels of Evidence Level I-B
Evidence obtained from at least one well-designed quasi-experimental controlled trial without randomization
Statistically significant positive effect, the effect is sustained for at least 1 year post-intervention
Evidence is stronger if the beneficial effect has been replicated in one or more settings
PENT Forum 2005

22. Levels of Evidence Level II-A
Evidence obtained using an experimental or quasi-experimental design
Outcome shows a significant positive effect
Beneficial effect has been replicated in at least one setting
Level II-B
Beneficial effect has been sustained for at least one year
PENT Forum 2005

23. Levels of Evidence Level III
Evidence obtained over time from strong and replicated results in studies with no control group
Level IV
Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees
PENT Forum 2005

24. Issues to consider? How much does the intervention cost?
Consider in relation to effect size
Are there expensive, ongoing requirements to work with the developers?
Does the intervention have a generalized, or specific effect (e.g., tobacco only)
Can teachers integrate the material into their daily routine?
Is there evidence of effectiveness (did typical people guide the intervention) vs., efficacy (did the researchers get the effect only when they ran it?
PENT Forum 2005

25. Rigorously and Periodically Evaluate
Annual surveys of students
Achievement scores
Assessing Behavioral Support survey (
System Wide Evaluation Tool
Direct observation
???
PENT Forum 2005

26. What works in school-based delinquency prevention
Building school capacity to initiate and sustain an intervention
Communicating and consistently enforcing behavioral norms
Comprehensive social skills instructional programs
self-control, stress-management, responsible decision-making, social problem-solving, and communication skills
PENT Forum 2005

27. What does NOT work
Counseling students, particularly in a peer-group context, does not reduce delinquency or substance abuse.
Offering youths alternative activities such as recreation and community service activities in the absence of more potent prevention programming does not reduce substance use.
Instructional programs focusing on information dissemination, fear arousal, moral appeal, and affective education are ineffective for reducing substance use.
PENT Forum 2005

28. What is promising?
Programs that group youth into smaller "schools-within-schools" to create smaller units, more supportive interactions, or greater flexibility in instruction.
Behavior modification programs and programs that teach "thinking skills" to high-risk youths.
Programs aimed at building school capacity to initiate and sustain innovation.
Programs that improve classroom management and that use effective instructional techniques.
PENT Forum 2005

29. PENT Forum 2005

30. Schools That Reduce Delinquency and Disruption
Shared values regarding school mission and purpose (admin, staff, families, students)
Clear expectations for learning and behavior
Multiple activities designed to promote pro-social behavior and connection to school traditions
A caring social climate involving collegial relationships among adults and students
Students have valued roles and responsibilities in the school
PENT Forum 2005

31. School wide Positive Behavior Support
Problem behaviors are defined clearly for students and staff members;
Appropriate, positive behaviors are defined for students and staff;
Students are taught these alternative behaviors directly and given assistance to acquire the necessary skills to enable the desired behavior change;
Effective incentives and motivational systems are developed and carried out to encourage students to behave differently;
PENT Forum 2005

32. School wide Positive Behavior Support
Staff commits to staying with the intervention over the long term and to monitoring, supporting, coaching, debriefing, and providing booster shots as necessary to maintain the achieved gains;
Staff receives training and regular feedback about effective implementation of the interventions; and,
Systems for measuring and monitoring the intervention's effectiveness are established and carried out.
PENT Forum 2005

33. Positive Behavior Supports
What can we expect?
Reductions in discipline problems
Improved academic achievement
Deviant peer groups less likely to form
Prevent the onset, or slow the trajectory of alcohol, tobacco and other drug use, and delinquency
PENT Forum 2005

34. What works with at-risk youth?
Universal screening and referral to services
Social and life skills instruction and support
Adult mentoring and case management
Specialized school and classroom supports
Academic
Function-based behavior support
Alternative discipline
Parent collaboration or parent training
Service coordination with community agencies
Service learning or Community Service
PENT Forum 2005

35. Does FBA “work”?
Is FBA only for students with developmental disabilities but not Charles Manson?
Why can’t I just use “clinical judgment”?
Behavioral interventions are shown to be the most effective
FBA can’t be tested in a group design because each student is unique
PENT Forum 2005

36. Does FBA work? Statistics turns 1000 tears into 1!
The ultimate validity criteria for FBA is whether it helps us design an intervention that works!
Think on your feet!
Think Functionally!
PENT Forum 2005

37. Are good behavior plans “evidence-based?”
Behavior is always related to environment
BSP’s should always include environment changes
Behavior always has a “function”
Students are never “not motivated”
Support plans must always teach and strengthen expected and replacement behaviors
Using data to make decisions improves our effectiveness
PENT Forum 2005

38. Are good behavior plans “evidence-based?”
Reactive strategies should be specified in case the behavior occurs again (it will) in four phases:
prompt to identified replacement behavior
specify how to handle the problem behavior while it is occurring
debrief following the behavior with staff and students
provide a consequence if relevant
Use data to communicate implementation and outcome of your plans
PENT Forum 2005

39. Think on your feet Think functionally
Words to live by
Think on your feet
Think functionally
Changing student behavior is about changing the environment and our own behavior
PENT Forum 2005

40. Recommended Web Resources
Institute on Violence and Destructive Behavior
SWIS data base for office referrals
OSEP Center on Positive Behavioral Interventions and Supports
Oregon Social Learning Center
Hamilton Fish Institute on School and Community Safety
Prevention Research Center (Mark Greenberg)
University of Oregon
Jeffrey Sprague, Ph.D.
PENT Forum 2005