School Based Interventions & Becoming a Consumer of Research P E N T Keynote PENT Forum 2005 School Based Interventions & Becoming a Consumer of Research

School-Based Interventions for Preventing Antisocial Behavior: What works, what doesn’t, what’s promising? Jeffrey Sprague, Ph.D. The University of Oregon Institute on Violence and Destructive Behavior PENT Forum 2005

Background on the problem Evidence-based or research validated? School-Based Interventions for Preventing Antisocial Behavior: What works, what doesn’t, what’s promising? Background on the problem Evidence-based or research validated? Efficacy vs. Effectiveness Foundation for effective school-based intervention Effective interventions Why do they work? PENT Forum 2005

Challenging Behaviors Exist in every school and community (always will) Vary in intensity and frequency Are associated w/ a variety of risk factors (no single pathway) Present our greatest public health problem! PENT Forum 2005

PENT Forum 2005

If you always do what you always did, you’ll always get what you always got ~Moms Mably PENT Forum 2005

Common Response to School Problems: Apply Sanctions Increase monitoring and Supervision Restate rules & sanctions Refer to office, suspend, or expel disruptive students PENT Forum 2005

Punishment practices may “work” in the short term Referrals, suspension and expulsion produces immediate, but short-lived relief Punishment practices may “work” in the short term Remove student Relieve ourselves and others Assign responsibility for change to student &/or others (family) PENT Forum 2005

But….false sense of efficacy! Punishment practices, when used alone, promote more antisocial behavior (Mayer,1991; Skiba&Peterson, 1999)! Vandalism, aggression, truancy, dropout Impairs child-adult relationships and attachment to schooling Weakens academic outcomes and maintains antisocial trajectory PENT Forum 2005

PENT Forum 2005

Where to Start: NCLB Principles of Effectiveness Conduct Needs Assessment Incidence of violence and illegal drug use Include consultation and input from parents Include analysis of risk and protective factors Select objective performance measures (goals and objectives) Select Evidence-based Practices Rigorously and periodically evaluate the programs PENT Forum 2005

Conduct Needs Assessment Incidence of Violence and Illegal Drug Use ODR’s www.swis.org California Healthy Teens Risk and Protective Factors Communities that care survey (Hawkins and Catalano) Oregon School Safety Survey Behavioral and Emotional Rating Scale Person-centered planning Systems assessment (what do we have in place?) SET (www.pbis.org ) Assessing Behavior Support in Schools Survey (www.pbis.org ) PENT Tools to develop, implement, and score a behavior plan (http://www.pent.ca.gov/behBbsps.htm ) PENT Forum 2005

Set objective performance measures (goals and objectives) Include consultation and input from parents Site council PBS team SAT/BIP/CST (behavior SWAT team) PENT Forum 2005

So what about “evidence-based” practices? Let’s buy a Second Step kit for the school counselor and that will fix all those tough kids I bought Project Alert, the Virtues project, and Life Skills Training for my school, and the teachers would not use it! FBA is too hard, can’t I just make my best guess? There are so many programs to choose from, I can’t decide what to do! PENT Forum 2005

Evidence-based or research validated? Several school-based studies indicate efficacy (Can it work?) Research funding Enhanced or high fidelity independent variable We are learning more about effectiveness (What does it take to work in typical settings?) PENT Forum 2005

Efficacy and Effectiveness Efficacy: can it work? Highly controlled research Expensive Effectiveness Larger, “real world” sample Less controlled Usually less “effective” PENT Forum 2005

Kinds of Research Qualitative: Descriptive with words (mostly) Descriptive: Tell about it with numbers Quasi experimental Single Subject: Focus on individual changes and the process Quantitative: Large samples and statistical treatments – focusing on student outcome PENT Forum 2005

What is “evidence-based” in schools? Randomized controlled trial design Quasi-experimental controlled design Statistically significant positive effect Positive effect sustained for at least one year post intervention Positive effect replicated in one or more settings and/or populations Opinions of respected authorities Source: U.S. Department of Education and HHS PENT Forum 2005

Select Evidence-based Practices PENT Forum 2005

Levels of Evidence Level I-A Experimental design Random assignment to intervention or control group Statistically significant positive effect, and the effect is sustained for at least 1 year post-intervention Evidence is stronger if the beneficial effect has been replicated in one or more settings PENT Forum 2005

Levels of Evidence Level I-B Evidence obtained from at least one well-designed quasi-experimental controlled trial without randomization Statistically significant positive effect, the effect is sustained for at least 1 year post-intervention Evidence is stronger if the beneficial effect has been replicated in one or more settings PENT Forum 2005

Levels of Evidence Level II-A Evidence obtained using an experimental or quasi-experimental design Outcome shows a significant positive effect Beneficial effect has been replicated in at least one setting Level II-B Beneficial effect has been sustained for at least one year PENT Forum 2005

Levels of Evidence Level III Evidence obtained over time from strong and replicated results in studies with no control group Level IV Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees PENT Forum 2005

Issues to consider? How much does the intervention cost? Consider in relation to effect size Are there expensive, ongoing requirements to work with the developers? Does the intervention have a generalized, or specific effect (e.g., tobacco only) Can teachers integrate the material into their daily routine? Is there evidence of effectiveness (did typical people guide the intervention) vs., efficacy (did the researchers get the effect only when they ran it? PENT Forum 2005

Rigorously and Periodically Evaluate www.swis.org Annual surveys of students Achievement scores Assessing Behavioral Support survey (www.pbis.org) System Wide Evaluation Tool Direct observation ??? PENT Forum 2005

What works in school-based delinquency prevention Building school capacity to initiate and sustain an intervention Communicating and consistently enforcing behavioral norms Comprehensive social skills instructional programs self-control, stress-management, responsible decision-making, social problem-solving, and communication skills PENT Forum 2005

What does NOT work Counseling students, particularly in a peer-group context, does not reduce delinquency or substance abuse. Offering youths alternative activities such as recreation and community service activities in the absence of more potent prevention programming does not reduce substance use. Instructional programs focusing on information dissemination, fear arousal, moral appeal, and affective education are ineffective for reducing substance use. PENT Forum 2005

What is promising? Programs that group youth into smaller "schools-within-schools" to create smaller units, more supportive interactions, or greater flexibility in instruction. Behavior modification programs and programs that teach "thinking skills" to high-risk youths. Programs aimed at building school capacity to initiate and sustain innovation. Programs that improve classroom management and that use effective instructional techniques. PENT Forum 2005

PENT Forum 2005

Schools That Reduce Delinquency and Disruption Shared values regarding school mission and purpose (admin, staff, families, students) Clear expectations for learning and behavior Multiple activities designed to promote pro-social behavior and connection to school traditions A caring social climate involving collegial relationships among adults and students Students have valued roles and responsibilities in the school PENT Forum 2005

School wide Positive Behavior Support Problem behaviors are defined clearly for students and staff members; Appropriate, positive behaviors are defined for students and staff; Students are taught these alternative behaviors directly and given assistance to acquire the necessary skills to enable the desired behavior change; Effective incentives and motivational systems are developed and carried out to encourage students to behave differently; PENT Forum 2005

School wide Positive Behavior Support Staff commits to staying with the intervention over the long term and to monitoring, supporting, coaching, debriefing, and providing booster shots as necessary to maintain the achieved gains; Staff receives training and regular feedback about effective implementation of the interventions; and, Systems for measuring and monitoring the intervention's effectiveness are established and carried out. PENT Forum 2005

Positive Behavior Supports What can we expect? Reductions in discipline problems Improved academic achievement Deviant peer groups less likely to form Prevent the onset, or slow the trajectory of alcohol, tobacco and other drug use, and delinquency PENT Forum 2005

What works with at-risk youth? Universal screening and referral to services Social and life skills instruction and support Adult mentoring and case management Specialized school and classroom supports Academic Function-based behavior support Alternative discipline Parent collaboration or parent training Service coordination with community agencies Service learning or Community Service PENT Forum 2005

Does FBA “work”? Is FBA only for students with developmental disabilities but not Charles Manson? Why can’t I just use “clinical judgment”? Behavioral interventions are shown to be the most effective FBA can’t be tested in a group design because each student is unique PENT Forum 2005

Does FBA work? Statistics turns 1000 tears into 1! The ultimate validity criteria for FBA is whether it helps us design an intervention that works! Think on your feet! Think Functionally! PENT Forum 2005

Are good behavior plans “evidence-based?” Behavior is always related to environment BSP’s should always include environment changes Behavior always has a “function” Students are never “not motivated” Support plans must always teach and strengthen expected and replacement behaviors Using data to make decisions improves our effectiveness PENT Forum 2005

Are good behavior plans “evidence-based?” Reactive strategies should be specified in case the behavior occurs again (it will) in four phases: prompt to identified replacement behavior specify how to handle the problem behavior while it is occurring debrief following the behavior with staff and students provide a consequence if relevant Use data to communicate implementation and outcome of your plans PENT Forum 2005

Think on your feet Think functionally Words to live by Think on your feet Think functionally Changing student behavior is about changing the environment and our own behavior PENT Forum 2005

Recommended Web Resources Institute on Violence and Destructive Behavior http://darkwing.uoregon.edu/~ivdb/ SWIS data base for office referrals www.swis.org OSEP Center on Positive Behavioral Interventions and Supports www.pbis.org Oregon Social Learning Center www.oslc.org Hamilton Fish Institute on School and Community Safety www.hamfish.org Prevention Research Center (Mark Greenberg) www.psu.edu/dept/prevention University of Oregon Jeffrey Sprague, Ph.D. PENT Forum 2005