What Works: Evidence-Based Practice in Juvenile Delinquency Dispositions James C. (Buddy) Howell Co-Director, North Carolina Evidence-Based Juvenile Justice Project Institute of Government UNC Chapel Hill March 29, 2006

Ways to Use Evaluation Research to Improve Program Practice Evaluate each program to provide feedback on implementation and outcomes. Implement a “model” program that evaluation has shown to be effective; monitor implementation compliance. Follow “best practices” guidelines for effective programs that have been derived from evaluation research; monitor compliance.

An Introduction to Evidence-Based Practice What do the following terms mean? Evidence-based programs Research-based programs Best practices They all mean the same thing: Programs that are based on scientifically sound research that shows what programs or specific services effectively reduce problem behaviors.

NC’s Comprehensive Strategy for Juvenile Delinquency Problem Behavior > Noncriminal Misbehavior > Delinquency > Serious, Violent, and Chronic Offending Prevention Target Population: At-Risk Youth Preventing youth from becoming delinquent by focusing prevention programs on at-risk youth Graduated Sanctions Target Population: Delinquent Youth Improving the juvenile justice system response to delinquent offenders through a system of graduated sanctions and a continuum of treatment alternatives >>>>>> Programs for All Youth Programs for Youth at Greatest Risk Immediate Intervention Intermediate Sanctions Community Confinement Training Schools Aftercare

The Scope of North Carolina’s Juvenile Delinquency Problem Because only a small fraction of adjudicated youths in NC are serious, violent, and chronic offenders (see next slide), the delinquency problem is quite manageable. We must use structured decision making tools, To build a continuum of effective programs With evidence-based services.

Non-Serious Non-Violent Non-Chronic 62% Serious or Violent 28% Chronic 10% Violent 2% C & V 0.5% Source: NC DJJDP Risk Assessment Data FY01-02 to FY (N=17,645) Juvenile Offender Court Careers North Carolina

AUTHORITY CONFLICT PATHWAY (before age 12) OVERT PATHWAY COVERT PATHWAY (before age 15) AGE OF ONSET: LATE EARLY %BOYS/GIRLS: FEW MANY Stubborn Behavior Defiance/Disobedience AUTHORITY AVOIDANCE (truancy, running away, staying out late) MINOR AGGRESSION (bullying, annoying others) PHYSICAL FIGHTING (physical fighting, gang fighting) VIOLENCE (rape, attack, strong-arm, homicide) MINOR COVERT BEHAVIOR (shoplifting, frequent lying) PROPERTY DAMAGE (vandalism, fire-setting) MODERATELY SERIOUS DELINQUENCY (fraud, pick-pocketing) SERIOUS DELINQUENCY (auto theft, burglary) Developmental Pathways to Serious and Violent Behavior* *Loeber©

Graduated Sanctions Component Driven by data from assessment of offender’s risk factors and treatment needs (Structured Decision Making tools) The first objective is to ensure public safety by restricting offender’s freedom to commit offenses. The second objective is to achieve a good match between offender characteristics and their treatment needs.

A Model of Graduated Sanctions Increasing SanctionsDecreasing Sanctions Diversion Youth Court Probation Intensive PS CB Resid. Residential Placement Intensive PS Probation Group Counseling Mentoring Day/Eve Report.

Key DJJDP SDM Tools DJJDP has a validated risk assessment instrument DJJDP has an effective needs/strengths assessment instrument The JJ Reform Act provided a Disposition Matrix The Disposition Matrix and risk assessment instrument are functioning well in guiding offender placements

What is risk assessment? A statistical procedure for estimating the probability that a “critical” event will occur at some time in the future. In the juvenile justice system, the “critical” event is generally a new delinquent offense committed by a juvenile offender who has been placed on probation or parole supervision.

A needs assessment is intended to do the following: Provide an overview of the level of seriousness of the juvenile offender’s treatment needs Provide information to assist court counselors in developing comprehensive treatment plans Comprehensive supervision and service plans establish specific objectives, time frames, and offender’s needs to be addressed by the responsible agencies/persons.

Disposition Matrix A disposition matrix organizes sanctions and programs by risk level and offense severity. It places offenders along a continuum of programs and sanctions Research shows that a reliable risk assessment instrument predicts differential recidivism rates at various risk levels.

Key Points of the Disposition Matrix Low risk offenders are placed in community programs with minimal supervision Medium risk offenders are typically placed in more structured community programs with intensive probation supervision High risk offenders may be placed in Youth Development Centers

North Carolina Offender Disposition Matrix Risk Level OffenseLowMediumHigh ViolentLevel 2 or 3Level 3 SeriousLevel 1 or 2Level 2Level 2 or 3 MinorLevel 1Level 1 or 2Level 2 Level 1 Community Level 2 Intermediate Level 3 Commitment to Youth Development Center

Dispositional Levels Risk Level by Disposition Low MediumHigh Total % % % % Level 1 – Community 65% 31%3%100% Level 2 – Intermediate 27%47%26% 100% Level 3 – Commitment7%23%70% 100% Protective Supervision 47%49%4%100% Total49%38%14% 100% Disposition of NC Court Referrals by Risk Level

(N=3722)(N=3197)(N=1115)

Admissions to North Carolina Youth Development Centers

A Practical Approach To Evaluating and Improving Juvenile Justice Programs Utilizing The Standardized Program Evaluation Protocol The Lipsey-Howell-Tidd Project

What is meta-analysis? “Meta” means beyond, more comprehensive A synthesis of evaluation results A technique for coding, analyzing, and summarizing research evidence; or a sophisticated way to extract, analyze, and summarize the findings of a collection of related research studies

Current database on the effects of intervention with juvenile offenders Offenders N of Studies Pre-adjudication (at-risk) 178 Court supervision (probation) 216 Committed 90 Aftercare 25 Total 509

From the bottom up: Evaluation study of intervention effects on recidivism

Recidivism Results from more than 500 Delinquency Outcome Studies (Lipsey, 2002)

A Surprising Revelation to Some Most everyday (“practical”) juvenile justice programs reduce recidivism! (see the following table) The challenge is this. Half of the evaluated everyday JJ programs do not produce meaningful reductions in recidivism. To improve this situation, JJ practitioners must use evidence-based program services and in configurations that, on average, produce larger reductions in recidivism.

Practical programs with different numbers of favorable features

Key Meta-Analysis Findings Research shows that, on average, most juvenile justice programs reduce recidivism. Most JJ programs developed by practitioners also reduce recidivism, but very little. Some programs have large effects but implementation is equally important. There is enough research to characterize best practice for most programs, but not all.

An important finding about factors associated with program outcomes Implementation is as important as treatment type Reduction in Recidivism from.50 Control Group Rate

The Standardized Program Evaluation Protocol (SPEP) What is it? A practical method for evaluating juvenile justice and delinquency prevention programs against best practices. The SPEP provides a scheme (protocol) for assigning points to programs according to how closely their characteristics match those associated with the best outcomes in research.

Four Main Characteristics Of Effective Program Services 1. The Program Type (primary service) 2. Supplemental Services 3. Amount of Service 4. Characteristics of Clients These are the four sections of the Standardized Program Evaluation Protocol (SPEP) program rating instrument

Point system based on what the research shows as “best practice” “Ideal” “Good” More Points = LessDelinquency Primary Service Supplemental Services Amount of Service Characteristics of Clients Room for Improvement

Prevention Programs (Primary Service Types) Parent training/counseling Interpersonal skills training Tutoring Group counseling Drug/alcohol therapy/counseling Employment related Individual counseling Mentoring Family counseling Effective – above average--60 points Effective – about average--50 points Effective – below average--40 points

Court Delinquency Supervision Programs (Primary Service Types) Family counseling Tutoring Mentoring Parent training/counseling Interpersonal skills training Drug/alcohol therapy/counseling Individual counseling Group counseling Employment-related Restitution Effective – above average--60 points Effective – about average--50 points Effective – below average--40 points

Programs for Committed Youths (Primary Service Types) EFFECTIVE--ABOVE AVERAGE Behavior management Cognitive-behavioral therapy Employment/job training Interpersonal skills training EFFECTIVE– ABOUT AVERAGE Family counseling Group counseling Individual counseling EFFECTIVE-- BELOW AVERAGE Career/vocational counseling Tutoring/remedial education

Example of Best Practice Rating System Based on Meta-Analysis Results: NC SPEP for Court Supervision Cases Service mix: Primary & supplementary services Amount of service completed Risk & age of juveniles served Points allocated in proportion to strength of relationship to recidivism found in meta-analysis Data on provider performance derived from client tracking forms Total points= match with “best practice”

Expected Recidivism with Features of Effective Court Delinquency Programs Comparable Juveniles not in a Program40% Average Supervision Program in Database34% Effective, Above Average Service (AAS)32% AAS +Best Supplemental Service (BSS)28% AAS +BSS+Optimal Service Amount (OSA)24% AAS +BSS+OSA+Appropriate Clients21%

Continuum Building Even if well implemented and effective, a single “model” program will do little to strengthen the overall continuum of program options. The SPEP can be used to evaluate and improve routine programs spanning the continuum from prevention to post-release supervision. At the same time, program structures must be recognized for their value in a continuum of program services and structures.

Arrest Counsel & release Diversion; Informal probation Probation Incarceration Level of Supervision Intervention Programs Recidivism Outcomes Program A Program B Program C Program D Program E Program F U% V% W% X% Y% Z% TotalReoffenseRate Prevention Programs T% Risk assessment and risk-based dispositions Effective programs Needs assessment; match needs to program Minimize reoffending

What the SPEP is Not It is not a whole blueprint for a program. It measures only the delinquency reduction potential a program type has, on average, based on prior research. It does not provide a treatment plan for individual clients, only a framework within which treatment can be planned.

Key Reference Works Available at: Howell, J. C., & Lipsey, M. W. (2004). A practical approach to linking graduated sanctions with a continuum of effective programs. Juvenile Sanctions Center Training and Technical Assistance Bulletin Vol. 2(1), Reno, NV National Council of Juvenile and Family Court Judges. Howell, J. C., & Lipsey, M. W. (2004). A practical approach to evaluating and improving juvenile justice programs. Juvenile and Family Court Journal, 55(1), Lipsey, M. W. (2002). Meta-analysis and program outcome evaluation. Socialvetenskaplig Tidskrift, 9(2-3), Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48,