What Makes A Juvenile Justice Program Evidence Based? Clay Yeager Policy Director, Blueprints for Healthy Youth Development

‘In theory, there’s no difference between theory and practice. In practice, there is’

Traditional Policy and Practice Early 1970’s – ‘Nothing works!’; therefore ANYTHING is worth trying If it sounds good, feels good or looks good: it must be good ‘If it helps just one kid, it’s worth it’ Reacting to political winds: Crack cocaine, boot camps, Scared Straight, get tough policies, super predators, Columbine, zero tolerance, etc. etc. etc. “I don’t care what the science says, the amount of evidence I’ve made up in my own mind is irrefutable’

Systems Know What They Spend, Not What They Buy To understand juvenile justice, must first understand procurement and funding Policy is traditionally is determined by how resources are distributed across programs Programs are determined by who is qualified to deliver services Services are what are actually purchased Traditional juvenile justice systems contract with program providers to buy outputs - not outcomes

A Guide to More Effective Governance Systems should buy outcomes – not outputs Fix how systems share information Evaluate current programs – invest in those that work, divest in those that don’t and evaluate those that look promising Connect the right kids to the right program(s) Scale up effective, proven programs, interventions and practices Implement with fidelity, don’t cut corners, monitor closely

Launched in mid-90’s - University of Colorado Center for the Study and Prevention of Violence and OJJDP Resource for government agencies, foundations, and community organizations to make informed decisions Identifies programs, based on scientific evaluations, having strong evidence of effectiveness Standards are widely recognized as the most rigorous in use. Programs are rated as either Promising or Model

Evaluation quality—Can we be confident in a program’s evaluation? Intervention impact—How much positive change in key outcomes can be attributed to the intervention? Intervention specificity—Is the intervention focused, practical, and logical? Dissemination readiness—Does the program have the necessary support and information to be successfully implemented?

Promising Programs (42): Evaluation quality: The evaluation trials produce valid and reliable findings. This requires a minimum of (a) one high quality randomized control trial or (b) two high quality quasi-experimental evaluations. Model Programs (15): Evaluation Quality: A minimum of (a) two high quality randomized control trials or (b) one high quality randomized control trial plus one high quality quasi- experimental evaluation. Positive intervention impact is sustained for a minimum of 12 months after the program intervention ends.

Multiple Clearinghouses Available Blueprints for Healthy Youth Development (Blueprints) California Evidence-Based Clearinghouse for Child Welfare (CEBC) Coalition for Evidence-Based Policy (Coalition) CrimeSolutions.gov (Crime Solutions) Office of Juvenile Justice and Delinquency Prevention Model Programs Guide (OJJDP) National Registry of Evidence-Based Programs and Practice (NREPP) Promising Practices Network (PPN) What Works Clearinghouse (WWC) What Works in Reentry Clearinghouse (WWR)

Exploration of Evidence‐Based Practices Needs: how well the program or practice meets identified needs. Fit: with current initiatives, priorities, structures and and community values. Resource Availability: for training, staffing, technology supports, data systems and administration. Evidence: of expected outcomes if the program is implemented well. Readiness for Replication: and how well the program is operationalized Capacity to Implement: as intended and to sustain and improve over time.

A Framework for Implementation ‘Doing more research on the serum will not produce a better syringe Effective Innovations – WHAT needs to be done Effective Implementation – HOW to establish what needs to be done and WHO will do it Enabling Contexts – WHERE effective interventions and effective implementation will thrive Adapted from the National Implementation Research Network

Georgia Juvenile Justice Reform Initiative Effective Innovations – fund ONLY programs meeting a high standard of effectiveness (Crime Solutions, Blueprints) Effective Implementation – dissemination readiness, training and TA, closely monitor, fidelity adherence, resource accountability Enabling Context – Governor Deal leadership, bi-partisan political will, Criminal Justice Coordinating Council, judicial partnerships Significant Outcomes – (reductions in placement): 15% goal; 62% outcome

Georgia Juvenile Justice Reform: Formula for Success in Action

Initial Steps: Understand range of evaluation standards Stop implementing harmful programs Evaluate all non-evidence based programs Give Priority to evidence based programs when they fit with local need Monitor return on investment measuring outcomes Use Blueprints as a Consumer Reports or FDA type registry of evidence-based programs

Resources Blueprints for Healthy Youth Development CrimeSolutions.gov Centers for Disease Control, National Center for Injury Prevention and Control, Violence Prevention Division National Implementation Research Network (NIRN) Pew Charitable Trusts Results First Clearinghouse Evidence Based Associates Contact