1. Washington State’s Experience With Research-Based Juvenile Justice Programs May 19, 2005 Robert Barnoski, Ph.D. (Barney) Washington State Institute for Public Policy www.wsipp.wa.gov

2. 2 Washington State Institute for Public Policy Created in 1983 by the state Legislature Mission: Carry out non-partisan research on projects assigned either by the legislature or the Institute’s Board of Directors 8 legislators 4 higher education provosts or presidents 4 state agency directors

3. 3 What We Will Talk About Today This presentation describes Washington State's experiences implementing research- based programs Some History Outcome Evaluations Quality Assurance Cost Benefit Cost Benefit Meta-Analysis

4. 4 Juvenile Justice in Washington State Determinant sentencing since 1977; based on prior record and current offense Two systems: county vs. state –Each county runs juvenile court, detention, and probation –State runs juvenile correctional institutions, parole, and distributes state funds to counties (Juvenile Rehabilitation Administration – JRA) Juvenile courts also manage non-offender cases: dependency, youth-at-risk, truancy, etc.

5. 5 History of Legislation 1995 Funding of intensive probation program. 1996 Preliminary outcomes are not good. 1997 Determine if research-based programs exist and can be implemented in Washington State juvenile courts. Eliminate parole but establish intensive parole in JRA. 1998 Switch funding from intensive probation to research-based programs in juvenile courts. Re-establish parole in JRA. 2002 Preliminary research-based program outcomes are positive and point to competent delivery. Parole and intensive parole outcomes are not good. 2003 Develop adherence and outcome standards for research- based programs. JRA moves toward research-based and Functional Family Parole (FFP). Cost-benefit analyses of prevention and intervention programs beyond juvenile offenders.

6. 6 1997: Community Juvenile Accountability Act (CJAA) Legislature said it would provide funding if research-based programs could be identified State, juvenile court administrators, and the Institute met regularly to identify “research-based” programs Six programs were identified, four were recommended 1998: Legislature funded the programs ~$3.3 million/year 1999: Programs started 2001: Evaluation groups filled 2003: Evaluation completed Initiation of Research-Based Effort in Washington State

7. 7 1.Find research-based programs scientifically shown to work 2.Develop an assessment to identify the most appropriate program for each youth 3.Implement quality assurance to ensure services are delivered as designed 4.Conduct a valid outcome evaluation Four Things Were Needed to Implement Research-Based Programs

8. 8  Functional Family Therapy (FFT)  Aggression Replacement Training (ART)  Coordination of Services  Multi-Systemic Therapy (MST)  Multidimensional Therapeutic Foster Care  Adolescent Diversion First Thing: Research-Based Program List

9. 9 Functional Family Therapy ($2,100 per youth) Blueprint Program: University of Colorado’s Center for the Study and Prevention of Violence Structured family-based intervention to enhance protective and reduce risk factors in the family FFT is a three-phase program Motivate family toward change Teach family how to change a specific critical problem Help family generalize their problem-solving skills Trained therapists have caseloads of 10 to 12 families Involves about 12 visits during a 90-day period

10. 10 Aggression Replacement Training ($700 per youth) Group training of 10 youth by two instructors Classes meet for one hour three times per week for 10 weeks Aggression cycle Skill development Moral reasoning Training, not therapy, that uses guided discussion, modeling, role play, and homework

11. 11 Coordination of Services ($400 per youth) Educational program for low-risk juvenile offenders and their parents. The goals: Describe the consequences of delinquent behavior Stimulate goal setting Review the strengths of youth and family Explain what resources are available Participants given vehicle to open lines of communication and make shifts in thinking Community groups present participants with information concerning services they provide

12. 12  Pre-screen for level of risk – more intensive service to higher risk youth and families; minimal for low risk  Comprehensive assessment only for higher risk youth – 10 life domains of risk and protective factors  Motivational interview to engage youth and family  Re-assess dynamic factors for progress – individual and groups  Common language for talking about youth and families Second Thing: Build the Washington Juvenile Court Assessment

13. 13 Mapping Problem to Intervention for Moderate to High Risk Youth Problem DomainIntervention Family (Primary)Functional Family Therapy (FFT), Multi-Systemic Therapy (MST), and Mentoring No FamilyMulti-Dimensional Treatment Foster Care AggressionAggression Replacement Training (ART) ImpulsiveDialectical Behavior Therapy (DBT) or ART Mental HealthDBT or Family Integrated Treatment (FIT) Sexual Behavior? - ART, DBT, FIT Alcohol/Drugs? - ART, DBT, FIT AttitudesART, DBT SkillsART, DBT School, Employment, Free Time, Relationships

14. 14 Case Management: Sequence to Self Reliance If family has problems: Family => Engage & motivate family => Specific Problems => Generalization If no family problems: Engage & motivate youth => Attitudes & Skills => Specific Problems => Generalization

15. 15 Role of Probation in Rehabilitation Assess youth’s strengths and weaknesses Assign youth to right caseload (low risk caseload) Assign youth to right program(s) Engage and motivate youth and family – Courts: motivational interview and change cycle – JRA: Functional Family Probation Support and help youth and family generalize what is learned during research-based program Deflect enabling undesirable behavior Monitor changes in risk and protective factors to see if approach is making a difference and adjust Does not make job easier, but might make it more effective

16. 16 Assessment Proliferation Must be something to WSJCA, since it is being used in at least a dozen states (including New York, Illinois, Florida, Utah, N. Dakota) and multiple courts – some in California Service providers particularly seem to like it We encourage continual refinement and improvement of the assessment; capitalizing on what has been done It is evolving to be a systematic collection of relevant social file information

17. 17 Third Thing: Quality Assurance Infrastructure Steering committee Contract with program designers Statewide program experts Regional consultants Initial training On-going consultation, feedback, and training Assessment of competent program delivery

18. 18 A “rigorous” test Not enough funding for everyone, so “Waiting List” youth assigned to a control group by court staff Includes completers and non-completers Youth’s eligibility for program is based on risk assessment Level of risk for re-offending Profile of risk factors Tests if probation plus a research-based program reduces recidivism more than probation alone Fourth Thing: Evaluation Design

19. 19 Control Group (N=313) Individual Therapists (N=387) Group Mean (Average) 18-Month Unadjusted Felony Recidivism Percentage 17 12 55 47 42 14 34 18 23 31 28 14 26 17 33 23 14 33 22 17 11 0 C1234567M8910111213M1415161718M19202122232425M Not Competent Borderline Competent Highly Competent 8 18 20 43 47 63 26 FFT Results

20. 20 6% 25% 49% 27% 7% 50% 7% 19% 45% Misdemeanor and Felony Recidivism Violent Felony Recidivism 18-Month Recidivism Percentage Control (N=525) Not Competent (N=203) Competent (N=501) ART Results

21. 21 19% 3% 17% 1% Felony 12-Month Recidivism Percentage Control (N=171) COS (N=171) Adjusted 12-Month Recidivism Misdemeanor and Felony Coordination of Services Results

22. 22 FFT delivered competently Reduces 18-month felony recidivism by 38 percent Benefit to cost ratio of $7.69 ART delivered competently Reduces 18-month felony recidivism by 24 percent Benefit to cost ratio of $12.60 Coordination of Services Reduces 12-month felony recidivism by 57 percent Benefit to cost ratio of $13.58 MST No findings because of problems implementing the Institute’s evaluation design Findings From Initial Studies

23. 23 A Clear Lesson… The key to reducing recidivism with research-based programs is competently delivering the service.

24. 24  Parole  Intensive Parole  Local Parole  Local Commitment  Group Homes ------------------------------------------  Basic Training Camp  Dialectic Behavior Therapy  Mentoring Program  Family Integrated Transitions Additional JRA Programs We Were Asked to Evaluate

25. 25 -$12,478 -$4.95$2,098-$10,379 Regular Parole (v. not having parole) -$5,992 $0.00$5,992$0 Intensive Parole -$1,500 $0.00$1,500$0 Intensive Probation Supervision $5,073 $1.78$6,471$11,544 Mentoring $5,128 $13.58$408$5,536 Interagency Coordination $8,805 $12.60$759$9,564 Aggression Replacement Training $14,315 $7.69$2,140$16,455 Functional Family Therapy $19,247 $3.15$8,968$28,215 Family Integrated Transitions $22,364 n/a-$7,586$14,778 Washington Basic Training Camp $31,243 $38.05$843$32,087 Dialectical Behavior Therapy Benefits Minus Costs Benefits per Dollar of Cost CostsBenefits Measured Benefits and Costs Per Youth Cost Benefit Estimates for Washington State Programs as of September 17, 2004

26. 26 Benefits and Costs of Prevention and Early Intervention Programs for Youth September 2004 Steve Aos et al. Washington State Institute for Public Policy www.wsipp.wa.gov 2003 Legislature Starts to Repeat Process for Other Intervention and Prevention Programs

27. 27 Question: Are there “research-based” programs or policies with a “real world” ability to:Question: Are there “research-based” programs or policies with a “real world” ability to: Reduce crime, Lower substance abuse, Improve educational outcomes, Decrease teen pregnancy, Reduce teen suicides? Lower child abuse or neglect, or Research Questions & Methods The six outcomes The quality of a study’s research design Whether the program is “real world” We screened evaluations written in English for:We screened evaluations written in English for: We computed effects (meta-analytically)We computed effects (meta-analytically) We then calculated monetary benefits and costsWe then calculated monetary benefits and costs

28. 28 Over 30 well-researched studies, mostly of programs for 3 & 4 year olds from low income families. Key findings: improved education outcomes, increased high school graduation higher test scores lower special education lower grade repetition reduced crime, reduced child abuse & neglect. Evidence of decay in early test score outcomes, but still statistically significant by high school graduation. Summary of Benefits and Costs (2003 Dollars) Dollars Per Youth (PV lifecycle) BenefitsCosts B - C Early Childhood Education$17,202$7,301$9,901 Nurse Family Partnership$26,298$9,118$17,180 A home visitation program, with active nationwide dissemination. NFP is delivered by nurses and is for low income, soon-to-be first time mothers. Evidence of: reduced crime for mothers and children, reduced child abuse & neglect, improved education outcomes. Website: www.nursefamilypartnership.org/ Functional Family Therapy$16,455$2,140$14,315 Two programs for juvenile offenders and their families, conducted by trained therapists. FFT has been implemented statewide in Washington State’s juvenile courts. Evidence of reduced crime when the model is followed. Websites: www.fftinc.com/ and www.mstservices.com/ Life Skills Training$746$29$717 A three-year program for middle school youth designed to prevent tobacco, alcohol, and marijuana use. Delivered by classroom teachers. Key findings: Delayed initiation of tobacco, alcohol, illicit drugs. Website: www.lifeskillstraining.com/ Seattle Soc. Dev. Project$14,246$4,590$9,837 A multi-year grade school and middle school training program for parents (family management training) and teachers (classroom management, interactive teaching) for children with low socioeconomic status. Evidence of: reduced crime, increased high school graduation, reduced grade repetition Website: http://depts.washington.edu/sdrg/ Guiding Good Choices$7,605$687$6,918 A multimedia training program (parenting skills, peer pressure refusal skills for students) implemented with families of middle school children. Evidence of: reduced crime, reduced alcohol initiation Website: www.channing-bete.com/positiveyouth/pages/FTC/FTC-GGC.html Multi-D Treat. Foster Care $26,748 $2,459 $24,290 Multidimensional Treatment Foster Care is an alternative to group facilities for youth with chronic & severe criminal behavior. Delivered by trained families, the goal is to return the youth to the family; the primary family also receives therapy. Evidence of: reduced crime Website: www.oslc.org/ Intensive Juv. Supervision $0 $1,482 -$1,482 We meta-analyzed 19 studies of intensive supervision programs for juvenile offenders. No statistically significant effect on recidivism rates. Big Brothers/Sisters (all costs) $4,058 $4,010 $48 HF (taxpayer costs only) $4,058 $1,283 $2,775 Mentoring program. Evidence: increased test scores, delayed alcohol and drug initiation. Website: www.bbbsa.org Selected Findings Multi-Systemic Therapy$14,996$5,681 $9,316

29. 29 Managing Competent Program Delivery Invest in increasing skills, not accountability Pick good people Provide required training Enhance skills Retain only competent providers

30. 30 Written knowledge test and feedback after initial training Follow-up skill training and consultations Certification at end of probationary period Initial Provider Training

31. 31 Form regional support teams On-site direct observation by an expert, including audio- or video-taping to improve competence Case-by-case consultations Give feedback Continually Enhance Provider Skills

32. 32 Please place a check in the box  that best describes what happened in this session. YES▼ NO ▼ 1.Was a positive climate established through welcoming students? 2.Were group norms reviewed, emphasizing positive participation? 3.Were all ACT concepts covered to this point reviewed? 4.Were issues from the last anger control training session reviewed? 5.Did all youth complete the hassle log(s)? Skill Feedback

33. 33 Establish process for corrective action Corrective action taken when needed Building morale; taking pride that we are good! Ensure Competent Providers Are Retained

34. 34 Youth eligible, assigned, and completing programs (75%) Changes in targeted risk and protective factors Program provider competency ratings Meeting expected recidivism rates Annual Legislative Accountability

35. 35 “Let’s get going on one of these programs.” Some of You May Be Thinking… -or- “We already do a great job, so why would we be interested in this?”

36. 36 Understand what is currently being done Understand what is being done in research-based programs Examine the differences Are you already doing it Does it seem like an improvement What needs to be tested Implement quality assurance for current practice Have competent researcher do valid outcome evaluations If You Are Not Comfortable With the Research Approach, a Possibility…

37. 37 Essential Ingredients Infrastructure for program fidelity Funding and de-funding contingent on continued outcomes Leadership Legislation Funding pilot research-based programs De-funding ineffective programs Juvenile court involvement (administrators, supervisors, and staff) Local provider involvement Skilled independent researchers Assessment Research-based programs Valid outcome evaluations Cost-benefit analysis