1. Reforming California’s Juvenile Corrections System: An Evidence- Based Change Strategy* James M. Byrne, Ph.D. Professor Department of Criminal Justice University of Massachusetts, Lowell *Plenary presentation at the Transfer of Knowledge Workshop, “Parole Re-Entry: A New Vision”, sponsored by The Division of Juvenile Justice Parole Operations, Nov. 1,2005 Pomona, California

2. What is Evidence-based Practice? It is the development and implementation of programs based on a systematic review of “what works” It is the development and implementation of programs based on a systematic review of “what works” There are three basic approaches to Evidence-based practice: There are three basic approaches to Evidence-based practice: Strategy 1: Conduct a comprehensive review of all available research on a particular topic: Strategy 1: Conduct a comprehensive review of all available research on a particular topic: Examples: Examples: The systematic reviews conducted by the Campbell Collaboration Crime and Justice Group (Sherman et. al, 2005; Sherman, et. al, 1997) The systematic reviews conducted by the Campbell Collaboration Crime and Justice Group (Sherman et. al, 2005; Sherman, et. al, 1997) The systematic reviews using meta-analytic methods including experimental and quasi-experimental research (Gendreau, et. al, 1990) The systematic reviews using meta-analytic methods including experimental and quasi-experimental research (Gendreau, et. al, 1990)

3. What is Evidence-based Practice? (continued) Strategy 2: Examine only a subset of all available research studies, using randomized field experiments as the “Gold Standard”, e.g. Farrington and Welsh’s recent review of all randomized experiments (2005) Strategy 2: Examine only a subset of all available research studies, using randomized field experiments as the “Gold Standard”, e.g. Farrington and Welsh’s recent review of all randomized experiments (2005) Strategy 3: Conduct a nonscientific review, simply say “evidence based”, and then offer your own listing of best practices. Strategy 3: Conduct a nonscientific review, simply say “evidence based”, and then offer your own listing of best practices. Reexamine/reposition scientific reviews Reexamine/reposition scientific reviews Only include a subset of all available research, often supporting either liberal or conservative ideology (Farabee, 2005; Cullen, 2002) Only include a subset of all available research, often supporting either liberal or conservative ideology (Farabee, 2005; Cullen, 2002) No specific identification of review procedures, inclusion/exclusion criteria, etc. No specific identification of review procedures, inclusion/exclusion criteria, etc.

4. What is Meta-Analysis? Meta-Analysis is a statistical technique designed to synthesize empirical relationships across a large number of studies. Meta-Analysis is a statistical technique designed to synthesize empirical relationships across a large number of studies. Meta-Analysis allows us to determine both the Size and Direction of effects across studies. Meta-Analysis allows us to determine both the Size and Direction of effects across studies. Meta-Analysis techniques can be translated into summary statistics that public policy makers can easily understand (e.g. a 10% recidivism reduction effect for all correctional interventions using multi-systemic strategies) Meta-Analysis techniques can be translated into summary statistics that public policy makers can easily understand (e.g. a 10% recidivism reduction effect for all correctional interventions using multi-systemic strategies) Sources: Wilson (2001); Rossi, Freeman, and Lipsey (2001)

5. When Should Meta-Analysis be Used? Meta-Analysis should only be used to summarize research findings from methodologically rigorous evaluation designs. Meta-Analysis should only be used to summarize research findings from methodologically rigorous evaluation designs. Meta-Analysis should not be used when there are only a small number of studies (10 or less) being included in the review. Meta-Analysis should not be used when there are only a small number of studies (10 or less) being included in the review. Meta-Analysis is most appropriate for studies examining the relationship between clearly defined independent and dependent variables. Meta-Analysis is most appropriate for studies examining the relationship between clearly defined independent and dependent variables.

6. What Review Criteria are Used in The Campbell Collaborative’s Systematic Reviews?

7. Study Inclusion Criteria For Systematic Reviews The scientific methods scale ranks evaluation studies from 1=weakest to 5=strongest on overall internal validity: What Works: For a program to be classified as working, there must be a minimum of two level 3 studies with significance tests showing effectiveness and the preponderance of evidence in the same direction. What Works: For a program to be classified as working, there must be a minimum of two level 3 studies with significance tests showing effectiveness and the preponderance of evidence in the same direction. What Does Not Work: For a classification of not working, there must be a minimum of two level 3 studies with significance tests showing ineffectiveness and the preponderance of evidence in the same direction. What Does Not Work: For a classification of not working, there must be a minimum of two level 3 studies with significance tests showing ineffectiveness and the preponderance of evidence in the same direction. What is Promising: For the classification of promising, at least one level 3 study is required with significance tests showing effectiveness and preponderance of evidence in support of the same conclusion. What is Promising: For the classification of promising, at least one level 3 study is required with significance tests showing effectiveness and preponderance of evidence in support of the same conclusion. What is Unknown: Any program not classified in one of the three above categories is considered to have unknown effects. What is Unknown: Any program not classified in one of the three above categories is considered to have unknown effects. Source: Welsh and Farrington, (2003: 169-170)

8. Criteria for Assessing The Quality of Evaluation Studies: Level 5: Well-designed randomized field experiment Level 5: Well-designed randomized field experiment Level 4: Randomized field experiment of moderate quality; well-designed quasi-experiment Level 4: Randomized field experiment of moderate quality; well-designed quasi-experiment Level 3: Quasi-experimental designs (moderate quality) Level 3: Quasi-experimental designs (moderate quality) Level 2: Quasi-experimental designs (minimal quality) Level 2: Quasi-experimental designs (minimal quality) Level 1: Non-experimental designs Level 1: Non-experimental designs

9. How do Systematic Reviews (and Meta-Analyses) help us answer the following public policy question: Should current sentencing and corrections policies – both institution-based and community-based – focus on offender rehabilitation or offender control? Should current sentencing and corrections policies – both institution-based and community-based – focus on offender rehabilitation or offender control?

10. The evidence in favor of rehabilitation is found in systematic reviews of correction research that estimate that the provision of treatment (in sufficient dosages and duration) is cost-effective and results in modest offender change (10% reduction) The evidence in favor of rehabilitation is found in systematic reviews of correction research that estimate that the provision of treatment (in sufficient dosages and duration) is cost-effective and results in modest offender change (10% reduction) The evidence opposed to rehabilitation is found in these same systematic reviews, which reveal that the vast majority of individual research studies do not find statistically significant differences between experimental and control groups in recidivism. The evidence opposed to rehabilitation is found in these same systematic reviews, which reveal that the vast majority of individual research studies do not find statistically significant differences between experimental and control groups in recidivism.

12. Current evidence-based reviews highlight the limitations inherent in offender-based change strategies: Only incremental, short-term changes in offender behavior should be expected from the full implementation of evidence-based practices in adult and juvenile corrections Only incremental, short-term changes in offender behavior should be expected from the full implementation of evidence-based practices in adult and juvenile corrections If we are interested in long-term offender change, we need to focus our attention on the community context of offender behavior If we are interested in long-term offender change, we need to focus our attention on the community context of offender behavior There is a growing body of research on the need to integrate individual and community-level change strategies (Sampson, et. al. 2005; Bursik, 2005; Carr, 2003) There is a growing body of research on the need to integrate individual and community-level change strategies (Sampson, et. al. 2005; Bursik, 2005; Carr, 2003)

13. The Need for Reform In California’s Juvenile Corrections System: The Youth Incarceration Rate in California is among the highest in the country The Youth Incarceration Rate in California is among the highest in the country Institutional costs represent a significant proportion (75%) of all juvenile corrections expenditures (400 million in FY ’04) Institutional costs represent a significant proportion (75%) of all juvenile corrections expenditures (400 million in FY ’04) Institutionalization is having a negative effect on both juvenile offenders and on the communities in which juvenile offenders reside Institutionalization is having a negative effect on both juvenile offenders and on the communities in which juvenile offenders reside Recycling/Churning: Recycling/Churning: Juvenile offenders released from CYA facilities are being rearrested and returned to prison/CYA at unacceptably high rates Juvenile offenders released from CYA facilities are being rearrested and returned to prison/CYA at unacceptably high rates Control-based community supervision strategies have failed to produce the desired recidivism reduction effects, either short- term or long-term Control-based community supervision strategies have failed to produce the desired recidivism reduction effects, either short- term or long-term

14. Source: Nadel-Hayes and Macallair, 2005: 32

15. Three Basic Organizational Change Strategies: Empirical Rational Strategies: based on the notion that information/education can lead to behavior change Empirical Rational Strategies: based on the notion that information/education can lead to behavior change Normative Reeducation Strategies: based on the notion that attitudes/values must change before behavior will change Normative Reeducation Strategies: based on the notion that attitudes/values must change before behavior will change Power Coercive Strategies: based on the notion that formal, coercive, sanction-based strategies can induce behavior change Power Coercive Strategies: based on the notion that formal, coercive, sanction-based strategies can induce behavior change

16. Common Characteristics of Successful Normative Reeducative Change Strategies: At The Individual Level: At The Individual Level: Cognitive Restructuring Cognitive Restructuring Emphasize Positive Lifestyle Change (Employment, Marriage, Relocation) Emphasize Positive Lifestyle Change (Employment, Marriage, Relocation) Focus on Informal Social Controls Focus on Informal Social Controls At The Organizational Level: At The Organizational Level: Leadership, Partnership, and Ownership Leadership, Partnership, and Ownership Evidence-Based Strategic Plan Evidence-Based Strategic Plan Focus on Culture Change (Moral Performance) Focus on Culture Change (Moral Performance) New Ways of Doing Business: Performance Measurement New Ways of Doing Business: Performance Measurement

17. The Challenges for Community Corrections Managers Involved In Large-Scale Organizational Change 1. Design a reform strategy that is defensible – based on “best practices”/evidence-based reviews 2. Implement a reform strategy that is compatible with the values, skills, and capacity of line staff (and Mgt.) 3. Evaluate a reform strategy from the outset, based on the premise that “what gets measured gets done”

18. Resistance to Change in Community Corrections: A Typology Conformists – Accept organization’s goals and follow standard policies and procedures Conformists – Accept organization’s goals and follow standard policies and procedures Ritualists – Reject organization’s goals but still follow policies and procedures, often at minimal compliance level Ritualists – Reject organization’s goals but still follow policies and procedures, often at minimal compliance level Artful Dodgers – May or may not accept organization’s goals but do not follow existing policies and procedures (catch me if you can) Artful Dodgers – May or may not accept organization’s goals but do not follow existing policies and procedures (catch me if you can) Loose Cannons – May or may not accept organization’s goals but often directly challenge leadership on policies and procedures (you can not touch me) Loose Cannons – May or may not accept organization’s goals but often directly challenge leadership on policies and procedures (you can not touch me)

19. A Research Agenda for California’s Division of Juvenile Parole Operations 1. Survey line staff on attitudes – Focus on organizational change initiatives/culture/resistance to change 2. Conduct workload study based on newly defined roles and responsibilities for line staff and management 3. Examine the reliability and validity of classification system (e.g. inter-rater reliability; prediction accuracy) 4. Monitor compliance with new policies and procedures using a clearly articulated performance measurement system 5. Evaluate both the implementation and impact of specific strategies (and programs) using level 3 (and higher) research designs (i.e. experiments and quasi- experiments)

20. NIC’s Eight Principles for The Utilization of Evidence-Based Practices in Community Corrections: 1. Assess Actuarial Risk/Needs 2. Enhance Intrinsic Motivation 3. Target Interventions a. Risk Principle: Prioritize supervision and treatment resources for higher risk offenders b. Need Principle: Target interventions to criminogenic needs c. Responsivity Principle: Be responsive to temperament, learning style, motivation, culture, and gender when assigning programs. d. Dosage: Structure 40-70% of high-risk offenders’ time for 3-9 months e. Treatment: Integrate treatment into the full sentence/sanction requirements 4. Skill Train with Directed Practice (use Cognitive Behavioral treatment methods) 5. Increase Positive Reinforcement 6. Engage Ongoing Support in Natural Communities 7. Measure Relevant Processes/Practices 8. Provide Measurement Feedback

21. In-Prison Meeting with Community Meeting with Police/Community Reintegration Team Housing Community Job Family Victim Outreach Sign Behavioral Contract Reentry Partnership Continuum Classification for Risk, Treatment, and Control Move Target Population to Select Facility Treatment in Prison Change Education Job Family Drug/ Alcohol Treatment In-Community Change Education Job Family Drug/ Alcohol Surveillance ● Services ● Outreach Structured Reentry Institution 6 months Community/ Reintegration

22. The moral performance of institutional and community corrections needs to be examined along with cost, and offender change. Redefining The Criteria for A Successful Juvenile Corrections System

23. Defining The Moral Performance of Prisons: Relationships: Relationships: Respect Respect Humanity Humanity Relationships Relationships Trust Trust Support Support Social Structure: Social Structure: Power/authority Power/authority Social relations Social relations Regime: Regime: Fairness Fairness Order Order Safety Safety Well-being Well-being Personal Development Personal Development Family Contact Family Contact Decency Decency Other: Other: Meaning Meaning Quality of life Quality of life Source Adapted From: Allison Liebling (2005)

24. Cullen’s “Liberal but not stupid” approach to crime control (1) This approach is informed by a general belief that people’s choices are bounded by social constraints, including constraints that are rooted in social inequalities—with many such inequalities tied to economic processes that may produce wealth but also have untoward, if not unfair, consequences. (1) This approach is informed by a general belief that people’s choices are bounded by social constraints, including constraints that are rooted in social inequalities—with many such inequalities tied to economic processes that may produce wealth but also have untoward, if not unfair, consequences. (2) Knowing that harsh living conditions are implicated in crime, it is irresponsible not to pursue social welfare policies that ameliorate these disadvantages. (2) Knowing that harsh living conditions are implicated in crime, it is irresponsible not to pursue social welfare policies that ameliorate these disadvantages. (3) Within the criminal justice system—a system that will affect millions of lives annually—it is irresponsible not to pursue policies that improve the plight of offenders, victims, and potential future victims. (3) Within the criminal justice system—a system that will affect millions of lives annually—it is irresponsible not to pursue policies that improve the plight of offenders, victims, and potential future victims.

25. Cullen’s “Liberal but not stupid” approach to crime control (continued) (4) For those with progressive sentiments, the key issue is not nonintervention but rather the quality of the intervention that takes place in the criminal justice system. (4) For those with progressive sentiments, the key issue is not nonintervention but rather the quality of the intervention that takes place in the criminal justice system. (5) Beyond the values that inform policy choices, criminal justice interventions should not be based on prejudice, custom, or fashion but on “what works”—on what the data show are the most effective policies to pursue. (5) Beyond the values that inform policy choices, criminal justice interventions should not be based on prejudice, custom, or fashion but on “what works”—on what the data show are the most effective policies to pursue. (6) Liberal welfare approaches to crime control that are revealed to be ineffective should not be defended but relinquished. (6) Liberal welfare approaches to crime control that are revealed to be ineffective should not be defended but relinquished.

26. Concluding Comments The focus of the next wave of adult and juvenile corrections programs should be on addressing the problems of both high- risk offenders and high-risk communities The focus of the next wave of adult and juvenile corrections programs should be on addressing the problems of both high- risk offenders and high-risk communities At minimum, correctional interventions should be designed to do no harm, either to offenders or communities At minimum, correctional interventions should be designed to do no harm, either to offenders or communities It is not just adult and juvenile offenders that need to change; It is not just adult and juvenile offenders that need to change; It is the moral performance of corrections – both institutions and community corrections – that needs to change as well. It is the moral performance of corrections – both institutions and community corrections – that needs to change as well.