California Child Welfare Outcomes and Accountability Legislation: Evolving Toward System Improvement with Longitudinal Data & Analysis Panel on Increasing Impact and Redistributional Efficiency in Public Child Welfare Daniel Webster, PhD Barbara Needell, PhD Terry Shaw, PhD Center for Social Services Research University of California, Berkeley Association for Public Policy Analysis & Management 28 th Annual Research Conference Madison, WI November 2, 2006 Slides in this presentation originally created by Barbara Needell, Lynn Usher, and Emily Putnam-Hornstein The Performance Indicators Project at CSSR is supported by the California Department of Social Services and the Stuart Foundation.

Presentation Outline Background on Assembly Bill 636 A Review of Progress –Relative Change on 20 Outcome Indicators –Multivariate Models on Permanency The Future of AB636

Entry Cohorts Exit Cohorts Point in Time Data 3 Key Approaches to Data

Caseload Snapshots Versus Entry Cohorts Jan. 1, 2005 Jan. 1, 2006Jan. 1, 2004

The Cycle of Experiences in the Child Welfare System The Cycle of Experiences in the Child Welfare System Counterbalanced Indicators of SystemPerformance PermanencyThroughReunification, Adoption, or Guardianship ShorterLengths Of Stay Stability Of Care Rate of Referrals/ Substantiated Referrals Home-Based Services vs. Out-of-HomeCare Maintain Positive Attachments To Family, Friends, and Neighbors Use of Least Restrictive Form of Care Source: Usher, C.L., Wildfire, J.B., Gogan, H.C. & Brown, E.L. (2002). Measuring Outcomes in Child Welfare. Chapel Hill: Jordan Institute for Families, Reentry to Care

Assembly Bill 636 Child Welfare Outcomes and Accountability System Legislation passed: October 2001 Baseline Quarterly Report: January 2004 Provisions of the Act –County self-assessments & self-improvement plans –Quarterly data reports to state and county officials –Public-private collaboration to support reform efforts –Longitudinal data publicly available

The Double-Edged Nature of Public Data PROS: –Greater performance accountability –Community awareness and involvement, encourages public-private partnerships –Ability to track improvement over time, identify areas where programmatic adjustments are needed - County/County and County/State collaboration CONS: –Potential for misuse, misinterpretation, and misrepresentation –Available to those with agendas or looking to create a sensational headline –Misunderstood data can lead to the wrong policy decisions

California: AB636 Federal Measures, Percent IMPROVEMENT from Baseline to Most Recent Report Period Permanency Safety Note: (+) indicates a measure where a % increase equals improvement. (-) indicates a measure where a % decrease equals improvement.

California: AB636 State-Enhanced Measures, Percent IMPROVEMENT from Baseline to Most Recent Report Period Permanency Safety & Participation Note: (+) indicates a measure where a % increase equals improvement. (-) indicates a measure where a % decrease equals improvement. indicates a measure where performance declined. Well-Being

Study Limitations & Next Steps Shortcomings of Administrative Data Study Time Period Examine All Performance Indicators Analyze Interplay Between Outcomes Further Specification for Multivariate Models

Whither AB636 ? Early Indication of Positive Change –Results of present study –Attitude shift in public child welfare staff A New Landscape with CFSR Round 2 –Composites, components, & measures (oh my!) –Rethink and re-tool quarterly report –Educate child welfare agency staff Turning Data into Knowledge –Familiarity/Use of data throughout agency –Making the link from outcome to practice * Developing human capital to use data