1. Linking administrative data sets for self- evaluation: Preliminary results from the Annie E. Casey Foundation Family to Family Initiative in California Anne K. Abramson-Madden & William C. Dawson Center for Social Services Research University of California Berkeley

2. Linking administrative data sets for self-evaluation Mandatory outcome reporting with Adoption and Safe Families Act of 1997 (ASFA) Statewide reviews of selected indicators as part of Child and Family Services Reviews In California, the California Child Welfare and System Improvement Accountability Act (AB636) requires quarterly county reports

3. The Family to Family Initiative’s Four Core Strategies Recruitment, Training and Support of Foster and Kinship Families Building community partnerships Team Decision Making Self-evaluation

4. Team Decision Making (TDM) Meetings held to make placement decisions. Meetings are led by trained facilitators who are not the case-carrying social workers. Decision is reached by consensus with a safety plan in place. If consensus cannot be reached, agency is ultimately responsible for the decision. Family decides who makes up the team and may reject members. May also have community members & child’s caregivers on the team. Meetings generally last one to two hours.

5. CA F2F Implementation 24 of 58 California counties Approximately 88% of the 85,286 children in child welfare supervised foster care live in a Family to Family county

6. CA F2F Implementation (Cont’d) ClusterNumber of Counties Implementation Year(s) I72000-2003 II42001-2003 III52004-2005 IV72004-2005 LA1 (3 clusters by SPA) Various

7. Self-Evaluation in California Self-Evaluation is part of the Annie E. Casey Foundation’s Family to Family Initiative, also funded in CA by –Stuart Foundation –Walter S. Johnson Foundation –California Department of Social Services

8. Family to Family Self-Evaluation in California Integration of data with practice: –Web reports using state administrative data provide information about child welfare outcomes –TDM database Self-evaluation and quarterly reports Linkage to state administrative data has potential to examine implementation progress and child welfare outcomes

9. CA CWS/CMS Child Welfare Services Case Management System: statewide computer system SB 370 (Chapter 1294, Statutes of 1989) Case management information for Child Welfare Services workers

10. CA CWS/CMS (Cont’d) Common database for California’s 58 counties Allows state and county administrators to monitor progress Consolidates information for state and federal reporting requirements

11. Administrative Data Source: UCB_FC at CSSR Longitudinal file containing foster care placement histories from 1998 to present Constructed from California's version of the federal Statewide Automated Child Welfare Information System (SACWIS)

12. Data Access and Analysis California Department of Social Services (CDSS) and the Stuart Foundation provide support for the California Performance Indicators Project (web reports) Analysts at CSSR produce a range of measures for use by California counties and the public: http://cssr.berkeley.edu/cwscmsreports/

13. CWS/CMS reports

14. Self-Evaluation using TDM CA Customized Microsoft Access database Counties collect TDM meeting and child information Create reports regarding attendance, meeting participants, involved children, etc Counties produce quarterly report for self- evaluation

15. TDM CA Export Form

16. TDM CA to UCB_FC linkage Linkage allows counties to keep TDM data collection to a minimum, only focusing on items unique to TDM process while still capturing needed information about outcomes

17. Characteristics of Sample Counties County 2004 Child Population (0-18) July 1, 2004 Child Welfare Caseload (0-18) County 1100,000-250,000<1,000 County 2100,000-250,000>1,000 County 3100,000-250,000<1,000 County 4<100,000<1,000 County 5>250,000>1,000

18. Preliminary Findings Five California Family to Family counties TDM database: information on all children for whom placement recommendations were discussed in a TDM meeting UCB_FC contains information on all child welfare-supervised out-of-home placements TDM meetings and child welfare events (placement moves) restricted to Quarter 1, 2005 (January 1, 2005-March 31, 2005)

19. Preliminary Findings (Cont’d) Data only as good as we get from counties- there may be errors (especially with respect to reason for involvement and recommendations)

20. Implementation Analysis 1.Start with a qualified event (entry, placement move, or exit). 2.What was the closest preceding event: another child welfare event or a TDM meeting? 3.If a meeting, was it a related meeting? Count number of associated meetings. 4.Count remaining meetings without associated child welfare events.

21. Recommendation Analysis 1.Group children by reason for involvement and recommendation type. 2.Was there a related move during timeframe? 3.What was the actual move during the timeframe? 4.If both #2 and #3 match the recommendation, then the recommendation is achieved. 5.If recommendation achieved, then we look to the time to achievement.

22. Entry to Care TDM Entry Meeting Qualified Moves TDM Plc Move Meeting TDM Data Start TDM Plc Move Meeting TDM Entry Meeting ? TDM Data Cutoff (Current Quarter) 1 2 3 Time Placement Move 4 2 1 3 6 5 1 Placement Moves: 6 Associated Meetings: 3 Events with Associated Meeting: 50% Meetings without Events: 1 Implementation Methods

23. Implementation Summary Numbers Section 1: Entered Placement Placement Move Exit from Placement 1) County CW Events 6561110479 2) Associated TDM Meet- Child Events 1381376 3) % CW Events with Assoc. TDM Meet- Child Event 21.04%12.34%1.25% 4) TDM Meet- Child Event with No Assoc. CW Event 12417127

24. Meeting Count Example: County 4 Primary Reason For Meeting Number of Meetings Number of Unique Children Max Number of Meetings Per Child Mean Number of Meetings Per Child Imminent risk of placement 121521.07 Emergency placement 131711.00 Placement move 253021.07 Exit from placement 4711.00 ALL 546721.07

25. Event Count Example: County 5 Primary Reason For Meeting Number of Meeting s Number of Meeting/Chil d Events Max Number of Children Per Meeting Mean Number of Children Per Meeting Emergenc y placement 427461.76 Placement move 9711131.14 Exit from placement 2211.00 ALL 14118761.33

26. Placement Move Meeting Attendance Attendee Type Number of Meetings Number of Attendees Mean Attendees Per Meeting Number of Meetings Attended By At Least One Percent of Meetings Attended By At Least One Facilitators2682891.08268100.0% Supervisors2681760.6616963.1% FR/PP Workers2681700.6315758.6% Children2681480.5512948.1% FFA Social Workers2681490.569435.1% Birth Parents2681050.398832.8% Relatives2681760.668130.2%

27. Placement Move Meeting Attendance Attendee Type Number of Meetings Number of Attendees Mean Attendees Per Meeting Number of Meetings Attended By At Least One Percentage of Meetings Attended By At Least One Mental Health Staff268890.337427.6% Other DSS Staff268760.287026.1% FFA Foster Parents268840.316624.6% Other Relative Caregivers268880.336323.5% Other Service Providers268760.284817.9% Interested Individuals268540.203613.4% County Foster Parents268490.183513.1% Family Maintenance Workers268390.153211.9%

28. Placement Move Meeting Attendance (cont’d) Attendee Label Number of Meetings Number of Attendees Mean Attendees Per Meeting Number of Meetings Attended by at Least One Percent of Meetings Attended by at Least One Adoptions Workers268310.122810.4% CASA Advocates268310.122710.1% Community Representatives268300.112710.1% Other Social Workers268250.09217.8% Other2681280.48 ALL26820137.51

29. Recommendations Analysis Analysis restricted to Placement Move as the Child’s Reason for Involvement Five counties: 301 recommendations Possible recommendations include: –Change to less restrictive placement –Maintain in present placement –Change to same level placement –Change to higher level placement

30. Placement Move Recommendation: Change to Less Restrictive Placement TDM Recommendation Related Move? N% Rec Achieved? N% Change to less restrictive placement No1852.9% Not achieved 1852.9% Yes1647.1%Achieved1029.4% Not achieved 617.6% Subtotal (less restrictive) 34100% 34 100%

31. Placement Move Recommendation: Change to Less Restrictive Placement Rec Achieved? Time to AchievementNPercent Percent of Total Not achievedN/A1852.9%9.4% AchievedOne week or less823.5%4.2% One to two weeks25.9%1.1% Not achievedN/A617.6%3.1% 34100%17.8%

32. Placement Move Recommendation: Maintain in Present Placement TDM Recommendation Related Move? N Percent Rec Achieved? N Percent Maintain child in present placementNo8779.1%Achieved7366.4% Not achieved1412.7% Yes2320.9%Not achieved2320.9% Total (maintain in present) 110100% 110100%

33. Placement Move Recommendation: Maintain in Present Placement Recommendation Achieved? Time to MoveN Percent of Total AchievedN/A7366.4% Not achievedN/A1412.7% Not achievedOne week or less32.7% One to two weeks32.7% More than two weeks1715.5% 110100.0%

34. Limitations Data –TDM data entry errors –Missing data Analysis –Logic errors –Paper to Practice errors

35. Implications for Research Linking small database to California’s full child welfare system has huge potential Longitudinal nature of database has wealth of information about children’s lives and child welfare histories Ability to evaluate practice quarterly

36. Implications for Policy TDM reports can influence county boards and state policy makers, leading to change in child welfare services allocations Integrating practice and evaluation may serve as a model for future initiatives

37. Implications for Practice Access to data provides a feedback loop Agency workers (TDMS facilitators, analysts, and management) can easily see data regarding the TDMs to inform practice TDM reports provide a nice way to communicate with community partners, county boards, and other agencies involved

38. Next Steps Continue refining methods for linkage and expanding analysis Analyze entry and exit meetings Consult with counties regarding linkage results Work with counties to improve data accuracy

39. For more information: Anne K. Abramson-Madden William C. Dawson TDMSupport@lists.berkeley.edu Child Welfare Services (CWS/CMS) Reports http://cssr.berkeley.edu/cwscmsreports/ TDM CA Support Page http://cssr.berkeley.edu/tdm/