1. Performance Based Contracting and Quality Assurance: Building Systems to Support Success National Quality Improvement Center on the Privatization of Child Welfare Services 1

2. The Quality Improvement Concept The QIC PCW continues the experiment by the Children’s Bureau to utilize QICs as a method of research and demonstration evidence-based topic selection rigorous evaluation targeted TA broad dissemination This is a knowledge development initiative—the goal is to move the child welfare field forward 2

3. Funded by the Children’s Bureau, the QIC PCW has the following goals: evidence-based and outcomes- focused approach To promote and support an evidence-based and outcomes- focused approach to child welfare system development and organizational improvement. collaborative information-sharing and problem-solving national network To facilitate a collaborative information-sharing and problem-solving national network among subgrantees, the Children’s Bureau’s training and technical assistance network, public child welfare agencies, private service providers, and other stakeholders. build consensus provide input To build consensus on appropriate models of reform, the respective roles and responsibilities of public and private agencies, and to provide input on areas on which the child welfare policy and evaluation fields should focus. 3

4. Triangulation of Data Led to Selection of Topical Focus Area for Sub-grants Initial NAB/CB Discussions Key Informant Discussions with PCW Administrators Discussions with Stakeholder Groups Targeted Forums with Experienced States Literature Review 4

5. Three Demonstration Sites: Florida, Illinois, Missouri Different PBC/QA Interventions Across Sites Case management – FL & MO Residential - IL Public/Private Structure Contract Specifications – Incentive/Penalties Quality Assurance Systems Organizational or System Supports Different Designs Across Sites Multi-county contractors vs comparison - FL 3 Regional private contractors vs public mirror sites vs public agency + random case assignment – MO State-wide private contractors - IL Different Outcomes Across Sites Process & Practice outcomes – FL CFSR outcomes – MO Treatment & Discharge outcomes - IL 5

6. Cross-Site Evaluation Research Questions RQ1: Does an inclusive and comprehensive planning process produce broad scale buy-in to clearly defined PBC/QA? RQ2: What are the necessary components of PBC/QA systems that promote the greatest improvements in outcomes for children and families? RQ3: When operating under a PBC/QA system, are the child, family and system outcomes produced by private contractors better than those under the previous contracting system? RQ4: Are there essential contextual variables that independently appear to promote contract and system performance? RQ5: Once initially implemented, how do program features and contract monitoring systems evolve over time to ensure continued success? 6

7. Improvement Decline Note: Scores represent the standardized difference between Y1 and Y2; 0 represents no change from Y1 to Y2 Relative Change in Performance (Standardized) From Project Year 1 to Project Year 2 7

8. Summary: Outcome Performance Raw data from each site shows that for the majority of outcomes, outcome performance by contractors improved from Y1 to Y2 under PBC Standardized data across sites and outcomes shows that the relative change or difference in performance from Y1 to Y2 was positive (+.0433) Translated: This represents a 4% improvement in overall outcomes 8

9. Common Elements for Success Across Sites PoliticalRight Time and Support for Change LeadershipRight Leaders Driving Change & Staying Involved CollaborationInclusive Planning Process Between Public & Private PlanningSufficient Time to Plan Communication Formalized, Transparent Communication Structure Meaningful Feedback to All Levels PracticeSupport for Practice Change DataHaving and Using Reliable Data QA/QIRestructuring QA/QI Process to Support PBC OutcomesSelecting Right Outcomes and Building a Contract Around Them 9

10. Site-Specific Supports for Achieving Success FLORIDAILLINOISMISSOURI Collaboration Support Neutral Facilitator Supervisory Roundtable Provider Forums & Info Dissemination Issue-Specific Workgroups Program Manager Meetings Issue-Specific Workgroups Outcome Support Supervisory Review Tool Family Finders Discharge & Transition Protocol Child Youth Investment Teams (CAYIT) & Centralized Matching Practice Support Statewide Practice Summits Decision Making Support Child Welfare Advisory Committee (CWAC) CEO Meetings Organizational/System Support University Research Partnerships Data Support Residential Treatment Outcome System (RTOS) Data Test Workgroup Random Case Assignment Quality Assurance Support Detailed Agency & Worker- Specific QA Reports Monitoring Shift to Quality vs Compliance Joint Public/Private QA/QI Alignment 10

11. Lessons Learned Across Sites Process Planned collaboration and communication process structures are critical Performance-based contracting is an evolutionary process that takes time If phasing in, need structured plan for new sites using lessons learned from experienced Use a fidelity checklist for implementation Public/Private Partnerships Put equal emphasis on reform in both the public and private sectors All providers are different entities - they don’t operate the same. May need to be more direct and prescriptive with the private sector Contracts Collaboratively choose right outcomes to match overall system goals Develop a longer term plan than the current contract Marry finance to outcome development at the start Need fluid peer record review across sectors Don’t have dual case management system Be flexible in contracts and allow innovation Data Develop or modify data collection/tracking system that is robust Must have reliable and accurate data to measure outcomes/performance 11

12. Conclusions Formal public/private partnerships via performance-based contracts and aligned quality assurance systems can lead to improved system and child/family outcomes This is a data-driven process that requires a robust data system infrastructure and commitment to using outcome evaluation to monitor performance Additional organizational or system supports can help ensure practice change and outcome attainment. Developing and sustaining collaborative relationship is key to successful planning, implementation, progress, adaptation Preview of Things to Come:  Portrait of Private Agencies in the Child Welfare System: Principal Results from the National Survey of Private Child and Family Serving Agencies  Strategic Planning to Enhance Public/Private Partnerships: Lessons Learned from Five States’ Efforts  Special Issue of The Journal of Public Child Welfare on this work Winter 2012 12