National Outcome Measures: Using Data to Show the Way Forward Second Canadian Roundtable on Child Welfare Outcomes Fred Wulczyn, Ph.D.

slide 2 Objectives  You have the data - now what?  Methods/analysis - getting a clearer picture  Risk or case mix adjustment  Better methods/better questions  Analysis - using the data to manage (reduce and induce) variation  Linking outcomes and finance  Admission, duration, and unit cost strategies

slide 3 BD: Before there was data Fact or Fiction - how much did it matter?

slide 4 AD: After we have data

slide 5 Develop a theory of change (Tell a story but no more fiction)  Look for differences  This happens more often for this group  This isn’t as common in this part of the province  Develop an intervention or interventions  If we do this this will happen more often  Asking good questions is important

slide 6 What is this notion of risk or case mix adjustment?  Why does performance vary?  Provinces  Providers  Children and families  Why do we care?  Promotes better theories of etiology and intervention  Targeting may improve effectiveness

slide 7 Incidence per 1000: The Likelihood of Placement

slide 8 Variation in Incidence Place and Age

slide 9 Placement Stability - Creating a better measure  Two views of stability  Who is at risk?  When is the risk highest?  Moves per child addresses the first  Moves per day does a better job with the first but not the second.

slide 10 Ask ‘Richer’ Questions: 3 view of stability Percent of children Moves per 100 days Moves per 100 days by time in care who moved at least once

slide 11 Monitoring  Looking for whether the changes you made induced the change you were looking for  Depends on a baseline

slide 12 Time to Permanency Median Duration

slide 13 Digging Deeper How important is it?  A hypothetical: Two counties each working to improve permanency. The counties have selected different strategies. After a period of time, the public agency pulls everyone together. The question: with the resources we have, which intervention represents the better bet?  Vital statistics Children served: 350 in each  Success

slide 14 Conclusion:  Invest resources in the County B intervention because the permanency rate is higher.  Children will go home sooner and tax dollars will be spent more wisely Everybody is a winner.

slide 15 What a minute: Are you sure?  What gives? Permanency rates in County A are actually higher. Why the difference? County B serves more Type A children. Type A children have higher overall permanency rates than Type B children; thus the aggregate data show a different result.  New Conclusion County A’s program is probably the better bet

slide 16 Return on Investment - Shifting the dialogue  Spending vs. investment  ‘Purchase’ more of what works, less of what does not  Underscores the importance of outcomes  Know what and why you do what you do  Underscores the importance of data for looking back in time and for setting goals and expectations.

slide 17 Basic Fiscal Model Revenue = Units * Unit Cost

slide 18 Link Outcomes and Funding

slide 19 Take Away  Gathering data is only the start  Collecting data begs the question: “What do you mean?”  Data dramatically improves the possibility of success  Data is central to the rights of children and families