1. Alan Duncan Director, Bankwest Curtin Economics Centre Economic Evaluation of the NSW ‘Keep Them Safe initiative’: A geographic Approach Presentation to the Queensland government Stronger Families Reforms: Evaluation Leaders Forum 15 June 2015 Bankwest Curtin Economics Centre, Curtin University Rebecca Cassells 1

2. Research Objective Methodological Approach  Overall model concept  Child Protection Basefile (CPB)  Challenges in model specification  Spatial attribution of KTS funding Findings  Investment and outcomes  Effectiveness of KTS investment Outline 2

3. Cost-effectiveness of KTS 1.Which areas throughout NSW demonstrate better outcomes for lower investment? 2.To what extent has KTS investment provided better protection for children and young people in NSW? 3.Has KTS PEI funding resulted in better outcomes for children? 4.Are KTS components more cost-effective for some groups (e.g. younger children)? Research Objective 3

4. Keep Them Safe Outcomes Evaluation Economic Evaluation 4

5. Econometric model – separates out effects of other factors likely to influence child wellbeing, tests if KTS investment has been successful and for whom. ROSH = function of (KTS funding; Federal government activity; socio-economic status; remoteness; pre-school attendance; AEDC; economic activity, other) Overall Model Concept 5

6. Geographic database (LGA level) Panel level data – time and space dimensions From 2006/07 – 2012/13 Child protection outcomes and investment Demographic and socio-economic indicators Child development Economic activity Child Protection Basefile (CPB) 6

7. Attribution – is there causality? Additional controls: Remoteness SEIFA Development vulnerability (AEDC) Pre-school attendance Federal government activity related to child wellbeing Male/female employment Economic activity – net business entries Challenges in model specification 7

8. Spill-over effects – Investment and drivers are not confined to a bounded area. Lags – it takes time for impact to be observed. Further challenges in model specification 8

9. Why do we need to allocate funding spatially? What are the issues and challenges? What methods did we apply as a solution? Results – next slide Note limitations – and hence the reason why we introduce additional controls in our econometric approach for spillover effects. Spatial attribution of KTS funding 9

10. 1.Comparable methods to that used in the Interim review undertook by Access economics, expanded to fully capture impact at investment destination. 2.Identify group of children targeted, e.g. all children, school-aged children, Aboriginal children or children in OOHC. 3.Use population level data about target group to distribute investment proportionate to intention to treat group. 4.Adapt distribution to intelligence about specific program level funding. Spatial attribution of KTS funding – further detail 10

11. Spatial attribution of KTS funding 11

12. Economic Evaluation Keep Them Safe Outcomes Evaluation 12

13. How to visualise relationship between KTS investment and child outcomes? Rank LGAs into the lowest to highest quintiles according to two dimensions: i.improvements in ROSH outcomes per 1,000 children 0-17 ii.changes in per capita KTS funding (other than OOHC) Distinguish between LGAs that have - relatively better ROSH outcomes and lower per capita KTS funding (blues) relatively better ROSH outcomes and higher per capita KTS funding (greens) relatively poor ROSH outcomes and lower per capita KTS funding (oranges) relatively poor ROSH outcomes and higher per capita KTS funding (reds) Comparing relative KTS investment with child outcomes 13

14. Comparing relative KTS investment with child outcomes 14

15. Two important conclusions can be drawn from the combined spatial examination of KTS funding and children reported at ROSH: i.there exists a good degree of variation in relative per capita KTS funding among LGAs that have similar risk profiles; and ii.a given level of per capita KTS funding is associated with a broad range of changes in ROSH reports between 2010/11 and 2012/13. Together with earlier arguments, these findings support the capacity for our modelling approach to identify the effectiveness of KTS funding on child outcomes Comparing relative KTS investment with child outcomes 15

16. In measuring the cost effectiveness of KTS funding, two child outcome indicators were examined: i.changes in the number of children reported to be at ROSH; and ii.changes in the number of children who are re-reported at ROSH having been reported in the previous year. Child outcomes were related both to overall KTS funding (other than OOHC) and to prevention and early intervention (PEI) funding only Effectiveness of KTS funding in promoting better child outcomes 16

17. Robustness of KTS cost effectiveness evaluation results A number of model variants were examined to establish robustness in the econometric evaluation of KTS cost effectiveness: i.Introducing controls for local child development indicators ii.Introducing lagged KTS funding measures iii.Introducing spill-over effects in the effectiveness of KTS funding between contiguous/adjacent LGAs 17

18. Effectiveness of overall KTS funding 18

19. Effectiveness of KTS PEI funding 19

20. Effectiveness of KTS funding on ROSH re-reports 20

21. Economic Evaluation Keep Them Safe Outcomes Evaluation 21

22. Impact of KTS investment on providing better protection for children and young people in NSW. Comprehensive data base was constructed to test this, using sophisticated econometric models. We find that:  Variation exists in outcomes and KTS investment across NSW LGAs and over time.  Some areas throughout NSW show worse child outcomes despite high per capita funding (Bourke, Brewarrina and Coonamble), while others show improvements with lower per capita funding (Wollondilly, Woollahra and Hawkesbury).  Strong ROSH improvements were observed in mid-west NSW (Lachlan, Central Darling and Carrathool) with relatively high levels of KTS investment per capita.  Similarly, areas with relatively low investment (Marrickvillle, Young and Urana) have seen increases in children reported at ROSH. Summary and Conclusions 22

23. Econometric analysis:  KTS funding is shown to have a direct link with improving child wellbeing outcomes, as proxied through ROSH reports and re-reports.  Funding directed towards PEI makes a significant contribution towards reducing rate of children at ROSH and re-reports of ROSH.  Model variants demonstrate robust findings.  NSW CPB and model development provides a unique opportunity for ongoing monitoring and assessment of regional trends. Summary and Conclusions 23

24. Alan Duncan Director, Bankwest Curtin Economics Centre Keep Them Safe Outcomes Evaluation Economic Evaluation http://www.keepthemsafe.nsw.gov.au/ kts_evaluation/outcomes_evaluation 24