1. Numbers are people too Using data for decision making in a community services setting Ben Smith, Robert Johnston, Owen Griffiths

2. 2 Contents 1.NSW Family and Community Services – What we do: purpose and interventions 2.Evolution of data for decision making in Family and Community Services 3.Client population forecasting 4.Statistical analysis and hypothesis testing 5.Evaluation, statistical modelling and economic analysis 6.Demonstrating the success of our programs: economic analysis 7.Development and analysis of Key Performance Indicators 8.Demand modelling for NGO transition planning

3. NSW Family and Community Services purpose and interventions Assessed Level of Intervention A. Safe or Safe with Plan and Low or Moderate Risk B. Safe or Safe with Plan and High or Very High Risk C. Unsafe at any Risk level Intervention pathways Universal Services. Child Youth and Family Support Services Brighter Futures Strengthening families FPIS/IFP/IFS/ IFBS* Out-of-Home Care (OOHC)/ Restoration Statutory intervention *FPIS = Family Preservation/Intensive Support Model, IFP = Intensive Family Preservation, IFS = Intensive Family Services, IFBS = Aboriginal Intensive Family Based Services Statutory intervention (e.g. court/case planning) Early intervention and preservation Out-of-Home Care Our purpose is to protect children and young people most at risk of abuse and neglect and its effect

4. Evidence based decision making through timely information from our Business Intelligence System The Corporate Information Warehouse (CIW) is a data warehouse CIW went live in 2005 Holds data from a number of sources and integrates information Provides online access to information from desktops around NSW Information is updated daily Produces Management and Compliance reports Produces reports for Annual, Quarterly and National reporting

5. Capability provided beyond standard counts and measures (simulations, forecasting, etc) Incorporation of external data sources to aid trend analysis and predictive modeling Focused on corporate reporting Replaces manual or ineffective reporting Proves the CIW concept Core Delivery Organisation-wide capability Potential for CIW to enable savings and improve efficiency across the organisation Recognises issues exist across business functions Leveraging the core information to provide value added insight Discussions surrounding data integration and common needs begin to appear Subject matter experts define priorities Value Delivered Insight-driven action Growth Over time, the CIW expanded in scope and capability to deliver greater value. Time Today We are Here

6. Director Child and Family 15 positions Regional Director 7 positions Manager Client Services 76 positions Director Practice Standards 7 positions Regional Business Managers 7 positions Regional Executive Officers 7 positions Service Support Manager 76 positions Roles targeted within the region (and included in Pilot Training Currently not in scope for CIW – need additional licenses to cover CIW is available to regional positions & to the Executives at the Head Office All Regions & CSCs trained All Regions with this role has been trained ~500 ~430 Now12 months Caseworker >2100 positions Max 6 per Mgr Casework Manager Casework 400 positions Max 6 per Mgr Client Service Casework Specialist 76 positions No of Regions : 7 No of CSCs: 82

7. FACS Internal Users (Client Service Delivery) FACS Internal Users (Management) FACS Internal Users (Policy and Planning) FACS Internal Users (COMS and MDS) (Federal) SAP FACS Internal Users (Finance and HR) SAP Finance SAP HR CIW Enterprise Applications Architecture Client facing systems-Internal “Back Office” systems CWU and other agencies (data matching) BSDP & Siebel KiDS COMSOOHC MDS Partner Portal CIW is a central component of Family and Community Services Software Solution Map BF MDS

8. Senior Executive dashboard – High level indicators, updated and analysed monthly, Executive and Minister briefed on arising issues Full dashboard currently contains 8 KPIs. All indicators are drillable to Summary and Trend levels

9. Indicator Dashboards - High level information is linked to “summary” reports by region and to “detail” reports at the child/record level

10. Data challenges Data quality and collection compliance issues Required information not collected Lack of source system linkages (Master Data Management) Insufficient data granularity Lack of structured data Complex counting rules used to overcome data quality issues

11. Our Continued Goal for the future …………To provide high quality information that meets the needs of all major stakeholders in their pursuit of delivering better services and improving performance.

12. Using the Data Client population forecasting Statistical analysis Program and project evaluation Cost benefit analysis Performance indicators

13. 13 Client population forecasting using simulation modelling FACS has developed a ‘simulation’ model that projects 10 years demand for child protection early intervention and placement prevention (EIPP) population for Out of Home Care (OOHC). Evaluate / measure the longer term impacts of new policies and services aimed at improving client outcomes. Assist policy development and matching future services to clients’ needs.

14. 14 Client population forecasting using simulation modelling, cont’d Each client’s characteristics drive its pathways, including re-entry Client characteristics also drive the type, intensity and duration of service it receives Services can change client pathways Each client’s characteristics also govern how long they stay in OOHC. The simulation model can be used to determine the demands / needs for the services: The results of the simulation model impact on potential policy development and decisions: Higher service intensity and duration means higher costs or fewer clients served Affects outcomes and re-entry Where budgets are fixed, increasing one service can be offset by reducing another The model projects the trade-offs, allowing decision-makers to make informed policy choices.

15. 15 Client population forecasting using simulation modelling, cont’d Demand, operational and cost implications of changes in one part of the system for the other parts of the system and for different client groups. OOHC Services + Expenditure Simulation Model (client movements) Scenarios policy demand practice procedure budget EIPP and CP service models Budget / unit costs = capacity Future CP and EIPP (un/met) demand

16. Demand modelling for NGO transition planning, cont’d 16 As illustrated in the graph, as at 30 June 2011, the percentage of CYP in Statutory care provided by FACS is approximately 70%, while NGO provides the remaining 30%. The results of the demand modelling indicate that FACS’ provision of Statutory care will reduce to approximately 10% in four years, and a corresponding 90% of services will be provided by NGOs. In 10 years, it is expected that 100% of Statutory Care services will be provided by NGOs.

17. 17 Forecast the number of additional NGO placements needed to achieve the targets under the Transition Plan over the coming years. NGO current contracted Estimated OOHC Service Levels ICA conversions + Entries to NGO sector + Transfers from FACS to NGO sector Baseline Projected capacity increase per year Baseline contracted including ICA conversions FY12 +FY12 growth +FY16 growth +FY15 growth +FY14 growth +FY13 growth Projected Capacity Increase Total Children & Young People Entry and exit patterns Restoration rates Adoption rates Placement prevention rates + + + Conceptual Overview – NGO OOHC Service Levels Forecasting

18. 18 Statistical analysis 1. Operational improvements Child and young person concern reports to FACS-CS increased by 324% from 1999/00 to 2008/09, and then plateaued following a change in criteria. The OOHC population increased 41% from June 2007 to June 2011. These changes have major impacts on FACS and so it is important to understand their source. FACS undertakes monitoring of this information and much is published in the Annual Statistical Report on the web.

19. Statistical Analysis Cohort analysis to look at re-reporting of children and young people by factors such as their age, Aboriginality, child protection history and OOHC history. Analysis of sibling groups, focusing on sibling groups that are frequency encountered by FACS. Where more information is required, detailed statistical analyses are undertaken. Some of these are available on the FACS website, including:

20. Statistical analysis 2. Program refinement/hypothesis testing As a variety of tools come into everyday use by FACS their performance needs to be assessed and impact monitored. Example – the Child Assessment Tool Refinement also applies to programs, with confirmation of better performance of some of the higher need families in Brighter Futures contributing to the development of Strengthening Families.

21. Statistical Analysis 3. Program and pilot evaluations Establishment of formal trials allows comparison over time of performance, based on criteria such as substantiated reports and time in OOHC before and after an intervention. This work provides the substructure for the cost benefit analyses which help determine whether specific programs continue. Example – Aboriginal Consultation Guide 4. Evidence base Information accumulated from these activities needs to be available and accessible so that it can be applied in future program development.

22. 22 An example: Evaluation - Applied Statistical & Economic Analysis Evaluations assess the objective performance of programs and projects both qualitatively and quantitatively using: (a) Internal evaluation staff who provide the conceptual framework, highly targeted support, i.e. robust measurement and assessment, for key organisational initiatives with minimal resource impact on project areas. (b) External staff (consultants). Internal staff manage the procurement process, from development of the brief to contract preparation and management.

23. 23 An example: Evaluation - Applied Statistical & Economic Analysis Evaluations contribute to the decision making process, including: Practice issues for FACS and its partners Existing service models Policy and program development FACS, CS, State Plan and COAG indicators Inter-agency programs Future funding arrangements

24. An example: Evaluation - Applied Statistical & Economic Analysis Evaluation staff also: Contribute to Multi-Agency evaluations, we providing advice, participating in governance and facilitating access to relevant data. Contribute results to the evidence base underlying NSW government policy settings such as early intervention, and intensive intervention.

25. 25 Conceptual Overview – Cost Benefit Analysis 2. Identify and measure costs and benefits 3. Calculate the costs and benefits for the time period of pilots 4. Apply discount rate to determine net present value of programs 5. Perform any additional sensitivity analysis or adjustments – e.g. age distributions 1. Identify programs and relevant stakeholders In performing a cost-benefit analysis, FACS typically adopts a five-step approach. Generally, the costs are determined using historical expenses of providing the programs. The benefits are derived from ‘cost savings’. Where the ‘cost savings’ exceed the expenses, there is a net benefit.

26. 26 Development and analysis of Key Performance Indicators for strategic planning, performance improvement and budgeting, cont’d Developing the performance management framework requires input from all levels of the organisation Developing a performance mgmt framework Implementing performance behaviour

27. The process for developing the performance management framework encompasses four stages. Monitoring is one component of successful service delivery. Monitoring should not be considered one off, rather an iterative process to continually refine practices. Development and analysis of Key Performance Indicators for strategic planning, performance improvement and budgeting, cont’d 27