1. The Essence of ‘Managing for Outcomes’ Planning

2. The Big Picture Review of the Centre and other reviews have identified Real strengths in New Zealand’s public sector management And recommended a second generation of reform - evolutionary

3. In Sum … Review of the Centre and other reviews have recommended Keep what works: –Transparency, outputs, freedom to manage, financial management Shift the balance: –More on outcomes & capability, “whole of government”, more central leadership

4. Managing for Outcomes Managing for outcomes is a significant change in the way departments manage their business It means an outcome focussed approach to planning, management, and reporting

5. Results are the Focus Intention is to improve the performance of departments Expected to strengthen link between public management and results Government wants to achieve “Managing the results chain”

6. Leadership Ministerial involvement in, and commitment to the devopment of the outcomes is encouraged CEs’ leadership crucial to the success of the new management approach

7. Accountability Departmental accountability for outputs is retained Departments also to be held accountable for ‘managing for outcomes’

8. We are All Involved Central agencies and TPK are involved in this change To contribute to its success, we have made a commitment to open and collaborative relationships with departments – providing support and adding value

9. We are All Involved This will include Support for collaboration on sector or whole of government outcomes and services Brokerage, helping to share best practice

10. What Our Involvement Will Mean Being able to give good advice to departments on the concepts behind managing for outcomes Collaborating with each other to ensure well integrated, challenging advice to departments

11. A Learning Curve A learning curve is involved for everyone Start early, don’t overreach Don’t miss the good in the pursuit of excellence

12. Generic Planning Process and Agency Models Bryan Dunne Pathfinder Secretariat http://io.ssc.govt.nz/pathfinder/

13. Identify Vital Few Outcomes Define in Precise Terms Measure as State Indicators Identify Areas for Change in State Define / Refine Intervention Logic Agree Impact Measurement Framework Design & Deliver Interventions Review Impact Evaluation Results Cost Effectiveness Analysis Strategy Formulation Optimising Performance Ex Ante Improving Performance A generic 10 step cycle Capability ID & Prioritise Intervention Options ID & Prioritise Intervention Options

14. Agency Function & Focus Some Work In Progress SocialEconomicEnvironmental Age Gender Ethnicity Region Service Risk, etc. O/C1 O/C2 O/C3 O/C4 O/C5O/C6 Domain Outcome

15. Operational Agency (primary focus: own clients) SocialEconomicEnvironmental O/C1 O/C2 O/C3 O/C4 O/C5O/C6 Age Gender Ethnicity Region Service Risk, etc. Domain Outcome

16. Sector Focused Agency (primary focus: community; sector Intervention Logic) Age Gender Ethnicity Region Service Risk, etc. Domain Outcome SocialEconomicEnvironmental O/C1 O/C2 O/C3 O/C4 O/C5O/C6

17. Population Agency (primary focus: population group of interest; the ‘vital few’) SocialEconomicEnvironmental O/C1 O/C2 O/C3 O/C4 O/C5O/C6 Domain Outcome Age Gender Ethnicity Region Service Risk, etc.

18. ‘Whole of Government’ Approach Age Gender Ethnicity Region Service Risk, etc. Domain Outcome SocialEconomicEnvironmental O/C1 O/C2 O/C3 O/C4 O/C5O/C6

19. Administrative Functions ?????? ?????? ?????? ?????? ?????? ?????? Age Gender Ethnicity Region Service Risk, etc. Domain Outcome SocialEconomicEnvironmental O/C1 O/C2 O/C3 O/C4 O/C5O/C6

20. Identify Vital Few Outcomes Define in Precise Terms Measure as State Indicators Identify Areas for Change in State Define / Refine Intervention Logic Agree Impact Measurement Framework Design & Deliver Interventions Review Impact Evaluation Results Cost Effectiveness Analysis Strategy Formulation Optimising Performance Ex Ante Improving Performance Capability ID & Prioritise Intervention Options ID & Prioritise Intervention Options

21. Identify Vital Few Outcomes Define in Precise Terms Measure as State Indicators Identify Areas for Change in State Define / Refine Intervention Logic Design & Deliver Interventions Review Impact Evaluation Results Cost Effectiveness Analysis Limited Evidence of Impact Strong Ex Ante Planning Cannot Measure Impact? Agree Impact Measurement Framework Capability ID & Prioritise Intervention Options ID & Prioritise Intervention Options

22. Identify Vital Few Outcomes Define in Precise Terms Measure as State Indicators Identify Areas for Change in State Define / Refine Intervention Logic Design & Deliver Interventions Review Impact Evaluation Results Cost Effectiveness Analysis Limited Evidence of Impact Conceptual Ex Ante Approach Cannot Measure State Either? Agree Impact Measurement Framework Capability ID & Prioritise Intervention Options ID & Prioritise Intervention Options

23. Remember Build management, not outcome measurement, systems Focus on what matters most – the ‘vital few’ can help here Defining the vital few in precise terms, measurable if possible, gives greatest gains But conceptual (non-measurable) approaches will give ‘one-off’ gains … so pursue all the ‘vital few’

24. Remember Different types / uses of outcome measures State Indicators – identify where problems persist, monitor trends and identify where change is desirable (ie priorities) Impact Measures - significant technical challenge to measure impact but ongoing gains in outcomes needs impact feedback Takes time & perseverance – you may not get it right first attempt

25. Outcome Definitions Greg Claridge Roger Waite Pathfinder Secretariat http://io.ssc.govt.nz/pathfinder/

26. The purpose: Performance management Outcome measures - a vital management tool for identifying problem areas and gauging the effectiveness of an organisation’s activities, services, and outputs. They indicate performance against ex ante targets or goals. Without this “feedback” information management can only guess the success (or failure) of their decisions. Evidence trumps opinion every time

27. Types of Outcome Measures:* State Indicator: is the condition experienced by the community (or a specific group, entity or geographic area) at a given point of time. It is the result of all influences and actions - the impact and consequences of government action and the impact and consequences of actions and activities beyond the control of government. *- see Pathfinder BB1 and Glossary ‡ - aligned to PFA definition of “Outcome” Impact Measure ‡ : the impact of one or more Government interventions in improving a state indicator.

28. Define / Refine Intervention Logic Agree Impact Measurement Framework Design & Deliver Interventions Review Impact Evaluation Results Cost Effectiveness Analysis Capability ID & Prioritise Intervention Options 1. Identify Vital Few Outcomes 2. Define in Precise Terms 3. Measure as State Indicators 4. Identify Areas for Change in State Strategy Formulation Optimising Performance Ex Ante Improving Performance A generic 10 step cycle Define / Refine Intervention Logic Agree Impact Measurement Framework Design & Deliver Interventions Review Impact Evaluation Results Cost Effectiveness Analysis Capability ID & Prioritise Intervention Options 4. Identify Areas for Change in State 3. Measure as State Indicators

29. 1. Identify ‘Vital Few’ Outcomes Remember: outcomes are most useful to management if they are measurable, attributable to core activities, and timely. The ‘vital few’ outcomes should: be well aligned with the agency’s or business unit’s mission or purpose; be linked to services, outputs and inputs (the things agencies manage); collectively measure major outcomes from or across all dominant output classes; measure the benefits experienced by clients or entities the agency manages; be timely; and support critical business decisions, including resource decisions.

30. 2a. Define Outcomes in Precise Terms Precise, unambiguous, written definition of changes partially attributable to the activities of one or more agencies which: Is limited to the ‘vital few’ – defines, not measures, outcomes; Clearly relates to the consequence of the agency’s outputs or activities for the community; Supported by knowledge of the influences driving outcomes; Is focused on target groups for services; Is linked to accessible source data; and Aligned to the department’s mission ….but, do not exclude unmeasureable outcomes

31. 1. The ‘Vital Few’ Outcomes 2b. Corrections: Outcome Definitions 2. Reducing Reoffending of offenders over a 12 month period following their release from a custodial sentence or from the begining of a community based sentence. (p19 Corrections Statement of Intent 2002/03) 1. Contribute to Safer Communities

32. 2b. Corrections: Outcome Definitions 2. Reducing Reoffending of offenders over a 12 month period following their release from a custodial sentence or from the begining of a community based sentence. (p19 Corrections Statement of Intent 2002/03) 1. Contribute to Safer Communities 2. Correction’s rehabilitation activities impact the behaviour of offenders

33. 2b. Corrections: Outcome Definitions 2. Reducing Reoffending of offenders over a 12 month period following their release from a custodial sentence or from the begining of a community based sentence. (p19 Corrections Statement of Intent 2002/03) 1. Contribute to Safer Communities 3. Supported by international research on reducing offender recidivism.

34. 2b. Corrections: Outcome Definitions 2. Reducing Reoffending of offenders over a 12 month period following their release from a custodial sentence or from the begining of a community based sentence. (p19 Corrections Statement of Intent 2002/03) 1. Contribute to Safer Communities 4. Focussed on a target group

35. 5. Accessible data source 2b. Corrections: Outcome Definitions 2. Reducing Reoffending of offenders over a 12 month period following their release from a custodial sentence or from the begining of a community based sentence. (p19 Corrections Statement of Intent 2002/03) 1. Contribute to Safer Communities

36. 6. Aligned to the department’s mission: “The Department of Corrections contributes to community safety by:… reducing re-offending.” 2b. Corrections: Outcome Definitions 2. Reducing Reoffending of offenders over a 12 month period following their release from a custodial sentence or from the begining of a community based sentence. (p19 Corrections Statement of Intent 2002/03) 1. Contribute to Safer Communities

37. 2c. Ministry of Health: Outcome Definitions Interested in DALY: to minimise the numbers of years of life lost…. DALY = years lost from premature mortality plus equivalent years lost to severe disability. 1. The vital few.

38. 2c. Ministry of Health: Outcome Definitions Interested in DALY: to minimise the numbers of years of life lost…. DALY = years lost from premature mortality plus equivalent years lost to severe disability. 2. Clearly relates to the consequences of the Ministry’s activities.

39. 2c. Ministry of Health: Outcome Definitions Interested in DALY: to minimise the numbers of years of life lost…. DALY = years lost from premature mortality plus equivalent years lost to severe disability. 3. Supported by knowledge of drivers…. “Evidence-based Health Objectives” 2001

40. 2c. Ministry of Health: Outcome Definitions Interested in DALY: to minimise the numbers of years of life lost…. DALY = years lost from premature mortality plus equivalent years lost to severe disability. 4. Target group for services… “All New Zealanders throughout their lives”

41. 2c. Ministry of Health: Outcome Definitions Interested in DALY: to minimise the numbers of years of life lost…. DALY = years lost from premature mortality plus equivalent years lost to severe disability. 5. Linked to accessible source data….NZ and other

42. 2c. Ministry of Health: Outcome Definitions Interested in DALY: to minimise the numbers of years of life lost…. DALY = years lost from premature mortality plus equivalent years lost to severe disability. 6. Aligned to mission… “ Good health and wellbeing for all New Zealanders throughout their lives.”

43. State Indicators Greg Claridge Roger Waite Pathfinder Secretariat http://io.ssc.govt.nz/pathfinder/

44. Define / Refine Intervention Logic Agree Impact Measurement Framework Design & Deliver Interventions Review Impact Evaluation Results Cost Effectiveness Analysis Capability ID & Prioritise Intervention Options 1. Identify Vital Few Outcomes 2. Define in Precise Terms 3. Measure as State Indicators 4. Identify Areas for Change in State Strategy Formulation Optimising Performance Ex Ante Improving Performance A generic 10 step cycle Define / Refine Intervention Logic Agree Impact Measurement Framework Design & Deliver Interventions Review Impact Evaluation Results Cost Effectiveness Analysis Capability ID & Prioritise Intervention Options 1. Identify Vital Few Outcomes 2. Define in Precise Terms

45. 3a. Measure as State Indicator Quantitative measure of prevailing situation that is: Clearly linked to agency services, and groups or entities receiving (or candidates for) services; Reported in aggregate terms, and separately for population groups, regions, business units, etc; Inform management decision-making, Measured in absolute terms; and Statistically robust

46. 3b. Outcome Measures – the FABRIC attributes: Focused Appropriate Balanced Robust Integrated Cost effective –On the agency’s missions, aims & objectives –To, and useful for the stakeholders who are likely to use it –Giving a picture of what the agency is doing & covering all significant areas of work –Measure outcomes that have a strong linkage to core activities, based on accurate observations and calculations –Into the agency, part of the business planning and management processes –Balancing the benefits of the informatioin against the costs of measurement and management responses (UK National Audit Office)

47. 3c. Corrections: Recidivism Index (State Indicator) Reconviction based on a single event Time to Reconviction = 12 and 24 months Seriousness = reconviction to a Department Managed Sentence and Imprisonment For Offences committed post release Aggregated into Services, age, ethnic, gender, offence and sentence length groups 1. Clearly linked to Correction’s services

48. 2. Data can be disaggregated 3c. Corrections: Recidivism Index (State Indicator) Reconviction based on a single event Time to Reconviction = 12 and 24 months Seriousness = reconviction to a Department Managed Sentence and Imprisonment For Offences committed post release Aggregated into Services, age, ethnic, gender, offence and sentence length groups

49. 3c. Corrections: Recidivism Index (State Indicator) Reconviction based on a single event Time to Reconviction = 12 and 24 months Seriousness = reconviction to a Department Managed Sentence and Imprisonment For Offences committed post release Aggregated into Services, age, ethnic, gender, offence and sentence length groups 3. An absolute (not relative) measure.

50. 4. Statistically robust 3c. Corrections: Recidivism Index (State Indicator) Reconviction based on a single event Time to Reconviction = 12 and 24 months Seriousness = reconviction to a Department Managed Sentence and Imprisonment For Offences committed post release Aggregated into Services, age, ethnic, gender, offence and sentence length groups

51. 3d. More Justice Sector State Indicators

52. 3e. Ministry of Health: State Indicator DALY = YLL + YLD Modifiability: The modifiability multipliers were then applied to the DALY estimates extracted from the New Zealand Burden of Disease Study to rank conditions. Distribution: Used to incorporate distributional concerns (Mäori compared to non-Mäori), or ‘equity’, into the burden of disease analysis. Equity adjustor = 1 + [p (RR – 1) / p(RR – 1) + 1]

53. 4a. Identify areas for change in State: Health Prioritisation

54. 4b. 13 Health Objectives The 13 population health objectives (not ranked) are to: reduce smoking improve nutrition reduce obesity increase the level of physical activity reduce the rate of suicides and suicide attempts minimise harm caused by alcohol and illicit and other drug use to both individuals and the community reduce the incidence and impact of cancer reduce the incidence and impact of cardiovascular disease reduce the incidence and impact of diabetes improve oral health reduce violence in interpersonal relationships, families, schools and communities improve the health status of people with severe mental illness ensure access to appropriate child health care services including well child and family health care and immunisation.

55. 4c. Identify areas for change in State: LTSA Target Group

56. 5. Steps to developing good State Indicators: Identify the outcome information and decision-making needs of the Minister, external stakeholders, and decision-makers and staff within the agency; Involve stakeholders early. Identify the crucial outcomes sought from the agency’s (or business unit’s) mission; Identify good practice via literature sources and knowledge of other agencies; Specify how dominant activities (outputs) are intended to influence outcomes; Specify indicators in measurable terms against each outcome; Specify the groups for which indicators will be reported; Specify data collection and verification systems; and Get Ministerial endorsement of the outcomes and measurement system, and senior management approval to access the resources needed to measure indicators.

57. Wrap Key attributes or FABRIC Time and patience: this is not an easy process Limitations of outcome based management Next steps: –Outcome definitions and state indicators are the necessary precursor to IMPACT MEASURES –Development of evidence based, demonstrated performance management systems