PATHFINDER Mission: Results for New Zealanders

Agenda for WG4 1.Introduction (Chair) - includes short website update (Greg) 2.WG / WS Process (Chair) - building effective relationships - ideas for improvements (human & output) - feedback on Q&A sessions 3.New WG Products (Roger / Greg) - 10 early step performance cycle (StW 1-3) - Intervention Logic (one-off presentation) - Building Block / Learning Point documents 4.Other matters (Chair)

Focus on Outcomes

Pathfinder ‘Building Blocks’ 1.Define & measure ‘mission critical’ outcomes (using ‘state’ or ‘situation’ outcome indicators) 2.Map causal logic linking outcomes to outputs 3.Assess impact of interventions / targeting 4.Assess cost-effectiveness (vs. outcomes achieved) 5.Management to maximise outcomes (e.g. core/pilots) 6.Benchmarking with outcomes (business units / nations) 7.Focus strategic / annual plans on improving o/c 8.Redesign planning & operations to maximise o/c (incl. feedback & continuous improvement) 9.Improving outcomes across agencies / sectors Steps are not sequential – If feasible, do steps #6-9 early

Observation vs. Action Status quo outcome (client /service groups) Risk, Need & Likely Change (individuals / cases) Improvement in outcome (‘treated’ groups) Situation Report Benchmarking Crude targeting Priority setting Demand analysis Risk manage mnt Continue? Modify? Grow? Kill? MeasureApplicationsOutcome Type

Improving Results – 10 Step Performance Cycle Identify Core Outcomes Define in Measurable Terms Measure as State Indicators Identify Areas for Change in State Define / Refine Intervention Logic Identify Intervention Options Prioritise Intervention Options Design & Deliver Interventions Impact (o/c) Evaluation Framework Cost Effectiveness Analysis

Intervention Logic 1.Aligning activities / outputs to outcomes 2.Defining a robust Intervention Logic (IL) 3.Using IL 'backbones' as a top-down (strategic) tool to scope intervention options 4.Testing intervention logic is robust, and produces best approach (ex ante, ex post) 5.Strengthening intervention logic using evidence-based approaches (R&D, eval) 6.Continuous evolution and testing of intervention logic / frameworks Decision: Sufficient interest to arrange half-day session?

Documenting Learning Points 1.What’s best for NZ as generic guidance 2.Want strong support of documents (-> SG) 3.(Recognise there is no ‘one-size-fits-all’) 4.Seek consensus on any disputed points 5.Initial comments via 6.‘Living document’ for discussion at WG 7.Send to SG (discussion / approval / -> web site) Decision: Approval of principles / generic process; First Priciple … Keep it short(ish), keep it pithy;

1.Executive summary 2.Purpose statement 3.Generic framework (10 step process) 4.Attributes of good outcome frameworks 5.Methods / steps / issues 6.Management and ownership 7.Best practice / good practice (integrated) 8.Worked examples (refer to supplementary docs?) Decision: Approval of format below as start point for writing; History: WG2 agreed focus on StW 1-3; no feedback on format; Proposal: ‘Trial by fire’ of the following format … Documenting Learning (Decision)

Documenting Learning (Contd.) 1.Keep it tight (one or few outcomes, clear priorities for activities) 2.Researched (outcomes derived from institutional knowledge and, where feasible, comparison with ‘similar / like’ agencies) 3.Clear causal links (outcome-output-input links clear; other means of achieving outcome(s) visible; clear fit to your services / clients; covers significant activities of your / relevant agencies) 4.Measures (objective / quantitative as possible; non-measurables identified; built on robust data; confidence intervals known; directly attributable to agency activities; can ‘drill down’ for underlying trends; ‘impact’ and ‘state’ outcomes; decision-making / quality outcomes) 5.Clear fit w. prioritisation / management decisions (e.g. measures inform key mgmt decisions, fit resource information) 6.Clear fit with vision / purpose (e.g. strategy / legislation) 7.Known relation to outcomes of other agencies Attributes of good outcome frameworks, e.g. …

Documenting Learning (Contd.) 1.Identify Core Outcome(s) (e.g. from mission / purpose identify key outcomes; compare to o/c / missions used by similar agencies overseas; ensure good fit w. results you expect from services you deliver) 2.Define in Measurable Terms (translate verbal statement of o/c into precise definition that matches data available at reasonable cost; ID key demographic / service / etc groups for which o/c are required) 3.Measure State / Environment Indicators (document system; gather / process data; measure o/c for all groups w. confidence intervals) 4.Identify Desired Change in State (analyse state indicators for statistically significant ‘highs and lows’; identify improvements sought; identify where results are good / some resources could be shifted) 5.Build Results into Business (e.g. report internally and externally; input into strategic plan, SoI, broad priority setting, KPI, benchmarking, etc) 6.Confirm Fit w. Outcome Management Framework (e.g. how can results be used to benchmark, strategise, prioritise, etc; how do measures have to change or be analysed to perform different functions) Generic processes, e.g. Building Block #1…

1.Executive summary 2.Purpose statement 3.Generic framework (10 step process) 4.Attributes of good outcome frameworks 5.Methods / steps / issues 6.Management and ownership 7.Best practice / good practice (integrated) 8.Worked examples (refer to supplementary docs?) Decision: Approval of format below as start point for writing; History: WG2 agreed focus on StW 1-3; no feedback on format; Proposal: ‘Trial by fire’ of the following format … Documenting Learning (Decision)

Agenda for WG4 1.Introduction (Chair) - includes short website update (Greg) 2.WG / WS Process (Chair) - building effective relationships - ideas for improvements (human & output) - feedback on Q&A sessions 3.New WG Products (Roger / Greg) - 10 early step performance cycle (StW 1-3) - Intervention Logic (one-off presentation) - Building Block / Learning Point documents 4.Other matters (Chair)