Regional Policy Result Orientation of future ETC Programes Veronica Gaffey Head of Evaluation & European Semester 23 April 2013

Learning from the Past Ex Post Evaluation of :  We could quantify the activity (outputs) of Interreg, but not the impact  Policy objectives and programme strategies not specific or focused  Indicators and target setting not aligned with objectives Trying to capture the outcomes of programmes:  Pilot exercise run by Interact  Useful reflection process, BUT  Impossible to build a result orientation into a programme ex post… 2

Future Result Orientation 3 o An intervention logic that incorporates:  Selection of investment priorities and corresponding specific objectives – which relate to the needs of the territory to be addressed  Result indicator which captures the intended change for the territory/sector – baseline and target (qualitative or quantitative)  Output indicators which reflect the direct activities of the priority (common and programme specific), cumulative target values  Allocation of resources

A Performance Framework 4  Monitor that implementation is as planned  A subset of programme indicators  Financial & Output indicators & Key Implementation Steps  Milestones for 2018 and Targets for 2022  Must be unambiguous and milestones and targets realistic  Possible suspensions and corrections for non- achievement  Will NOT answer the impact question (what have been the effects of the policy?)

Regional Policy Difference: Results Orientation and Performance Framework? Interlinked but distinct:  Results orientation is wider and locates the programme in its context  Performance Framework is about efficient implementation of the programme 5

Regional Policy 6 Results Focus in Programme Design Guiding questions:  What do you want to change?  What indicator can capture this change?  What is the baseline (the situation before the programme)?  How will the outputs of the programme contribute to change? Results relate to change in the region/sector – not just for supported entities Regional Policy

7 Role of Result Indicators  Capture what you want to change  Should be close to policy – so that the policy may be reflected in the evolution of the result indicator  Targets quantitative or qualitative  Regular monitoring to prompt policy debate (not sanctions)  Selected by programmes – not imposed top down – recognising the different "journeys" to EU2020  Evaluation to disentangle the contribution of the policy to change from the influence of other factors (impact) Regional Policy

8 Role of Output Indicators Output Indicators:  Capture what the resources are spent on  Common & Programme Specific indicators  Baselines zero, Quantified cumulative targets  Rooted in an intervention logic - how should this amount of resources spent on these outputs contribute to change in result indicator? – to be assessed in ex ante evaluation Regional Policy

9 Common Output Indicators  Relate to the most frequently implemented actions  Provide aggregate information for communication purposes  46 indicators for ETC – 40 general and 6 ETC-specific  Should be complemented with programme specific output indicators when necessary  Avoid numbers of projects, or numbers of solutions/mechanisms/agreements – as in aggregate these are not meaningful Regional Policy

10 Challenges  Defining the specific objectives and associated result indicators – this requires choices to be made  Establishing the baseline and targets – qualitative or quantitative(the ex ante evaluation can help)  Understanding the intervention logic  Translating the result indicators into selection criteria to ensure projects will contribute to the results  Making sure that performance frameworks represent the priorities and that milestones and targets are fixed at an appropriate level Regional Policy

Reference documents - Guidance Monitoring and Evaluation for ERDF/CF Performance Framework & Reserve – being consulted Pilot Tests on Result Indicators Ex ante evaluation (ERDF, ESF, CF) e_en.cfm#1 11 Regional Policy

Thank you for your attention!