1. Veronica Gaffey & Antonella Schulte-Braucks
Results and Performance for Cohesion Policy A Step Change for the Future
Veronica Gaffey & Antonella Schulte-Braucks
DG REGIO / DG EMPL
Vilnius, 4 July 2013
Cohesion Policy

2. Effectiveness an overall principle
Commission and Member States shall ensure the
effectiveness of the ESI Funds, in particular
through monitoring, reporting and evaluation (Art.4 CPR)
Cohesion Policy 2

3. Four building blocks for result orientation
Result orientation of programmes
Indicators
Performance framework, review and reserve
Evaluation and Reporting
Objective is the same but technical details vary by Fund – due to the scope of the Funds
Cohesion Policy

4. What is a result?
= what each priority axis and investment priority wants to change / achieve for each category of region or target group
E.g., accessibility of a region, increase the number of start-ups of young people below 25.
The measurable dimension of results must be captured with indicators:
E.g. Reduction in travelling time
Number of start-ups of young people.
Cohesion Policy 4

5. Common Indicators
Included in annexes to the fund specific Regulations (indicator title and measurement unit)
Definitions in Guidances, developed in consultation with the Member States
Special attention to measurement units and reporting conventions 5

6. Investment priorities
Result indicators
Specific objectives
what the MS wants to achieve, taking into account national and regional challenges, needs and potentials
Investment priorities
Description of actions needed to achieve the specific objectives
Output
Indicators
Categories of intervention
Cohesion Policy

7. Performance Framework Building Blocks
Priority level
Milestones (intermediate targets)
Targets
To be achieved by the end of 2018
Formally reviewed in 2019
To be achieved by the end of 2022 (2023)
Formally reviewed in 2024
Indicators: Financial, Output, Result (?), Key implementation steps
Possible postponement of the deadline due to N+3 rule.
Realistic and achievable (taking into account – inter alia – the timeframe, form of intervention and resources)
Relevant to the objective of the priority
Ex-ante evaluation may help to judge.
Revision possible (art. 26) in duly justified cases, such as a significant change in the economic, environmental and labour market conditions in a Member State or region, and in addition to amendments resulting from changes in allocations for a given priority.
Cohesion Policy

8. Challenges for Performance Frameworks
Keeping it as simple as possible (limited number of indicators)
Coverage of the Priority Axis
Setting realistic milestones and targets
Consistency within a MS for similar priorities (should be described in Partnership Agreement)
Cohesion Policy 8

9. Performance review & Allocation of Reserve
In 2019, examines the performance of the programme priorities against the milestones set for the end of 2018
the European Commission adopts a decision on priorities which have attained their milestones
MS proposes the attribution of the performance reserve (7%) among the performing priorities.
Foreseen success rate: all indicators reach 85% of a milestone.
Where a Member State fails to submit the information in accordance with Article 44(4) and (5), the performance reserve for the programmes or the priorities concerned shall not be allocated to the programmes concerned]
Cohesion Policy

10. Suspensions or corrections
EC may suspend all or part of an interim payment of a priority if:
serious failure to achieve the milestones due to clearly identified implementation weakness.
EC has communicated previously this clearly identified implementation weaknesses and the MS has failed to take the necessary corrective action.
At the end of programming period, the EC may apply financial corrections if, in addition:
no significant socioeconomic, environmental developments or implementation delays beyond control of the MS prevented the achievement of targets.
What a serious failure is? At least two indicators below 65%.
Cohesion Policy

11. Evaluation Ex Ante: Ongoing:
Focused more strongly on intervention logic
Possibility to use ex ante to obtain data for baselines
Assessment of performance framework
Ongoing:
Evaluation Plan – to MC no later than one year after programme: timing, evaluations, data, methods, communication/use, budget
Impact evaluation(s) covering each priority
Report summarising evidence and main outputs and results of the programme in December 2021
Cohesion Policy
Regional Policy 11

12. Impact Evaluation No one method favoured over any other
All evaluations should pay attention to the theory of change, and mobilise an appropriate mix of methods to conclude on the effects of interventions, including:
Literature review, including economic theory, previous evaluation results, etc.
Review of administrative and monitoring data – on beneficiaries/ participants/entities
Quantitative counterfactual work where appropriate
Qualitative techniques: interviews, focus groups, case studies, performance story reporting, etc., etc
Cohesion Policy
Regional Policy 12

13. Reporting
Annual report by Commission to Council and Parliament, summarising AIR data (outputs, results, performance framework progress) and all available evaluations of Programmes
General Affairs Council to discuss every 2 years
Quality of Reporting – quantitative and qualitative – must improve for all MS and all Funds
Cohesion Policy
Regional Policy 13

14. ERDF/CF Specific Requirements
New Focus on Results:
What do you want to change?
What indicator can capture this change?
What is the baseline (the situation before the programme)?
How will the outputs of the programme contribute to change?
Results relate to change in the region/sector – not just for supported entities

16. Role of Result Indicators for ERDF/CF
Capture what you want to change
Should be sufficiently close to policy – policy to be reflected in the evolution of the result indicator
Targets quantitative or qualitative
Regular monitoring to prompt policy debate (not sanctions)
Selected by programmes – not common– recognising the different "journeys" to EU2020
Evaluation to disentangle the contribution of the policy to change from the influence of other factors (impact)
Possibility to evaluate impact because the objective was clear 16

17. ERDF/CF Output Indicators: Common & Specific
Capture what the resources are spent on
Common & Programme Specific
Baselines zero, Quantified Cumulative Targets
Intervention logic - how should this amount of resources spent on these outputs contribute to change in result indicator – to be assessed in ex ante evaluation
Common Indicators:
Relate to most frequently implemented actions
Provide aggregate information for analysis & communication purposes 17

18. ESF Specific requirements
Focus on results:
What do you want to change?
What indicator can capture this change?
What is the baseline?
How will the outputs of the programme contribute to change?
Results relate to change for supported persons/entities, not in the region/sector
= Short link – in terms of causality 18

19. 19

20. ESF Common Result Indicators:
Immediate results: change upon leaving the intervention
Longer term results: change after e.g. 6 months 20

21. ESF common output and result indicators
IPs shall always report against all common indicators
Limited number of indicators linked with a target
Baselines for indicators with a target
Baselines could be set using experience from previous programmes
Cumulative targets, quantified for common indicators, quantified or qualified for programme- specific 21

22. Recommendations
Draft OPs with a clear intervention logic = prerequisite for evaluability
Prepare comprehensive evaluation plans which will allow to assess the effects of the interventions
Set up appropriate data collection arrangements which will deliver good quality and timely data 22

23. Guidance documents ERDF + Cohesion Fund ESF
ESF
Ex ante evaluation (ERDF, ESF, CF)
Performance framework (ERDF, ESF, CF, EAFRD, EMFF)
To follow…
Cohesion Policy