1 Monitoring & evaluation 2013+: concepts and ideas (ERDF & CF) CMEF meeting, 17 th June 2011, Kai Stryczynski, DG REGIO Evaluation Unit

2 Why a focus on results? Policy Debate emphasises need for: –Focus on demonstrable results and impacts rather than inputs; –Concentration to maximise effects and European added value of cohesion policy; –Strong links with EU2020 objectives; –Better programming with clearer articulation of strategic objectives; –More impact evaluation. Focus on results an even greater imperative with pressures on public budgets.

3 What is a result*? Each programme priority should articulate what you want to change (intended result). Express the dimension of change with a result indicator. Need to know the baseline. Who does this? Member States, regions. * Result or outcome. What would be the French word for outcome?

4 How do you want to change things? Put on paper what factors you think are likely to affect your result indicator – and how Select the factor(s) the public programme should influence – what would the programme produce? You have found your output indicator and described an intervention logic Monitor outputs: Obligatory common indicators Custom indicators

5 What is an impact? Impact (effect) is the change that can be credibly attributed to an intervention.. Change in result indicator = effect of intervention + effect of other factors

6 Monitoring: what can be observed. Can changes in result indicators be observed? Yes! - For the whole targeted population - For beneficiaries. This is the left-hand side of the equation. And has nothing to do with impact.

7 Data sources for result indicators Regional EU-wide statistics – example: The European Union Statistics on Income and Living Conditions (EU-SILC), Eurostat. Regional, nation-wide statistics – example: Local Data Bank, GUS Poland. National or sub-national registers (administrative sources). Surveys, expert judgement,… Application forms.

8 Evaluation: an inquiry into the non-observable Distinguishing the contribution of the programme – the impact – from the effect of other factors. Theory-based evaluations inquire the mechanisms that make a contribution of the programme happen. Counterfactual evaluations put a number on the contribution of the programme to the change.

9 Combine approaches and mix methods Both types of impact evaluation answer complementary questions – so combine them. Any inquiry delivers hints and partial answers – look at problems with different methods for a safe judgement.

10 Key changes Precise place for „impact“ –Plus better methods to put a number on it Better distinction monitoring – evaluation. No more „impact indicators“. –To indicate implies something observable. Impacts cannot be observed. –Impact occuring in the longer term or in the wider economy can be evaluated ! Call it long-term results, final goal,…

11 Next steps Discussion of Concepts and Ideas paper with MS, Change into draft REGIO guidance paper after publication of legislative proposals (September) EVALSED: methods for evaluation – continuing update sed/index_en.htm

A shift of focus towards outcome (results) Post 2013 Inputs Outputs Results Impacts Global objectives Specific objectives Operational objectives Programme objectives Programme operations Logical framework

13 Other Factors Monitoring and Evaluation Programming Strategy Needs Thematic Objective Intended Result Contribution ( Impact ) Policy Allocated INPUTS Targeted OUTPUTS Actual INPUTS Achieved OUTPUTS Actual Result New logical framework

14 Weblinks: Concepts and ideas EVALSED Thank you for your attention!