1. 1 Monitoring & evaluation 2013+: concepts and ideas (ERDF & CF) CMEF 2013+ meeting, 17 th June 2011, Kai Stryczynski, DG REGIO Evaluation Unit

2. 2 Why a focus on results? Policy Debate emphasises need for: –Focus on demonstrable results and impacts rather than inputs; –Concentration to maximise effects and European added value of cohesion policy; –Strong links with EU2020 objectives; –Better programming with clearer articulation of strategic objectives; –More impact evaluation. Focus on results an even greater imperative with pressures on public budgets.

3. 3 What is a result*? Each programme priority should articulate what you want to change (intended result). Express the dimension of change with a result indicator. Need to know the baseline. Who does this? Member States, regions. * Result or outcome. What would be the French word for outcome?

4. 4 How do you want to change things? Put on paper what factors you think are likely to affect your result indicator – and how Select the factor(s) the public programme should influence – what would the programme produce? You have found your output indicator and described an intervention logic Monitor outputs: Obligatory common indicators Custom indicators

5. 5 What is an impact? Impact (effect) is the change that can be credibly attributed to an intervention.. Change in result indicator = effect of intervention + effect of other factors

6. 6 Monitoring: what can be observed. Can changes in result indicators be observed? Yes! - For the whole targeted population - For beneficiaries. This is the left-hand side of the equation. And has nothing to do with impact.

7. 7 Data sources for result indicators Regional EU-wide statistics – example: The European Union Statistics on Income and Living Conditions (EU-SILC), Eurostat. Regional, nation-wide statistics – example: Local Data Bank, GUS Poland. National or sub-national registers (administrative sources). Surveys, expert judgement,… Application forms.

8. 8 Evaluation: an inquiry into the non-observable Distinguishing the contribution of the programme – the impact – from the effect of other factors. Theory-based evaluations inquire the mechanisms that make a contribution of the programme happen. Counterfactual evaluations put a number on the contribution of the programme to the change.

9. 9 Combine approaches and mix methods Both types of impact evaluation answer complementary questions – so combine them. Any inquiry delivers hints and partial answers – look at problems with different methods for a safe judgement.

10. 10 Key changes Precise place for „impact“ –Plus better methods to put a number on it Better distinction monitoring – evaluation. No more „impact indicators“. –To indicate implies something observable. Impacts cannot be observed. –Impact occuring in the longer term or in the wider economy can be evaluated ! Call it long-term results, final goal,…

11. 11 Next steps Discussion of Concepts and Ideas paper with MS, Change into draft REGIO guidance paper after publication of legislative proposals (September) EVALSED: methods for evaluation – continuing update http://ec.europa.eu/regional_policy/sources/docgener/evaluation/eval sed/index_en.htm

12. 12 1999-2013 A shift of focus towards outcome (results) Post 2013 Inputs Outputs Results Impacts Global objectives Specific objectives Operational objectives Programme objectives Programme operations Logical framework 1999-2013

13. 13 Other Factors Monitoring and Evaluation Programming Strategy Needs Thematic Objective Intended Result Contribution ( Impact ) Policy Allocated INPUTS Targeted OUTPUTS Actual INPUTS Achieved OUTPUTS Actual Result New logical framework

14. 14 Weblinks: Concepts and ideas EVALSED Thank you for your attention!