1. 1 Impact Evaluation in the European Commission Adam Abdulwahab Evaluation Unit, DG Regional Policy Budapest, 6 th May 2010

2. 2 What are we talking about? Counterfactual impact evaluation Identify what happened compared to What would have happened in the absence of the intervention (the counterfactual) Originates from medical experiments Treated group – control group (– placebo?)

3. 3 “Clear cut separation should help prevent antagonism, which is rife when proponents of alternative methods vie for the attention of the same policy makers and compete for the same resources. Claims of the alleged intellectual superiority of a set of methods over the other is the most deleterious manifestation of such rivalry and should be discouraged” Alberto Martini EVALSED, DG Regional Policy

4. 4 Why and how to do it? CIE => impact of a particular intervention Many CIEs => impact of the tool Needed: –Medium-sized population of eligible units –Possibility to divide the population into treated and control groups –Similar treatment –Good before/after data

5. 5 Where to do it? If methodologically applicable (investments in SME, R&D, human capital) AND Relevant for decision-making; the intervention –Aims to change (vs. pure redistribution) –Is replicable (inductive)

6. 6 When to do it? Retrospective: –Needs good “before data” –Answers accountability questions Prospective: –Accompanying programmes from the design phase allows informed decisions (vs. simple monitoring of outputs) –Strengthens focus on results –Improves monitoring –Improves selection criteria

7. 7 Example I Enterprise support in Eastern Germany Several forms of matching with similar lesser assisted firms Clear leverage effect: approx 8,000 euro/employee aid generated 12,000 euro/person additional investment Supported enterprises invested 2.5 times the amount of non-supported enterprises

8. 8 Example II R&D support (aid to enterprises) in Thuringia, Germany Comparison: firms not receiving assistance, selected by propensity score matching R&D investment: 11,500 euro/employee in supported enterprises vs. 4,000 euro/employee in non-supported

9. 9 Example III Ex post evaluation of URBAN Community Initiative No results because of Poor data availability Complex interventions

10. 10 The Future Further enterprise and R&D studies, to verify results Social and employment measures Stratification – a key tool to boost efficiency

11. 11 Thank you for your attention For further information, see: http://ec.europa.eu/regional_policy/sources/docgener/evaluation/evaluation_en.htm