Presentation is loading. Please wait.

Presentation is loading. Please wait.

Regional Policy Counterfactual impact evaluation What it tells us… and what it doesn't Daniel Mouqué DG Regional Policy, European Commission.

Similar presentations

Presentation on theme: "Regional Policy Counterfactual impact evaluation What it tells us… and what it doesn't Daniel Mouqué DG Regional Policy, European Commission."— Presentation transcript:

1 Regional Policy Counterfactual impact evaluation What it tells us… and what it doesn't Daniel Mouqué DG Regional Policy, European Commission

2 Regional Policy Reminder: Counterfactual = comparison

3 Regional Policy In practice, comparison group

4 Regional Policy This imposes conditions… Similar intervention over large "n" (law of large numbers)

5 Regional Policy … which only hold for certain measures Interventions which target individuals or enterprises Not infrastructures (exception: impact of infrastructures on individuals) Perhaps for area based initiatives (provided similar goals/means)

6 Regional Policy There are also data needs 1.Good data on the intervention (nature, scale, dates) 2.Good data on target indicators (before and after, including for non-beneficiaries) 3.The ability to link 1 and 2

7 Regional Policy Lessons learned from enterprise support studies DG Regional Policy doing & encouraging since 2008 What are we learning? And what would we like to know? In terms of: 1.Investment, capital constraints and other market failures (and how vary by firm and support size) 2.Impact of support on the enterprise (productivity, innovation, employment)

8 Regional Policy … about investment, capital constraints and other market failures? What do we learn…

9 Regional Policy Impact on investment in Eastern Germany (GEFRA 2010)

10 Regional Policy E. Germany not an isolated example

11 Regional Policy Small is beautiful 1 – the firms SchemeStudyFinding on large enterprises Comparison between SME size classes Investment grants, IT ASVAPP (2012) No or negative impact firms >250 employees Thorough exam: no difference RSA invest grants, UK Crisculo et al (2012) No impact for firms > 150 employees Impacts slightly higher for firms < 50 employees? DK Innovation Consortia CEBR, Denmark (2010) No impacts for firms > 150 employees Not examined Innovation support, DE Czarnitzki et al (2011) Small much better, but firm or grant size? Smaller may do better Invest support, E. Germany GEFRA/IAB (2010) (Did not study large enterprises) No difference by SME size class

12 Regional Policy Small is beautiful 2 – the support ASVAPP (2012) even controlling for firm size, smaller grants more effective (cpj 79,000 for smallest grants, rising to 489,000 for largest). ASVAPP (2012) outright grant to SMEs similar effect to soft loan of same size Czarnitzki et al (2011) presence or absence of a grant was the crucial factor - smallest grants had almost the same innovation impact as the largest Comparing across studies: schemes of smaller support tended to have better results (eg RSA, UK)

13 Regional Policy Business advice can be cost effective Better survival rates 2-4 years later in North Jutland. 7500/net firm 1500/net job (Rotger and Gørtz, 2009)

14 Regional Policy What do we learn? Capital rationed for SMEs, but only partially Grants help – do not substitute private money This argument applies to small enterprises and (probably) to medium sized but not large firms Less support and/or financial instruments would still work Capital constraints not the only market failure: success of advice => information failures more serious, at least for smallest and newest firms?

15 Regional Policy What would we like to know? The mechanism for capital constraints? Knowing this would help for… Targetting by firm? And what too big for support? More effective solutions than direct financial support? (E.g. change capital market) What is the optimal level and form of support? What information failures? What is good soft support (incl. business advice)? How to target/tailor by context and firm? => Need more CFs and other types of evaluation

16 Regional Policy … about impacts on the firm? Productivity, innovation & jobs What do we learn…

17 Regional Policy Broader more often than deeper SchemeStudy> empl> productivity Law 488 invest grant, IT ASVAPP (2012) ++None SME support in Piemonte ASVAPP (2012) ++For loans, not for grants Enterprise support NI Hart & Bonner (2011) Small but stat. sig. Small but statistically significant SME grants, PL in Poland Trzciński (2011) ++None RSA invest grant UK Criscuolo (2012) ++Statistically insignificant DK Innovation Consortia CEBR (2010)Statistically insignificant Supported profits grew 12% more over 10 yrs

18 Regional Policy A closer look at some exceptions CEBR (2010) in DK: innovation consortia increased profitability 12% vs controls over a 10 year period (adds up to 260,000 extra profits per firm). Czarnitzki (2007): R&D subsidies in Germany had a significant effect on research and innovation where the firm also benefitted from networking Czarnitzki (2007): in Finland both financial R&D support and networking effective, and additive

19 Regional Policy CIS indics, Germany (Czarnitzki, 2011)

20 Regional Policy But innovation is not a panacea GEFRA (2010) investment impact of R&D grants < modernisation grants (leverage vs ). Innovation benefits worth loss in impact? De Blasio, Fantino & Pellegrini (2009) No additional impact from investment scheme: less tangible nature => more possibilities for deadweight

21 Regional Policy Jobs created, but < monitoring data SchemeStudyJobs supported (monitoring) Jobs created (from CF) Investment support, E. Germany GEFRA/IAB (2010) 107,000 "created", plus 439,000 "safeguarded" 27,000 Law 488 invest support, IT ASVAPP (2012) 82,000 "gross created" 36,000 "net" (beneficiary survey) 12,000 SME invest grants, PL Trzciński (2011) 25,000 "created"10,500

22 Regional Policy Job quality good ASVAPP (2012) average firm salary and productivity same or slightly greater Trzciński (2011) jobs created in SMEs received similar pay rises to those in the control group – and that jobs were maintained five years after support.

23 Regional Policy What do we learn? Relatively easy to make firms proportionately bigger (e.g. with grants) More difficult to make firms more innovative/productive (soft support better?) Measures with less tangible targets eg innovation can be abused (maybe we knew this already?)

24 Regional Policy What is left unanswered? And how would we answer this? Is soft support really the key to innovation? What is the mechanism for productivity and innovation? What types of innovation influenceable, how to target by firm etc? What constitutes a "smart" support package? What soft support, what mix with financial support? How to avoid abuse of innovation and networking measures? => Need more CFs and other types of evaluation

25 Regional Policy In conclusion …

26 Regional Policy In summary New lessons from CFs about impacts (partial capital constraints for SMEs, scaling up effect, importance of information failure) More to learn about impacts from CFs (e.g. soft support, financial instruments) Need other evaluation methods to open "black box" of mechanisms (targetting most effective solutions, best investments) Some factors too intangible for quantified approach? (innovation)

27 Regional Policy Beware the man whose only tool is a hammer… … for every problem comes to resemble a nail - Abraham Maslow

28 Regional Policy

29 For further information InfoRegio: Impact evaluation centre: evaluations/guidance_en.cfm#2

Download ppt "Regional Policy Counterfactual impact evaluation What it tells us… and what it doesn't Daniel Mouqué DG Regional Policy, European Commission."

Similar presentations

Ads by Google