1. Global Workshop on Development Impact Evaluation in Finance and Private Sector Rio de Janeiro, June 6-10, 2011 Impact evaluation of R&D support program for entrepreneurs in Poland Impact Evaluation Concept Note

2. Intervention Summary Matching grants for R&D works used for satisfying the particular needs of entrepreneurs.  Entrepreneurs can conduct R&D on their own, assuming they possess the infrastructural base and other necessary resources, or  Enterpreuners can contract R&D out to a scientific institution. This is only small, but important part of the system of enterprises support Marcin has described yesterday.

3. Intervention Summary  Programme started in 2008;  Total allocation - 390 mln euro;  So far c. 80 per cent of allocation contracted, and further 13 per cent in selection procedure;  every fourth applicant received grant;  ca. 500 projects started;  median support ca. 380 thous. EUR,  average ca. 850 thous. EUR.

4. Evaluation Questions  Hypothesis 1: For SMEs, strong positive input additionality for private R&D expenditures can be observed (support causes increase both in total R&D expenditures and private R&D expenditures).  Hypothesis 2: For large enterprises, substitution effect is substantial (private expenditures go down – negative additionality of private R&D) and only minor input additionality for total R&D expenditures can be observed.  Hypothesis 3: Selection process is designed in the way that promotes low-risk, less innovative projects versus high-risk, more innovative projects.

5. Evaluation Design  Randomisation: projects selected in non-random way – companies apply for funds in responce to open calls,  Control group problem: no statistical or administrative data that we could use as a control group.  Crucial characteristics are non-observable or hardly observable (attitude toward innovations, quality of the research project, risk involved in the project)  so survey on a sample of general population is not a solution (you cannot control for most important differences).

6. Evaluation design  We do not know the distribution of scores given to the projects at the moment (but we will) – from what we know it seems that using discontinuity is the way to go;  Matching the observation (PSM - propensity score matching) is a possible solution, as we possess a lot of information on companies that applied, including financial statements and information on staff and assets;  The problem is that we are unable to observe project/company “innovativeness”, but this is part of the selection process;  If “innovativeness” assessment seems to be reliable, it can be included in the regression of R&D input.

7. Sampling and Data 2010 call for proposals

8. Sampling and Data  Scope of the IE: projects supported/rejected in 2008 and 2009;  Important: project usualy takes 2-3 years to complete;  c. 900 projects in the pool (submitted and accepted formally);  c. 500 projects funded (the basis for treatment group);  c. 400 projects rejected (the basis for control group).

9. Technical issues  The study should be conducted by the Polish Agency for Enterprise Development (responsible for the programme), with external company to do the survey;  One of the difficulties to overcome: gathering the data within PARP (we do not know the distribution yet…);  The study should consist of in-depth survey of the companies in the IQ or IIQ 2011 (many projects finished or advanced by then, results on time to be used in preparation of the next programming period;  Control group firms will need incentives to take part in the research;  The total costs unlikely to exceed 160 000 EUR.

10. Future issues Ex-ante evaluation:  fake assessment – to compare results of assessment process  random selection of projects to be financed from waiting list  comparison of results achieved by firms in different programmes Changes in the programme:  more risky projects, no incremental innovations  preventing crowding SMEs out by large companies (separate calls/no financing for large companies?)  new selection process (committee, meetings with company, gatekeeper)

11. Team  Vincenzo Di Maro (World Bank)  David McKenzie (World Bank)  Marcin Piątkowski (World Bank)  Anna Tarnawa (Polish Agency for Enterprises Development)  Sylwia Waśniewska (Ministry of Economy)  Lixin Colin Xu (World Bank)  Julian Zawistowski (Institute for Structural Research)