1. 2010 Improvement of the competitiveness of enterprises: analysis of net effects Prof. Jarosław Górniak Rafał Trzciński 26.02.2010

2. Objective of the evaluation study The main objective of the study was to estimate and assess the real effects of Sectoral Operational Programme Improvement of the Competitiveness of Enterprises, years 2004-2006 (SOP-ICE). Two measures were covered by the study: Measure 2.1 – Improvement of competitiveness of SMEs through advice, Measure 2.3 – Improvement of competitiveness of SMEs through investments

3. Background of the problem ? Factor I Factor II Problem: low competitiveness of enterprises SOP-ICE Effects

4. The research problem To what extent has the intervention contributed to the observed effects? What is the net effect of the implemented programmes? Net effect – the difference between what is observed and what would have been observed had the intervention not been implemented

5. How to measure the net effect? Revenues from sale: 2 million 1,5 million Net effect = Counterfactual 0,5 million = ? ? Counterfactual = ?

6. Propensity score matching Propensity score matching method helps to ensure that the treatment and control groups are similar in terms of observable characteristics (reduction of selection bias). The propensity score is the probability of assignment to treatment (program, project, etc.) given a set of covariates X. Propensity score is unknown and needs to be estimated, for instance, by using logistic regression. The main assumption: control group shows what would have happened to participants if they had not participated in the programme (selection bias is eliminated by conditioning on a set of observable variables).

7. Propensity score matching in its simplest form NON-PARTICIPANTSCONTROL GROUP ps= 0,6 ps= 0,5 ps= 0,8 ps= 0,1 ps= 0,2 ps= 0,3 ps= 0,2 ps= 0,01 ps= 0,4 ps= 0,9 TREATMENT GROUP (PARTICIPANTS) ps= 0,8 ps= 0,3 ps= 0,9 ps= 0,4 ps= 0,1

8. Evaluation of the Sectoral Operational Programme Improvement of the Competitiveness of Enterprises

9. Evaluated programs Measure 2.1 was aimed at increasing the competitiveness of Polish SMEs through facilitating their access to specialised advisory assistance. Budget: PLN 59,6 million (average level of co- financing amounted to PLN 27,8 thousand). Measure 2.3 was aimed at improvement of competitiveness of Polish SMEs through modernisation of their product and technological offer. Budget: PLN 1 489,6 million (average value of co- financing amounted to PLN 505,4 thousand).

10. Data sets used in the evaluation PAED data sets (databases of all applicants; databases containing information on the participation in other programs – Phare SME, SOP-HRD), 19 covariates such as age, revenue, size, legal form, assets, the use of other funding etc. Central Statistical Office data sets collected in the so-called F-01/I-01 form (Report on revenues, costs and financial results and the cost of fixed assets) 30 outcome variables such as net revenues from sale, average paid employment, expenditures, costs, etc. In the first case the data were used for the selection of the control group, In the second case the data were used to estimate the net effect of the support.

11. Model review (Measure 2.3) Variable Treated (mean value) Untreated (mean value) Control group (mean value) The standardized difference in percent (before matching) The standardized difference in percent (after matching) Total employment73,3095203,678070,2619-1,480,03 Employment of women20,5097171,304718,4324-1,710,02 Revenues68032,5452133853,212077556,1103-5,27-0,76 Assets39844,9119158897,639831228,8509-3,170,23 Amount of deminimis14170,845510479,629014917,19594,66-0,94 Age10,651210,799210,6376-1,770,16 Percentage of women,2714,3343,2745-25,75-1,26 Employment growth before the programme,6571,5367,670024,72-2,66 Revenue growth before the programe,7288,5981,717127,922,51 Assets growth before the programe,7082,6067,700621,521,62 Number of contracts signed in Phare 1,2365,48251,308857,34-5,50 Value of signed contracts in Phare 17417,79685771,359717886,012749,78-2,00 Application in SOP-ICE 2.1,0776,0121,070032,063,74 Contract in SOP-ICE 2.1,0518,0059,045327,673,90 Contract in SOP-HRD 2.3,1406,0152,119448,108,12 ………………

12. Results

13. Effects of advisory program (Measure 2.1) Net effect: PLN 2 942 thousand 43,3%

14. Effects of advisory program (Measure 2.1) Net effect: PLN 861 thousand

15. Effects of advisory program (Measure 2.1) 79,2% 24,3% Net effect: PLN 1 077 thousand

16. Effects of advisory program (Measure 2.1) Net effect: PLN 2 864 thousand 30% 44,6%

17. Effects of advisory program (Measure 2.1) 29,4% 20,8% Net effect: 5 FTE

18. Effects of advisory program (Measure 2.1) Net effect: PLN 335 thousand 65,4% 41,9%

19. Effects of advisory program (Measure 2.1) Net effect: PLN 542 thousand 48,4% 85%

20. Effects of advisory program (Measure 2.1) Net effect: PLN -72 thousand 71,4% 18,4%

21. Effects of advisory program (Measure 2.1) Net effect: PLN 1 thousand 105,9% 100%

22. Effects of investment program (Measure 2.3) Net effect: PLN 2 359 thousand 47,6% 48,1%

23. Effects of investment program (Measure 2.3) Net effect: PLN 1 391 thousand 55,5% 70,1%

24. Effects of investment program (Measure 2.3) Net effect: PLN 829 thousand 42,4%

25. Effects of investment program (Measure 2.3) Net effect: PLN 2 429 thousand 46,9%

26. Effects of investment program (Measure 2.3) Net effect: 14 FTE 20,5% 34,1%

27. Effects of investment program (Measure 2.3) 59%

28. Effects of investment program (Measure 2.3) Net effect: PLN 652 thousand 74,7% 95,6%

29. Effects of investment program (Measure 2.3) Net effect: PLN 223 thousand 72,4% 303,7%

30. Effects of investment program (Measure 2.3) 9,5% 260,4%

31. The main conclusions

32. SOP-ICE improved competitivness of participating SMEs. The effects of advisory programme (2.1) clearly surpassed the effects of investment programme (2.3) at the time of evaluation. Results assessment in relative terms differs from calculated in absolut terms – percent versus money.

33. Estimation of the net effect evaluation Net effect measurement addresses effectivness of the programme and does it well. Among others, it does not cover non-intended consequences of the programme – is fully goal based. It has to be supplemented with insight addressing other evaluation criteria guided by the (re)constructed programme theory.

34. Thank you for your attention! Polish Agency for Enterprise Development 81/83 Pańska Street 00-834 Warsaw, Poland Tel + 48 (22) 432 80 80 Fax + 48 (22) 432 86 20 + 48 (22) 432 84 04 Infoline: + 48 (22) 432 89 91/92/93 www.parp.gov.pl