1. Active labour market measures and entrepreneurship in Poland Rafał Trzciński Impact Evaluation Spring School Hungary, 18.05.2011

2. case 1 – Evaluation of @lternative II project Objective of the project: preventing unemployment among young people. The territorial scope: 59 poviats of Poland (NUTS 4) with unemployment rate above 20% and high unemployment among young people. Eligibility: young unemployed (27 or younger), registered at the labour office. Total number of beneficiaries: 5 657. Budget: 4 090 702. Type of services: pre-training assistance (recruitment, needs assessment, guidance); vocational training services linked with ECDL, both at basic and advanced level, as well as related areas; practical training (temporary employment/ on the job training organised under the agreements signed with employers; training allowances; vocational guidance and post training assistance; job-broking. Period of implementation: 2005-2006.

3. ? Selection to the project Self-selection to the project Problem: High unemployment rate among young people @lternative II project Efect: lower unemploy- ment Factor x 1 Factor x n SELECTION BIAS Evaluation problem

4. The selection bias problem in control/comparison group approach To estimate the impact of the intervention we cannot simply compare beneficiaries (treated) with those who did not participate in the project (non-treated) This is because of factors affecting both participation and outcomes. If we don't control for those factors, we can overestimate or underestimate the impacts(picking the winners/ picking the losers).

5. Data used in the evaluation PULS System, which: is used for services for the unemployed is present in approximately 90% of Poviat Labour Offices in Poland (2006) provides a wide range of data on each unemployed person (socio-demographics, employment characteristics, previous qualification improvement, skills etc. ), includes a detailed history of unemployment and other activities on each person (registration in the office, deregistration, trainings, use of the benefits, etc.).

6. Data collection We collected data from 55 of the 59 Poviat Labour Offices involved in the project. In total we managed to identify 5 065 participants of the @lternative II project (90% of all beneficiaries). Moreover we collected data on 126 633 persons (non-treated), which meet the formal conditions for eligibility for the project (registration in the labour office, age condition).

7. The regional distribution

8. Variables Socio-demographic characteristics Sex Age Marital status Single parenting Number of children Education Poviat Employment characteristics Profession (ten categories) Number of days of work Number of professions (in total) Number of days being unemployed before participating in the project Number of days receiving the unemployment benefit before… Number of job offers during the one year period before… Number of days participating in subsidised work Number of days of permanent unemployment (during the two years period before…) Previous qualification improvement Number of training courses, in which the person participated during the one year prior to participation in the project Total number of days spent on training Having a work placement before participating in the project Motivation to find a job Percentage of showing up in the Poviat Labour Office, Having the right to unemployment benefit Skills Possession of driving license (B category)

9. Back to the selection problem…

10. Counterfactual action Treatment Bearing in mind the assumptions… Conditional Independence Assumption Population A Population B Treatment Counterfactual action We assume that if we can control for observable differences in characteristics between the beneficiares and non-treated population, the outcome (observable change) that would result in the absence of treatment (counterfactual action) is the same in both populations. Ergo, we assume that unobservables do not affect the outcomes!

11. Eligible non participants (N=126 633) Control group (N=5 065) ps= 0,6 ps= 0,5 ps= 0,8 ps= 0,1 ps= 0,2 ps= 0,3 ps= 0,2 ps= 0,01 ps= 0,4 ps= 0,9 Beneficiaries (N=5 065) ps= 0,8 ps= 0,3 ps= 0,9 ps= 0,4 ps= 0,1 Propensity score matching (1-1; nearest neighbour)

12. What we have achieved using PSM?

13. Impact Source: Ex-post evaluation of Phare 2003 Economic and Social Cohesion – Human Resources Development component, PAED, Warsaw 2007

14. Impact

15. Cost-benefit analysis

16. Objective of the project: encouraging business activities among unemployed people. Beneficiaries: unemployed people (with priority to young job-seekers). Type of services: initial business training; guidance on conducting economic activities; training allowance; relevant specialised training; coaching after setting up a business. Time of implementation: 2004-2005. Evaluation framework: the same approach as in the @lternativa II exaple (the same methodology, source of data, analysis...). (Counter)example 2 – Entrepreneurship promotion project

17. Impact? Source: Ex-post evaluation of Phare 2002 Economic and Social Cohesion – Human Resources Development component, PAED, Warsaw 2006

18. Lessons learned/points for the discussion What data we were lacking in both examples? Missing covariates? (Are our assumptions plausible?) Missing outcome variables? What do we know and what we don't know after completing the evaluation (towards theory based impact evaluation)? How we could modify the plan of the evaluaton to get more insight on impacts (targeting issue)? What is the avaibility of systems such as PULS in other EU countries (looking for possibilities of implementing IE)? What is the utility of data collected in public statistics? Do we need new data systems for IE or maybe we need to modify existing ones? (towards more systematic discussion on IE planning).

19. Thank you!!!