1. Data collection, Indicators and Evaluation ESF TWG Sevilla, 17 March Unit 03 Evaluation and Impact Assessment, DG EMPL

2. Problems with data collection
Counting participating acts instead of participants;
No knowledge of carry over or leavers;
Data missing or inputted too late;
Data quality not reliable.

3. Problems with indicators
Data not always consistent within a MS;
Limited coverage of certain target groups;
Most indicators cannot be aggregated to EU level;
Indicators often do not cover mainstreamed priorities;
Result indicators are ill-defined;
Limited linkage to policy processes.

4. Problems with the evaluation system
Repartition of responsibilities for evaluation causes difficulties;
No system of on-going evaluations;
Fixed timing for certain evaluations;
Data for ex-post evaluation insufficient.

5. Have the problems been solved in 2007-2013 (1)?
At the level of data collection?
Common database which includes data on participants.
But:
Data inputted too late;
Data quality not always reliable.
At the level of indicators?
OP Indicators at the level of priorities;
Annex XXIII.
But
Target setting remains difficult;
Data not always consistent within MS and between them;
Result indicators remain difficult;
Limited linkage to policy processes.

6. Have the problems been solved in 2007-2013 (2) ?
At the level of the evaluation system?
Shared responsibility for evaluation, Art ;
On-going evaluation system in the Member States.
But:
Ex post evaluation at EU level by COM;
Limited knowledge of MS evaluations at EU level;
Evaluation plan only in convergence regions.

7. Possible sources of these problems
Slow start of implementing programmes and sfc2007 not available early enough;
Uncertainty about reporting requirements (e.g. women in Annex XXIII);
Data is available but not inserted in SFC;
Inputting data is time consuming and sometimes complicated;
Incompatible or not functioning monitoring systems;
Incompatible monitoring timetables and/or slow data transfer;
Data protection legislation;
Reporting is a low priority;
Enforce reporting requirements on beneficiary difficult.

8. Monitoring and evaluation logic
COM
Monitoring and evaluation system
MS evaluation system
MS monitoring system

9. Feedback from Working Groups Partnership Meeting Seville, 16 March Unit 03 Evaluation and Impact Assessment, DG EMPL

10. Questions for the Working Groups (1)
Data collection:
How could the compatibility of data collection systems between COM and MS be improved?
How could the timetables for reporting at MS and COM level be made more attuned?
Should payments be linked to providing monitoring data?
How can the reliability and effectiveness of monitoring systems be further improved?

11. Questions for the Working Groups (2)
Indicators:
How could the quality of indicators be improved?
Do you see an added value in linking indicators with ESF policy domains or categories?
How could the process of target setting and target revising be improved?
How could reporting on participants (Annex XXIII) be improved?

12. Questions for the Working Groups (3)
Evaluation system:
Is the system of on-going evaluations working?
Do you see an added value for an EU-wide ex post evaluation?
Should every OP be subject to an ex post evaluation?
Should there be a link between payments and MS evaluations?
How could the added value of evaluation plans be strengthened?