1. ESF evaluation partnership
November 16th, 2012
The French evaluation system: Main achievements
Claude RACK
Responsible for evaluation
ESF Managing Authority
Ministry of Labour, Employment, Vocational training and Social dialogue

2. Summary Presentation of the structure of the system
Ongoing evaluation : a stronger link between monitoring and evaluation
Sharing the information : collection and uses of data
Challenges for -

3. A double orientation for combined objectives :
Structure of the system
A double orientation for combined objectives :
Operational : performance oriented, through outputs and outcomes;
Strategic :qualitative studies on relevance, coherence, added value, specific role of ESF.
In order to highlight implementation options.

4. Structure of the system
The ESF Scientific committee for evaluation : - 20 members: experts and stakeholders of the programme, European Commission
- Guarantees the quality of the evaluation work, chooses the studies, approves the terms of reference, validates the methodology proposed
- Ad hoc technical groups are set up
The evaluation team in the managing authority of ESF (Ministry of Labour) assists the Committee; handles the evaluation work as a whole, manages the calls, informs the local teams of ESF, builds bridges between evaluation and policy makers
A team of independant evaluators within the framework of a 3+3 years agreement

5. Structure of the system : Tools
An information system for monitoring and evaluation :
A tool to monitor files, collect qualitative indicators (transversal priorities)
A tool to collect data on participants (annex 23 and exits)
A platform with agregated data for accountability and communication
Presage_web
interface D1-D2
Viziaprog FSE 5

6. Structure of the system : Tools6

7. Structure of the system : Tools
Annex 23
Different exit categories :
- Business creation
- Access to temporary or seasonal work
- Access to supported contract
- Access to lasting employment
- Access to qualifying training
- Certified training
- Access to validation procedures of end of course by a jury made of professionals
- Return to school training
- Other types of positive exits
- Total of positive exits
- Breakdown/failure
- Other exits
-Total exits

8. Structure of the system : Tools

9. Monitoring and evaluation
24 quantified targets per priority. Examples:
35% senior participants in the training of employees;
80% low qualified people in the training of employees;
Increase of 10 percentage points of the rate of companies, who have survived three years after creation;
38% female entrepreneurs benefiting from the support of ESF;
30% young participants in employment after leaving a « job center for young people »;
65% of positive exits after leaving the action in a « plan départemental d’insertion ».

10. Monitoring and evaluation
Lessons learned in 2011
Targeting: share of senior workers in training :
Still very difficult but increasing, the share of low qualified workers in training is easier to reach.
Target
Total
Female

11. Monitoring and evaluation
Lessons learned
Relevance of action taken to face the crisis
The share of disabled young people has slightly improved
Good results for entrepreneurship, especially for females (chart below)
Active inclusion results < target but remaining approximately the same
Target
Total

12. Monitoring and evaluation
Lessons learned :
The quantitative part enriches the qualitative studies
Ex: Equality between women and men
Share and results of women compared to men’s (gender profile and exits) to bring an objective assessment, before analysing how the issue is taken into account in the files, as well as in regional strategies avoiding « story telling »
Estimating the level of gender mainstreaming: share of files with actions specifically dedicated to the issue

13. Monitoring and evaluation
Exits broken down by gender 13

14. Sharing the information
estimating the number of participants
: struggling for data collection : from 12,7% to 92% of complete files
files, 420 local managers, 4 millions participants
The managing authority delivered: guidance, Q&A, notes, staff meetings, monitoring of regions and bodies, national training
What for ?
Monitoring : AIR, annual checks
Evaluation : regional meetings twice a year, AER, part of the qualitative studies

15. Sharing the information
A double aim
Monitoring/ operational evaluation
Effectiveness
Efficiency of implementation
Performance :
Result based approach
Monitoring : outputs, results
AIR, annual checks
Strategic evaluation
Relevance
Coherence
Added value
Role of ESF
Performance :
Relevance of the strategy
Evaluation on outputs,
results, policies, priorities,
implementation,
unexpected effects
Annual evaluation report
Qualitative studies

16. Sharing the information
6 qualitative studies Ex ante
Evaluation of the communication plan 2009;
ESF and equality between men and women 2010, updated in 2012
ESF and active inclusion services at the local level
ESF and participants in training courses : characteristics and outcomes 2011;
the OP ESF evaluative monitoring system implementation, benchmark and prospects 2012
ESF securing professional transitions 2012
Ex ante

17. Sharing the information
The uses of evaluation :
The annual evaluation report written by the evaluators summarizes quantitative and qualitative results. A unique practice ?
Evaluation for action :
- Communication : has fed the communication plan
- Active inclusion : re examining the role of ESF ,
- Training : questionning the categories of intervention types of training (unexpected result)
- Equality between women and men recommendations to local managers for 2011 and 2014
- Ex ante : inputs to thematic discussion groups to build the logical framework of the next programmes

18. Challenges for 2013
Quality of data: which breaks? the story of «other positive exits »
The information system: a unique tool
Electronic data exchange system
Indicators all through the audit trail
Keep on sharing the information
Conclusion :
- A system which has improved a lot since , still to be completed,
- A cultural revolution, which prepared 2014
- A collective involvement
- A larger communication to undertake

19. Thank you for your attention