ESF evaluation partnership November 16th, 2012 The French evaluation system: Main achievements 2007-2013 Claude RACK Responsible for evaluation ESF Managing Authority Ministry of Labour, Employment, Vocational training and Social dialogue

Summary Presentation of the structure of the system Ongoing evaluation : a stronger link between monitoring and evaluation Sharing the information : collection and uses of data Challenges for 2014-2020 -

A double orientation for combined objectives : Structure of the system A double orientation for combined objectives : Operational : performance oriented, through outputs and outcomes; Strategic :qualitative studies on relevance, coherence, added value, specific role of ESF. In order to highlight implementation options.

Structure of the system The ESF Scientific committee for evaluation : - 20 members: experts and stakeholders of the programme, European Commission - Guarantees the quality of the evaluation work, chooses the studies, approves the terms of reference, validates the methodology proposed - Ad hoc technical groups are set up The evaluation team in the managing authority of ESF (Ministry of Labour) assists the Committee; handles the evaluation work as a whole, manages the calls, informs the local teams of ESF, builds bridges between evaluation and policy makers A team of independant evaluators within the framework of a 3+3 years agreement

Structure of the system : Tools An information system for monitoring and evaluation : A tool to monitor files, collect qualitative indicators (transversal priorities) A tool to collect data on participants (annex 23 and exits) A platform with agregated data for accountability and communication Presage_web interface D1-D2 Viziaprog FSE 5

Structure of the system : Tools 6

Structure of the system : Tools Annex 23 Different exit categories : - Business creation - Access to temporary or seasonal work - Access to supported contract - Access to lasting employment - Access to qualifying training - Certified training - Access to validation procedures of end of course by a jury made of professionals - Return to school training - Other types of positive exits - Total of positive exits - Breakdown/failure - Other exits -Total exits

Structure of the system : Tools

Monitoring and evaluation 24 quantified targets per priority. Examples: 35% senior participants in the training of employees; 80% low qualified people in the training of employees; Increase of 10 percentage points of the rate of companies, who have survived three years after creation; 38% female entrepreneurs benefiting from the support of ESF; 30% young participants in employment after leaving a « job center for young people »; 65% of positive exits after leaving the action in a « plan départemental d’insertion ».

Monitoring and evaluation Lessons learned in 2011 Targeting: share of senior workers in training : Still very difficult but increasing, the share of low qualified workers in training is easier to reach. Target Total Female

Monitoring and evaluation Lessons learned Relevance of action taken to face the crisis The share of disabled young people has slightly improved Good results for entrepreneurship, especially for females (chart below) Active inclusion results < target but remaining approximately the same Target Total

Monitoring and evaluation Lessons learned : The quantitative part enriches the qualitative studies Ex: Equality between women and men Share and results of women compared to men’s (gender profile and exits) to bring an objective assessment, before analysing how the issue is taken into account in the files, as well as in regional strategies avoiding « story telling » Estimating the level of gender mainstreaming: share of files with actions specifically dedicated to the issue

Monitoring and evaluation Exits broken down by gender 13

Sharing the information 2000-2006 estimating the number of participants 2008-2011 : struggling for data collection : from 12,7% to 92% of complete files 32 611 files, 420 local managers, 4 millions participants The managing authority delivered: guidance, Q&A, notes, staff meetings, monitoring of regions and bodies, national training What for ? Monitoring : AIR, annual checks Evaluation : regional meetings twice a year, AER, part of the qualitative studies

Sharing the information A double aim Monitoring/ operational evaluation Effectiveness Efficiency of implementation Performance : Result based approach Monitoring : outputs, results AIR, annual checks Strategic evaluation Relevance Coherence Added value Role of ESF Performance : Relevance of the strategy Evaluation on outputs, results, policies, priorities, implementation, unexpected effects Annual evaluation report Qualitative studies

Sharing the information 6 qualitative studies 2009-2012 + Ex ante Evaluation of the communication plan 2009; ESF and equality between men and women 2010, updated in 2012 ESF and active inclusion services at the local level 2009- 2010 ESF and participants in training courses : characteristics and outcomes 2011; the OP ESF evaluative monitoring system implementation, benchmark and prospects 2012 ESF securing professional transitions 2012 Ex ante 2012 2013

Sharing the information The uses of evaluation : The annual evaluation report written by the evaluators summarizes quantitative and qualitative results. A unique practice ? Evaluation for action : - Communication : has fed the communication plan 2010-2013 - Active inclusion : re examining the role of ESF , - Training : questionning the categories of intervention types of training (unexpected result) - Equality between women and men recommendations to local managers for 2011 and 2014 - Ex ante : inputs to thematic discussion groups to build the logical framework of the next programmes

Challenges for 2013 Quality of data: which breaks? the story of «other positive exits » The information system: a unique tool Electronic data exchange system Indicators all through the audit trail Keep on sharing the information Conclusion : - A system which has improved a lot since 2008, still to be completed, - A cultural revolution, which prepared 2014 - A collective involvement - A larger communication to undertake

Thank you for your attention www.fse.gouv.fr