Evaluation Arrangements for : A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development Rossella Tarantino Evaluation Unit – Basilicata Region Bruxelles June 2008

Presentation Outline Presenters background: collective evaluation capacity building Overall approach to evaluation: decentralization, responsibility and Evaluation plans Main issues: –Evaluation questions –Involvement of social and economic partners A regional perspective on a central endeavour

National Evaluation System Work group: collective work for evaluation capacity development Individuals from: –Evaluation Unit of Ministry for Development (responsible for work group coordination) –ISFOL (Agency responsible for evaluation of ESF) –INEA (Agency responsible for evaluation of Rural Development) –Regional Evaluation Units

Decentralization and responsibility Decentralization: –each Regional government responsible for evaluating its interventions –Coordination ministry responsible for evaluating from central point of view –Regional ownership: each Region asks the evaluations it needs and times it for when it needs the results No common evaluation questions No prescribed timing Responsibility: –No sanction –Evaluation plans and evaluations are public –National Evaluation System guarantees guidance, training, support, systematic observation and reputational mechanisms

Evaluate the effects of regional policy Until the end of 2009, mainly ex post evaluations of past interventions Evaluate EU- and nationally funded interventions Evaluate effects of joint interventions on areas/groups Evaluation questions on specific issues, groups, areas, NOT generic program

Steering Groups Ensure involvement of socio-economic partners: Steering Group at Plan level Ensuring qualityand independence of each evaluation: Steering Group (+ other methods) at each evaluation level. No MA. Risks: creating a new profession: Steering Group member Duplicating the problems of Monitoring Committees

Evaluation plans One plan per Region/central ministry regarding all programs (past and present) (unitary approach) Need to build a new internal organization 30 programs Only two Regions are still developing the plan (out of 21)

Evaluations planned for 2008 e 2009 Human Resources: 5 Education: 1 Urban policies: 1 Transport: 2 Implementation analysis: 13 Program evaluation: 18 Research & innovation : 6 Enterprises: 3 Environment: 3 Cultural heritage: 1 Gender: 3 Area-based: 9 ICT: 2

Satisfaction at innovation spread, but Out of 67 evaluations planned for , 31 (+ all the regional development evaluations) still have a generic Structural Fund program focus: –Inertia in accepting innovations from regulations and from NSF –Need for general overview –Incomplete or contradictory support from EC SNV: –strengthening our common work on ECD with EC –Increase capacity of national coordination agencies Further work needs to be done to involve social & economic partners Is a Plan Steering Group the right way to go about it?

The National Evaluation System and the evaluation plans Guidelines for the organisation of the evaluation activities of the regional policies: the evaluation plans for ; Working groups that went through the evaluation plans drafted by Regions and Ministers according to a peer group approach. These groups were created on volunteer basis and composed of both regional and national representatives. This cross-regional and national scrutiny of the evaluation plans enabled a pooling of experiences and know-how. for instance, after going through the evaluation plan of the Liguria Region I changed/improved the evalution plan of my Region

The evaluation plan: the unitary approach Each Region has a unitary strategy implemented through OPs One Evaluation Plan for Each Region that assess: ERDF OP ESF OP EARDF OP OP funded by National Fund for deprived areas Interventions financed by regional resources

The evaluation plan: the unitary approach to assess all the development policies implemented by the Region –independently by the financial source – in order to have a global overview of the effects that these policies produce. Obviously in the respect of: –Specific rules concerning each Fund; –The need of the European Commission and of the single Managing Authorities to underline the specific community added value.

To coordinate the evaluations and avoid overlapping and duplication; To build a global overview of the individual evaluations and thus have a big picture of the effects of the regional development strategy; To assess the results produced jointly by different actions on the same area or target group To have a shared knowlegde to improve the quality of programs and of their implementation The unitary approach- why?

Specific evaluation activities focus on themes/areas/target groups and take into account results produced by different interventions An ex-post evaluation in my region concerns the information society and the contribution to: The quality of the public services (i.e; health) The creation of community networks (i.e. Schools) The territorial and social inclusion The competitive capacity of SMEs How to deal with organizational and financial issues? Pooling of funds (liguria) Steering Group with the participation of all Managing Authorities and departments involved in these policies (Basilicata) To address specific national resources (Emilia oRmagna) The unitary approach-how

In some regional plans there is a divide between the ERDF and ESF evaluations In the implementation phase, the unitary approach is even more difficult Therefore, for us (Regions) it is important: EC common approach EC Common guidance EC (DG Regio and DG Employment + DG Agri) joint meetings and analysis with National Evaluation Systems The unitary approach: not easy

EC support and guidance The joint analysis on the mid-term evaluation conducted on 2004 was very useful and gave important hints to improve the quality and awareness of evaluations: –focus on specific questions, stakeholders involvement in the definition of questions and in the dissemination of evaluations results, emphasis on effects, etc Guidance Participation in our (SNV) ECD activities