1. United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano & Laura Tagle Italys National Evaluation System

2. Content 1.Cohesion policy in Italy: long-term background & contrasting directions 2.The Italian evaluation policy 3.Efforts & (unlikely) achievements… plus some drawbacks

3. COHESION POLICY IN ITALY: LONG TERM BACKGROUND & CONTRASTING DIRECTIONS

4. 2007-2013 - Evaluation as one of the pillars of a (much needed) renewal of Italys regional policy Citizen-centered policies Strong focus on essential servicesrather than on procedures & expenditure Emphasis on local knowledge Responsibility to regional governments Links with national policies Greater importance to administrative capacity

5. The regional (= cohesion) & rural development policy in Italy Intentionality of territorial goals Additionality vis-à-vis ordinary policy Resources come from EU & National funds

6. Levels of programming: a complex framework National umbrella strategies for regional & rural development policies 7 Ministries develop their own programs Each of the 21 Regions develops –its multi-sector strategy –5 Programs (each program is funded by one source) Naturally, each of the 5 programs is usually managed by a separate Directorate Programs end up being bureaucratic constructs rather than guides for action

7. Over time, silos have been created Sources of money EU Regional Fund National funds EU Social Fund EU Rural Development Coordination organisms Ministry for economic development Ministry for agriculture Ministry for labour Other departments

8. ….Beyond silos 2007-2013: intentional efforts to bridge gaps & introduce policy dialogue among and within authorities in a new perspective

9. At the same time, EU regulations ask for separate programmes by EU source of money (& within Italy the momentum for integration dwindles) The National Strategy for 2007-2013 tried to make the various sources of funding work together through common: Programming Implementation Monitoring Surveillance Evaluation

10. 21 Regions (70%) 6 Ministries (30%) Centre-North: 15% South: 85% 3bln to be given to Southern Regions on the basis of performance in waste, water, education and social services Regional development strategy Between 2007 & 2010: lower national resources (other than national cofinancing) different distribution North-South the instruments needed to implement nationally-relevant innovations (among which greater integration) dwindle

11. THE ITALIAN EVALUATION POLICY

12. Evaluation capacity building/1 From EU evaluation requirements towards a national policy Institutional building: –Creation of Evaluation Units within executive branch of Regional authorities –Network of Evaluation Units Specialized: –Regional and Rural Development policies only Both demand & supply side –Improve regional authorities ability to Demand for & Use evaluations –Improve Evaluation Units ability to Manage and Conduct evaluations

13. Evaluation capacity building/2 Organized guidance & support (National Evaluation System) Enabling approach: –No rigid prescriptions –Reputational mechanisms Legitimize & support innovators at regional level

14. Evaluation policy /1 Nationally-funded interventions must be evaluated--not only those funded by EU Objects: – effects of a well-defined (however complex) intervention or group of interventions on a problem, a social group, or an arearather than a bureaucratic construct (e.g., a multisectoral program tipically operating on a large area, affecting millionsof people) – across bureaucratic borders: the evaluation object may have been financed by more than one source – selectivity: choose to answer defined questions about crucial & controversial issues

15. Evaluation policy/2 Each Region drafts an Evaluation Plan: –What is evaluated (and what is not) –Who (internally or externally) performs each evaluation –When –Organization of evaluation function –Resources (human and financial) devoted to evaluations

16. The National Evaluation System UVAL – Ministry of Economic Development (Coordinates) INEA – Ministry for Agricultural, Food and Forestry Policies ISFOL – Ministry of Labour Department for Equal Opportunities Evaluation Units of Regional Authorities Its composition shows a clear intention to work together across silos & involving Regions in a federal function

17. The National Evaluation System Each component differs from the others Central units: –Have differing rules, interpretations, goals, tasks & status Regional evaluation units: –Conduct or manage evaluations… –… but also provide project selection, planning, knowledge management to the Region –Have background and skills closer to regional fund policies (or national funds) rather than rural development or active labour policies

18. EFFORTS & (UNLIKELY) ACHIEVEMENTS… PLUS SOME DRAWBACKS

19. Common guidance - include & adjust for differing rules, traditions, goals Personalized support to Regions (all managing authorities together) Ensure quality & evaluation of evaluations (....) Observe processes & disseminate information ( jointly ) Facilitate dissemination ( conference-like meetings, workshops, create & maintain eval database www.dps.tesoro/valutazione.it ) www.dps.tesoro/valutazione.it A federal function performed across silos & by central & regional units

20. Some results All Regions developed an Evaluation Plan http://www.dps.tesoro.it/uval_linee_valutazione.asp#10 http://www.dps.tesoro.it/uval_linee_valutazione.asp#10 Evaluations have been launched: –and even completed: 50! (in addition to the mandatory ex post evaluations of rural development) –and a lofty 60% available on the internet Evaluation units (established between 2000 & 2002) have started conducting internal evaluations Still far from perfection but …

21. Why we are happy anyway… Activities started (and 50 concluded) even though –(or just because?) there are no enforceable requirements (no sanctions) –there is still strong inertia in the system old evaluations fulfilled needs of managing authorities, national coordinators, and Commission rapporteurs restricted market with marriages between managing authorities & evaluators –still little pressure from social partners & general public Innovations go together: evaluations seem to be used as an instrument by Regions willing to improve Innovations implemented where there has been more support from central state (not only on evaluation)

22. Thank you Laura.tagle@tesoro.it s.ciampi@isfol.it Valutazione.uval@tesoro.it Please send us feedback at these e-mail addresses