EU-Regional Policy Structural actions 1 GROWING EVALUATION CAPACITY THE MID TERM EVALUATION IN OBJECTIVE 1 AND 2 REGIONS 8 OCTOBER 2004

EU-Regional Policy Structural actions 2 Process of the Mid Term Evaluation

EU-Regional Policy Structural actions 3 1.Partnership Working 2.Evaluators 3.Quality 4.Cost PROCESS

EU-Regional Policy Structural actions 4 Partners: Commission, Member State, Managing Authority, Monitoring Committee, Evaluator, Beneficiary Steering Groups: positive contribution Organisation: generally a strong point Timing: work started appropriately early PARTNERSHIP

EU-Regional Policy Structural actions companies Growing incidence of consortia Limited transnational working Some evaluators over-stretched No significant increase in the size of the marketplace of evaluators EVALUATORS

EU-Regional Policy Structural actions 6 QUALITY (1) (Assessment by Commission on basis of MEANS criteria) Draft Reports %Final Reports % Good/Excellent3465 Acceptable4835 Unacceptable180

EU-Regional Policy Structural actions 7 QUALITY (2) Positive Result showing increased evaluation capacity, but Reports not too severely judged given breadth Too much analysis and editorial work compressed into short period (draft final reports often not finished)

EU-Regional Policy Structural actions 8 COST (1)

EU-Regional Policy Structural actions 9 COST (2)

EU-Regional Policy Structural actions 10 COST AND COST EFFECTIVENESS Over 35 million in total for Objectives 1 and 2: Resources appropriate, but cost effectiveness limited by: timing, breadth of evaluation, and methodological weaknesses

EU-Regional Policy Structural actions 11 Methodologies

EU-Regional Policy Structural actions 12 METHODOLOGIES Mix of methodologies including desk and primary research to include evaluation of: developments in programme environment implementation mechanisms performance of measures, priorities and programme

EU-Regional Policy Structural actions 13 DESK RESEARCH Included in all evaluations Difficulties with monitoring information: late or slow start of programmes monitoring systems not yet functioning

EU-Regional Policy Structural actions 14 PRIMARY RESEARCH Surveys Focus Groups/Workshops Case Studies

EU-Regional Policy Structural actions 15 METHODOLOGIES - CONCERNS Limited primary research Weak monitoring data Analysis Over-emphasis on financial analysis Conclusions and Recommendations Presentation of the reports

EU-Regional Policy Structural actions 16 Findings of the Evaluations

EU-Regional Policy Structural actions 17 FINDINGS OF THE EVALUATIONS 1. Key Evaluation Questions Appropriateness of Strategy Effectiveness Efficiency Quantification of Objectives Implementation Systems 2. Contribution to Lisbon Priorities Transport Knowledge Based Economy Entrepreneurship and Financial Instruments Social Inclusion Environment and Sustainable Development

EU-Regional Policy Structural actions 18 KEY EVALUATION QUESTIONS Limited Need for Strategy Shifts Effectiveness – Much Activity Underway after a Slow Start Macro-economic Impacts Results and Outputs Efficiency – A Start Made in the Analysis of Unit Costs Quantification of Objectives – an Urgent Need for Improvements Implementation Systems – Significant Improvements compared to the Past

EU-Regional Policy Structural actions 19 CONTRIBUTION TO LISBON PRIORITIES Transport – Good progress in large Road and Rail projects; smaller scale projects behind schedule Knowledge Based Economy Research, Innovation, Information and Communication Technologies (ICT) – A growing priority with good results in Objective 2 regions; slower but definite progress being made in Objective 1 regions Human Capital

EU-Regional Policy Structural actions 20 CONTRIBUTION TO LISBON PRIORITIES (1) Entrepreneurship and Financial Instruments – Performing well but affected by the global economic downturn Social Inclusion – Integrated actions are most effective but are resource intensive Environment and Sustainable Development – more consideration needed of the practical implementation of horizontal priorities

EU-Regional Policy Structural actions 21 Use of the Evaluations

EU-Regional Policy Structural actions 22 USES OF THE EVALUATIONS Mid Term Review Adaptation of Implementation Systems Disseminating information about the Structural Funds Feeding debate on public policies

EU-Regional Policy Structural actions 23 USING THE EVALUATIONS Who used the evaluation? In what way? Were financial allocations changed? Were implementation methods changed? Are the evaluations freely available? Has there been public debate? Is there any intention to further consider findings? Did any factors militate against use?

EU-Regional Policy Structural actions 24 Conclusions

EU-Regional Policy Structural actions 25 STRENGTHS & WEAKNESSES Strengths: Organisation and planning Partnership Allocation of Resources Increased evaluation capacity Weaknesses Rigid deadline Breadth of requirements ToR not sufficiently adapted Quality concerns

EU-Regional Policy Structural actions 26 CHALLENGES FOR THE FUTURE 1.Further Strengthening Evaluation Capacity Managing evaluations – human and financial resources Evaluators – commitment to quality and more targeted methodologies New Member States – building on experiences 2.Improved Monitoring Systems 3.Creating a Platform for exchange of experience