EU-Regional Policy and Cohesion Structural Funds and Accession 1 SPP BUILDING IMPLEMENTATION CAPACITY Training seminar on evaluation Prague February 2004 A VIEW FROM THE COMMISSION ON EVALUATION IN THE CZECH REPUBLIC Pascal Boijmans European Commission DG Regional Policy

EU-Regional Policy and Cohesion Structural Funds and Accession 2 STRUCTURE OF PRESENTATION Experience with ex-ante evaluation in the Czech Republic - strong points - points that need further attention Building up evaluation capacity in the Czech Republic - evaluation plan - organisation - financing Evaluation vs. Monitoring Conclusions and recommendations Sources of information

EU-Regional Policy and Cohesion Structural Funds and Accession 3 EXPERIENCE EX-ANTE EVALUATION IN THE CZECH REPUBLIC NDP: Berman Group, Charles University, Faculty of Science OP Industry & Enterprise: VSE, March Consulting, IFO Institute OP HRD: National Observatory for Employment and Training OP RDMFA: Institute for Economic and Environmental Policy, University of Economics (VSE) OP Environment: Cityplan OP Transport: DHV CR OP Tourism: Europroject, Redeco JROP: VSE Prague Obj. 2: DHV CR Prague: Obj. 3: National Observatory for Employment and Training

EU-Regional Policy and Cohesion Structural Funds and Accession 4 EXPERIENCE EX-ANTE EVALUATION IN THE CZECH REPUBLIC genuine efforts to follow EU guidelines: working paper no 2 on ex ante evaluation - lessons from the past - socio-economic analysis - choice of strategy and priorities: internal and external coherence - quantification of objectives - expected socio-economic impact and allocation of resources - implementation arrangements separate Strategic Environmental Assessments (SEAs) for each programme involvement universities and research institutes

EU-Regional Policy and Cohesion Structural Funds and Accession 5 EXPERIENCE EX-ANTE EVALUATION IN THE CZECH REPUBLIC standard Terms of Reference restricted tender or direct agreement guarantee of independence evaluator - internal or external evaluator? financing - ex ante evaluations : national budget interactive process with programming exercise

EU-Regional Policy and Cohesion Structural Funds and Accession 6 EXPERIENCE EX-ANTE EVALUATION IN THE CZECH REPUBLIC Case study: ex-ante evaluation NDP by Berman Group - consortium of Czech and foreign experts - involvement Dept. of Social Geography and Regional Development, Faculty of Science, Charles University - interactive exercise with programming: large influence on final NDP - concise report with clear recommendations - input for negotiation mandate EC

EU-Regional Policy and Cohesion Structural Funds and Accession 7 BUILDING UP OF EVALUATION CAPACITY IN THE CZECH REPUBLIC COMMITMENT IN CHAPTER 21 FOR THE CZECH REPUBLIC: Binding rules will be set up for the evaluation of programmes and projects a permanent evaluation body will be created within each Managing Authority ate the level of OPs MRD will be responsible for the overall coordination of separate evaluation units

EU-Regional Policy and Cohesion Structural Funds and Accession 8 BUILDING UP OF EVALUATION CAPACITY IN THE CZECH REPUBLIC TEXT IN COMMUNITY SUPPORT FRAMEWORK STATES: A part of the budget under heading TA will be reserved to develop the evaluation capacity for programmes and for future programmes. Examples of areas to be covered are macro- economic assessment, reliability of indicator system and horizontal priorities A specific evaluation unit in Ministry for Regional Development is in charge of coordination

EU-Regional Policy and Cohesion Structural Funds and Accession 9 BUILDING UP OF EVALUATION CAPACITY IN THE CZECH REPUBLIC DEVELOPMENT OF AN EVALUATION PLAN Evaluation is a learning instrument Differentiation in approach between CSF and OPs CSF: - horizontal issues and priorities - Lisbon strategy - macro-economic assessment (HERMIN model) - reliability monitoring and indicator system - preparation for period OPs: - ex-ante next period sectoral issues Coordination role for MRD: exchange of information Focus on limited number of evaluations

EU-Regional Policy and Cohesion Structural Funds and Accession 10 BUILDING UP OF EVALUATION CAPACITY IN THE CZECH REPUBLIC ORGANISATIONAL ASPECTS Specific evaluation unit in MRD Interministerial working group Strengthen evaluation capacity within national authorities Development of tools how to use and spread added value of evaluations Discussion of outcome evaluations in Monitoring Committees: trigger for in depth discussions

EU-Regional Policy and Cohesion Structural Funds and Accession 11 BUILDING UP OF EVALUATION CAPACITY IN THE CZECH REPUBLIC FINANCING Technical Assistance: rule no. 11 of Reg. 1145/2003 (not capped) separate heading in TA-plan Eligible expenditure: - external expertise - additional staffing (capped) - training - workshops, seminars, study visits

EU-Regional Policy and Cohesion Structural Funds and Accession 12 EVALUATION VS. MONITORING different activities, but clearly depending on each- other importance of well functioning and reliable monitoring system is a precondition for good evaluation - MSSF: Central, Benefit, Monit - VIOLA - and other systems training of staff on input of data limited number of high quality and representative indicators - baseline - target - source of information Guidelines EC on indicators for new Member States

EU-Regional Policy and Cohesion Structural Funds and Accession 13 CONCLUSIONS & RECOMMENDATIONS Part I Development of standard Terms of Reference for all evaluations EC: preference for restricted tenders Ex-ante evaluation: parallel and interactive process with programming exercise Continue involvement of universities and research institutes Evaluation reports: concise and clear recommendations

EU-Regional Policy and Cohesion Structural Funds and Accession 14 CONCLUSIONS & RECOMMENDATIONS Part II Evaluation plan: focus on limited number of evaluations Differentiation in evaluation approach between CSF and OPs Develop tools how to use best the added value and results of evaluations Develop evaluation expertise within national authorities Assure reliability of monitoring systems as basis for evaluation tasks

EU-Regional Policy and Cohesion Structural Funds and Accession 15 INFORMATION SOURCES Evaluation Section of Inforegio website: ces/docgener/evaluation/evaluation_en.htmhttp://europa.eu.int/comm/regional_policy/sour ces/docgener/evaluation/evaluation_en.htm Commission working documents Evaluations undertaken for the Commission The new Evaluation GUIDE (forthcoming)