Evaluating the Hungarian NSRF Concept of on-going evaluation Sándor Csengődi National Development Office – Hungary Prague,

2 Structure of the presentation ______________________________________________________________________________________________________________________________ Background and culture of evaluation in Hungary Concept of evaluation system  goals and stakeholders  elements of the system  institutional framework Conclusions, remarks

3 Evaluation in Hungary _________________________________________________________________________________________________________________________________ Weak culture Lack of strategic thinking in use and allocation of development resources Evaluation considered as external obligation Misuse of evaluations Lack of interest Money allocation is connected to performance Weak democtratic participation in policy controll Needs and evaluation criteria come from outside the system Use of the results is rare Lack of capacity No real quality reqirements in the demand side Concentrated market and embedded evaluators on the supply side

4 DANGERS Low quality evaluations for high amounts of money Danger of supply-driven evaluation market development HOPES Public administration reform might generate (even political) needs for evaluation Openness of the program planning and implementing bodies => basis for bottom-up development Evaluation in Hungary _________________________________________________________________________________________________________________________________

5 Strategic documents (NSRF and OPs) representing goals and targets should be accepted by EC Flexible, continuous programming with more need of information More responsibility by the member states Flexible system of evaluation  EC has own evaluation capacities  No obligatory mid-term evaluation  Ongoing evaluation Changes in regulation _________________________________________________________________________________________________________________________________

6 Experiences from period _________________________________________________________________________________________________________________________________ Evaluation was a secondary issue when writing the plan =>Indicators’ system „added” after finalizing the plan Implementation focusses on absorption => low quality and quantity of result and impact indicators in the monitoring information system Planning and implementation are separated => information of implementation has priority MAs lack capacity and motivation for collection of evaluation data

7 Our concept ______________________________________________________________________________________________________________________________ Ongoing evaluations shall be carried out in support of raising the quality of planning and programming making program implementation more effective and efficient infroming responsible policy makers (+ Monitoring Committee, Government, EU) about results in their fields of interest serving information for external interest gropus (professionals, CSO’s, beneficiaries/nonbeneficiaries, etc.) The system must generate information for planning and programming.

8 Elements of the system _____________________________________________________________________________________________________________________________ Indicators’ system and regular reporting  Complex indicators’ system – information for more than one group of stakeholders  Basis: unified information system of implementation, monitoring and evaluation data  More hierarchies  Own data collection for evaluation indicators  Yearly reports on system performance and progress in terms of reaching targets

9 Evaluation of operations  Operations highlighted in the regular reports  Best and worst performing operations  Operations where at least one member of the Monitoring Comittee is interested in performance, results and/or specific problems Elements of the system _____________________________________________________________________________________________________________________________

10 Program evaluations  Ex-ante and ex-post evaluation of 2-year Action Plans  Ex-post evaluation of the I. NDP and its OP’s  Mid-term evaluation of the NSRF and OP’s  Ex-post evaluation of the NSRF and its OP’s Elements of the system _____________________________________________________________________________________________________________________________

11 Institutional framework of evaluation in the period _________________________________________________________________________________________________________________________________________ Elementary changes: Centralized strategic planning and programming Planning and implementation in one agency Centralized implementation: agency and MA’s in the same agency Strong and direct coordination  real demand for good evaluations  opportunity to good coordination of evaluations

12 Experiences and conclusions ______________________________________________________________________________________________________________________________ Bottom-up evolution of evaluation culture is essential, but not enough Evaluations will be carried out, main goal is to desing a system that assures quality and use of evaluations Self-sustaining system must be built Well defined users ensure use of evaluation results Continuous evolution of development policy making => In lack of previous evaluations the system must generate data and information from itself

13 Thank you for your attention! Sándor Csengődi National Development Office – Hungary