1. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 1 The Monitoring- and Evaluation System of the Austrian RDP 07-13 Karl M. Ortner (AWI) Otto Hofer (BMLFUW)

2. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 2 Tasks of programme leader  Implementation of monitoring and evaluation system  Setting up and consulting the monitoring committee  Monitoring the quality of programme implementation  Provision of the data base  Annual progress reports  Consulting with EU-Commission  Administration of the evaluation  Contracting evaluation projects  Organization of meetings and networking of evaluators  Contracting supplementary projects (in cooperation with evaluators)  Editing and publishing evaluation reports

3. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 3 European Commission Programme leader (ministry) Evaluators Information flows Paying agency

4. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 4 Organisation of the evaluation in A Evaluation coordinator Project leader Project assistence Karl Ortner Evaluators Axis 2 Evaluators Axis 3 Evaluators Leader Evaluators Axis 1 Representatives Axis 1 Representatives Axis 2 Representatives Axis 3 Representatives Leader ÖPUL committee Monitoring committee Programme leader

5. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 5 Evaluators of Austrian RDP 07-13 20 evaluators from 6 organisations Projects -> various institutes Abt. Immissons- u. Klimaschutz 11 measure representatives

6. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 6 Information flow in the evaluation project Evaluation projects Measure representatives * Evaluation unit in the ministry Evaluation coordinator Research and other projects Project leader* Evaluators Project reports Paying agency

7. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 7 Tasks of project leader in respect of the ongoing evaluation Provision of data -Applications and payments data (from PA) Input indicators (payments) Output indicators (investments, areas, etc.) -Baseline indicators Official statistics -Tools for data analysis (RDBs, ACCESS, etc.) Commissioning projects to improve implementation and evaluation (research, surveys, handbooks, etc.)

8. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 8 Decentral data collection online-input into the system of the paying agency at the district offices of the chambers of agriculture etc. of  applications and  payment requests  evaluation data

9. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 9 Online application form: Modernisation

10. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 10 Mn Evaluation form Payment forms Other databases Application form Structure of databases ASS FADN...IACS Mn

11. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 11 Annual coordination meetings Purpose: Clarification of tasks and needs Points of discussion (2009) Report on EU developments, evaluation helpdesk  Report on Rural Development Network  Report on Programme implementation Support by evaluation project leader Evaluation studies Evaluation homepage State of databases  Report by Paying Agency State of databases, evaluation forms Statements by the evaluators and measure representatives Evaluation activities, problems, requirements Statement of the evaluation coordinator Achievements, Tasks in the ongoing year, timetable Monitoring Evaluation

12. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 12 Evaluation concepts Evaluation concepts by measure how to evaluate the quality, effectiveness and efficiency of the measure? what impact and additional indicators ? what data are required, how obtained ? what analyses, studies are required / planned ? by what time? Estimation of effects what will be the reference situation (without support), how determined / estimated / assumed ? how will indirect effects be estimated / assumed ? how will payments / projects be compared in order to assess their relative efficiency / best practice ?

13. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 13 Problem areas  Monitoring requirements  Definition of terms  Definition of effects (static or dynamic, project / sector / macro level, time horizon)  Distribution of effects  Reliability of data  Sophistication of estimation methods  Costs and reliability of results  Comparability of results across measures / regions  Aggregation of results, consistency  Conclusions / recommendations

14. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 14 Need for exchanges  European evaluation network newsletter  Informal meetings of evaluators of particular measures  International workshops of evaluators of particular measures  Workshops of evaluators with common languages -f. i. German „Länder“ and Austria  Seminars of evaluation societies across measures and structural programmes (DeGEval)  Training courses (f.i. EasyECO, European Academy)  European workshops?

15. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 15 Success factors of the evaluation  evaluation by measure  clear focus, specific methodology  respected specialised research institutes -continuity, flexibility, know-how, infrastructure -motivated evaluators, research background  strong support by -implementing authorities -project leader -research projects -databases and tools  demand by EC, administration, implementing authorities, monitoring committee, research community, general public

16. Evaluation Experts Meeting, DG AGRI L4, Brüssel, 23.06.2009 16 Further information www.gruenerbericht.at