Presentation on theme: "2_Monitoring and Evaluation of CAP 2014 – 2020 Approach of PII"— Presentation transcript:
1 2_Monitoring and Evaluation of CAP 2014 – 2020 Approach of PII Jela Tvrdonova 2015
2 ContentsWhy do we need a common approach to M&E of common agriculture policy?Who is responsible?How does the policy and the legal framework look like?How is the M&E system built up and what guidance is provided?Evaluation questionsIndicatorsEvaluation approachWhat lessons can be drawn from the assessment of impacts in rural development of the current programming period?
3 Need for coordination and collaboration Who is responsible?One policy = one evaluation→ CAP common impactsPillar IEuropeanCommissionPillar IIMemberStatesWhen?Who?How?Need for coordination and collaborationChallenges of the CMEF & Ongoing Evaluation4/5 June 2009
4 Monitoring & Evaluation For the CMEF (Common Monitoring and Evaluation Framework) relates only to rural development.For there will be one monitoring and evaluation framemork for the whole CAP (Art. 110 of CAP Horizontal Regulation proposal).Important components will be (among others) indicators and common EQ
5 Legal framework for the evaluation RDR, Art.75: Monitoring and evaluation system aims at:Improving the quality of the design and the implementation programmes;Demonstrating the progress and achievements of rural development policy;Assessing the impact, effectiveness, efficiency and relevance of the RD policy interventions.Objectives of monitoring and evaluation of the CAP
6 Technical Handbook of the M&E framework of the CAP Provides information on:Objectives and purpose of M&E,Framework for M&E of the CAP (intervention logic and indicators),Actors and responsibilities in M&E (incl. Expert group of M&E the CAP),Data sources,Rural development specificities for monitoring (MC, AIR, PF) and evaluation (EP, ex ante, CEQ, ex post), including links to guidance documents already published,Use of M&E information,Future development,Annexes: pillar I and II indicators fiches.
7 CMES as part of CMEFCAP Common Monitoring and Evaluation Framework (1306/2013, Art. 110)Common Monitoring and Evaluation System(1305/2013, Art. 67 and 808/2014. Art.14)Pillar I specific elementsCommon elementsPillar II specific elementsCommon contextCommon context indicatorsCommon evaluationquestionsRD priorities/specificobjectivesPillar II measuresPillar I specific objectivesPillar I instrumentsIntervention logic:Overall CAP policy objectivesCommon impactsPillar II result /targetindicatorsPillar II output indicatorsPerformance frameworkCommon impact indicatorsPillar I result indicatorsPillar I output indicatorsData sourcesOperation databaseResponsibilityfor evaluation:DG AgriResponsibilityfor evaluation:MAs
8 Elements of the CMES (Reg.808/2014, Art. 14.1) Intervention logic showing interactions between priorities, focus areas and measures,Common context (including RD related impact indicators), result/target, output indicators a performance review indicators,Common evaluation questions,Fiches for common context, impact, result/target and outputindicatorGuidance on use and establishment of proxy indicatorsGuidance on the Indicator PlanGuidance on performance review and reserveCEQ related to:→focus areas of RD priorities→other aspect of RDP (RDP synergies and TA)→Union level objectives
9 Elements of the CMES (Reg.808/2014, Art. 14.1) Data collection, storage and transmissionReporting on monitoring and evaluation activitiesEvaluation planEx ante, AIR and 2019 and ex post evaluationSupport to actors in M&E to fulfil their obligationsCommon data sources: FADN, Eurostat, others(updated tables for CCI), II, RI.Operations database and electronic transmission ofmonitoring data (OI)Guidance on monitoring covering elements includedin the AIRGuidance on the Evaluation Plan preparationand implementationGuidance on the ex ante evaluation of RDPOther guidance documents
10 Intervention logicStarting point for evaluation (ex ante, evaluation plan for evaluation during the programming period, ex post)Composed of hierarchy of objectives (CAP, RD priority, focus areas, measures, operations)Linked to financial allocations (inputs)Linked to hierarchy of indicators
13 1. Fostering knowledge transfer and innovation in agriculture, forestry and rural areas 1A Fostering innovation, cooperation, and the development of the knowledge base in rural areas1B Strengthening the links between agriculture, food production and forestry and research and innovation, including for the purpose of improved environmental management and performance1C Fostering lifelong learning and vocational training in the agricultural and forestry sectorsRD Focus areasRelevant measuresArt. 16 Advisory services, farm management and farm relief servicesArt. 15 Knowledge transfer and information actionsArt. 36 Co-operation13
14 RDP Intervention Logic An Intervention logic will be drawn up for each RD priority showing their contributions to selected Focus Areas (including potential contribution of particular measure to several focus areas).A basic intervention logic is proposed by the EC covering the most commonly expected combinations ( see following slides)MAs have the flexibility to develop a specific intervention logic appropriate to their territory and its needs.
15 Ex ante evaluation (Art. 8.1 (a) and Art.77) Evaluation plan (Art. 8.1 (a) )Common evaluation questions
16 Evaluation Questions (Pillar II) Common Evaluation Questions: Horizontal and focus area evaluation questions:define the focus of evaluations in relation to policy objectives, and;help to demonstrate the progress, impact, achievements, effectiveness, efficiency and relevance of rural development policy.Programme-specific evaluation questionsPolicy objectivesEvaluation questionsIndicators
17 Common evaluation questions – 30 Focus area related – 18 questionsOther aspects of RDP:Operational performance - 1Technical assistance - 1National rural networks - 1Horizontal evaluation questionsEUCAP objectives - 4Challenges of the CMEF & Ongoing Evaluation4/5 June 2009
18 “Horizontal” Evaluation Questions Linked to the overall policy objectives and other aspects of RDPs in order to demonstrate the achievementsEU 2020 objectivesCAP objectivesRD cross-cutting priorities (environment, CC, innovation)National Rural NetworksTechnical AssistanceOperational performance (synergies)
19 “Horizontal” Evaluation Questions Capture the contribution of the programme towards the overall policy objectives in terms of impacts.Answered with the means of common impact indicators, common context indicators and complementary result indicators (and additional information when necessary)Reporting in the AIR in 2019 and in the ex post evaluation.
20 Horizontal Evaluation Questions Example CAP objectiveFostering the competitiveness of agricultureTo what extent has the RDP contributed to the CAP objective of fostering the competitiveness of agriculture?JUDGEMENT CRITERIAThe agricultural entrepreneurial income has increasedThe agricultural factor income has increasedAgricultural productivity has increasedCOMMON RD INDICATORAgricultural entrepreneurial incomeAgricultural factor incomeAgricultural productivity
21 FA-related Evaluation Questions Linked to the objectives of the Focus Areas in order to demonstrate the achievements towards the policy objectivesCapture the contribution of the interventions under each FA in terms of programme resultsAnswered with the means of result indicators (and additional information when necessary)Reporting in the AIRs in 2017 and 2019 and in the ex post evaluation
22 FA-related Evaluation Questions Example Focus Area 4c: Preventing soil erosion and improving soil managementTo what extent have RDP interventions supported the prevention of soil erosion and improvement of soil management?JUDGEMENT CRITERIASoil erosion has been preventedSoil management has improvedCOMMON RD INDICATOR% of Agricultural land under management contracts to improve soil management (Result indicator)% of forestry land under management contracts to improve soil management (Result indicator)ADDITIONAL INFORMATIONAdditional information on soil erosion of the land under management contracts
23 Common indicators Programme-specific indicators (RD) Impact indicators (16) refer to benefits bothat the level of measure or RDP/policyResult indicators – PI (15)and PII complementary results (6)measure direct and immediate effects of RDP/policyOutput indicators – PI (22) and PII (27) measureactivities implemented within RDP/policyFinancial (input) indicators refer to resourcesallocated to measuresCommon context indicators (45) refer toRDP context and include impact indicators
24 Context indicatorsThe Common Context Indicators (+ common impact indicators) of : compulsory minimal set for the SWOT analysis, the RDP strategy and assessment of impacts45 common context indicators13 of 16 common impact indicatorsSocio economic; Sectorial; EnvironmentalUsed in territorial description and SWOT, evaluationsIncluded in RDP via structured table (all required!) - the use of the common context indicators will be taken into account for the approval of RDPs by the ECBase on the availble data, sources from EUROSTAT and other EU-level data sources, OR from national/regional sources. (Proxies may be used at regional level)
25 Context (Pillar II) and impact indicators EU price variabilityAgriculture trade balanceConsumer price evolutionof food productsChallenges of the CMEF & Ongoing Evaluation4/5 June 2009
26 Impact indicators Cover both Pillar 1 and Pillar 2 Some are more relevant for Pillar 1 (e.g. trade related)Some are more relevant for Pillar 2 (e.g. territorial development)As far as possible existing datasets (EUROSTAT, Farm Structure Survey, FADN etc.) available at EU, national and/or regional level will be used for quantification.
27 Results indicators 25 result indicators Show direct achievements of policyLinked to Focus Areas of P2-5Mix of:targets (19) - monitoringcomplementary result indicators (6) - evaluationRequirement to flag operations contributing to Focus Areas with complementary result indicatorsChallenges of the CMEF & Ongoing Evaluation4/5 June 2009
28 Target indicatorsAt least one quantifiable target indicator is required for each Focus Area.Mostly based on result indicators (19),Some are closer to output indicators – monitoring (e.g. Priority 1).Show direct achievements of policyLinked to Focus Areas of P2-5The target indicators will be reported on annually in the AIRs using:Direct monitoring data,Estimates based on coefficients supplied in the guidance (e.g. to estimate the production of renewable energy from new investments).In few cases the calculation of the target indicators will be conducted by the evaluator when assessing the achievements of RDP (e.g. water and energy savings)
29 Output indicators 26 indicators at measure level: Output indicators - for all measures (total public expenditures)Output indicators linked to several measures (area under…)Output indicators linked to specific measures (support for Leader start up)Only a selection in the Indicator plan (Planned Outputs to be quantified)New indicators for NRN, Risk Management….
30 Programme Specific Indicators (Article 50(2) 1303/2013; draft RD IA Annex I Part 1 Point 4(a)(i))Design in SMART way,Shall:provide additional info specific to RDP territory e.g.Definition of rural areaForestry sectordescribe issues where common data lacking e.g.InnovationShort supply chainsLocal marketssupport and justify particular interventions e.g.Thematic sub-programmesProgramme specific Focus Areas
32 “Standard” AIR – every year from 2016 Reporting“Standard” AIR – every year from 2016Enhanced AIR 2017Enhanced AIR 2019
33 “Standard” AIR – every year from 2016 (Art /2013 and /2013)Key information on implementation of the programme and its priorities (Financial and monitoring)Issues which affect the performance of the programme and the measures takenSteps taken to implement technical assistance and programme publicity requirementsActions taken to fulfil ex ante conditionalities (in 2017 and in where relevant)+ Annex template on implementation of the financial instruments
34 Content of "standard" AIR (reporting on evaluation activities) EP modificationsEvaluation activities undertakenActivities related to provision and management of dataList of completed evaluationsSummary of completed evaluationsCommunication activitiesFollow-up of evaluation results
35 Enhanced AIR 2017 Information from evaluation activities on: Reporting and quantification of programme achievements, in particular through assessment of the complementary result indicators, and relevant evaluation questions.Description of implementation of sub-programmesImplementation of actions to take into account the principles set out in art 6, 7 and 8 CPR (Art. 50 CPR)a) Promotion of equality between men and women and non- discriminationb) Sustainable development (Art. 8 CPR)c) The role of the partners referred to in Article 5 CPR in the implementation of the programme and preparation of the progress report
36 Enhanced AIR 2019 Information from evaluation activities on: Reporting and quantification of programme achievements, in particular through assessment of the complementary result indicators, and relevant evaluation questions.Reporting on interim impact of the RDP: contribution to programme and EU strategy and objectives, in particular through assessment of the programme's net contribution to changes in CAP impact indicator values, and relevant evaluation questions.Progress made in ensuring integrated approach