Presentation is loading. Please wait.

Presentation is loading. Please wait.

122nd EAAE Seminar Approaches for assessing the impacts of the Rural Development Programmes in the context of multiple intervening factors Findings of.

Similar presentations

Presentation on theme: "122nd EAAE Seminar Approaches for assessing the impacts of the Rural Development Programmes in the context of multiple intervening factors Findings of."— Presentation transcript:

1 122nd EAAE Seminar Approaches for assessing the impacts of the Rural Development Programmes in the context of multiple intervening factors Findings of a Thematic Working Group Bernd SCHUH, Robert LUKESCH and J. Michalek, P. Kaufmann, A. Pufahl, S. Schiller, P. Koorberg, G. Beaufoy, G. Pinay, D. Moran, H. Gomann, D. Storti, P. Rossi, M.L. Paracchini European Evaluation Network for Rural Development

2 122nd EAAE Seminar 2 CONTENT OF THE PRESENTATION  Assessing impacts oThe process oKey issues oRecommendable methodologies  The seven common impact indicators: oEconomic growth oEmployment creation oLabour productivity oBiodiversity oWater quality oHNV oClimate change

3 122nd EAAE Seminar 3 Assessment of impacts of rural development programmes – the basics: CMEF – the „guiding principle“ Intervention logic Additional programme specific indicators Evaluation questions Contextbaseline SWOT StrategyObjectivesImpact Result Outputs InputsObjectiverelatedbaseline Hierarchy of Objectives Definition of measure

4 122nd EAAE Seminar 4 Process of assessing RDP impacts: Three phases: Gauging the evidence of change Identifying the drivers of change Understanding change and concluding on future interventions 231

5 122nd EAAE Seminar 5 Key issues to address  Assess the programme impacts against their counterfactual, i.e. calculating the changes that would have occurred without the specific programme intervention  Measure both the micro level and the macro level effects to disentangle the effects of single measures of the programme as a whole from effects of other intervening factors  Estimate the net effects of the programme by netting out deadweight, substitution and multiplier effects  Construct a data and information base which allows for the unbiased computation of the effects as stipulated above  Bridge the gap between indicator measurement and a judgement of the impact of the rural development programme  provide answers to evaluation questions

6 122nd EAAE Seminar 6 General methodological recommendations  The counterfactual assessment of impacts  Quasi-experimental design, Propensity Score Matching, DiD method  Possibilities to deal with apparent non-availability of control groups, the assessment of impacts of project-type interventions and other collateral methodologies  Micro and macro level: statistical, econometrical methods, CGE models, system dynamic modelling  Methods for netting out deadweight, leverage, displacement, substitution and multiplier effects (specifically for socio-economic impacts)  Additional indicators, especially for environmental impacts  Complementing quantitative with qualitative methods, specifically for socio-economic impacts: interviews, focus groups, case studies, process monitoring  Measuring indicator values is just one step towards the main issue: answering the evaluation questions

7 122nd EAAE Seminar  The three common impact indicators are interrelated and require applying a coherent methodological approach  Challenges in common to the 3 indicators: –Determining true causation –Elimination of a selection bias at micro- and macro-levels –Aggregation of various direct and indirect programme effects –Looking into the “black box” and identifying cause-effect relationships –Gathering additional information and arguments to answer to the evaluation questions –Based on the presumed patterns of change: provide policy recommendations 7 The Socio-economic impact indicators

8 122nd EAAE Seminar Economic Growth (NAGVA-PPS) Definition & Measurement: CMEF: impact of a RD programme on economic growth is to be measured in terms of the Net Additional Gross Value Added in purchasing power standard: NAGVA-PPS. Specifics: The indicator on economic growth should not be taken as a proxy for sectoral competitiveness, rural diversification or quality of life. To make conclusions of this kind, it requires looking at a set of additional indicators at aggregated level (e.g. output shares) or indices (e.g. rural development index). 8 The Socio-economic impact indicators

9 122nd EAAE Seminar Employment creation Definition & Measurement: CMEF: Measure employment effects in Full Time Equivalent (FTE) jobs created. Caveats for interpreting the outcome: e.g. - time lag until an investment brings forth lasting employment; - missing critical mass (especially for non-agricultural beneficiaries). Employment effects should be interpreted in a common context. For instance, rising total factor productivity (labour, capital, land) may explain why jobs have been lost in the agricultural sector. Specifics: Mix of qualitative and quantitative methods to be applied to cover possible effects on employment outside of agriculture. Of interest: Not only the magnitude of effects, but also how RD policies affect individuals, communities and regions.. 9 The Socio-economic impact indicators

10 122nd EAAE Seminar Labour productivity Definition & Measurement: CMEF: change in Gross Value Added per Full Time Equivalent (GVA/FTE). Specifics: Indicator is intra-sectoral and does therefore not express the competitiveness of one sector against another. Does not allow side effects to be taken into consideration: e.g. if funding is provided to companies whose improved performance makes no direct contribution to rural development. To overcome the limitations of GVA/FTE, competitiveness of the agricultural sector can be measured in alternative ways: e.g. Competitive Performance or Revealed Comparative Advantage (RCA). 10 The Socio-economic impact indicators

11 122nd EAAE Seminar 11 The environmental impact indicators – specific issues Data availability. System boundaries –Environmental impacts originate from multiple Measures. –Conception of the environment in the evaluation framework (ecosystem functions vs. ecosystem services). Difficult to depict the full range of effects in complex fields of environmental phenomena like ‘climate change’ or ‘biodiversity loss’. “Evaluation” vs. assessment: aggregation methods not easily applicable. Cumulative impacts: crosswise effects between environmental impacts. The environmental impact indicators

12 122nd EAAE Seminar Biodiversity Definition & Measurement: CMEF: change in trend (biodiversity decline) in the area targeted by the intervention. Farmland Bird Index (FBI). Specifics: Where this is appropriate, use alternative composition of bird species and different reference year. FBI can be complemented by other indicators: e.g. population trends of agriculture related butterfly species, or trends in important bird areas (IBAs) considered as threatened. Appropriate monitoring scheme indispensable. A variety of sources of information will have to be taken into account to understand what is going on in the area-specific context. 12 The environmental impact indicators

13 122nd EAAE Seminar 13 The environmental impact indicators High Nature Value farming / forestry I Definition & Measurement: CMEF: changes (UAA ha) in High Nature Value farmland and forestry. HNV farmland: characterised by particular land cover types and patterns. presence of populations of particular wildlife species. Refers to both, the land cover (farmland or forest) and the way it is managed for production by a particular farming system. Specifics: Evaluation of intended and unintended influences of RD measures on farmers’ decisions the extent of participation the coincidence of participation...with the observed changes and the distinction of programme-induced changes from those induced by other factors (climate, commodity prices, etc.).

14 122nd EAAE Seminar High Nature Value farming/ forestry II Data: HNV criteria (farming or forestry practices) measured in a points-based system to allocate HNV score for a given unit of land. The baseline (number of HNV hectares) has not been sufficiently established  complement the quantified estimate with qualitative assessment. Sample surveys of areas with a concentration of HNV farming and forestry. Only investment in appropriate data collection and monitoring schemes will allow a full evaluation of the effects of rural development programmes on HNV farming and forestry. 14 The environmental impact indicators

15 122nd EAAE Seminar Water quality 15 The environmental impact indicators Definition & Measurement: CMEF: changes in gross nutrient balance (GNB) attributable to the intervention. GNB indicates potential nutrient losses to the water bodies likely to be detrimental for the quality of water. Specifics: Farm = unit of analysis. Several methods have been developed for assessing a farm nutrient budget: –based on an aggregate of individual fields –analysis of the farm as a whole  more recommendable since it takes into account transfers of matter between fields and farming practices. Difference in differences (DiD): Method to determine the impact of RD measures on the change in GNB. Macro level analysis = farming region. Several models (e.g. CAPRI, RAUMIS) estimate soil gross or net nutrient balance.

16 122nd EAAE Seminar Climate change 16 Definition & Measurement: CMEF: quantitative and qualitative change in the production of renewable energy. I.e. reduction of net greenhouse gas emissions (i.e. carbon dioxide) attributable to the substitution of fossil fuels by non fossil alternatives. Specifics: For fuel crop areas: –bottom-up (based on qualitative surveys of a cross section sample of recipients), or –top-down (based on representative modelling of a range of farm types using linear or dynamic programming methods). Measure level: Outcomes of climate change, water quality and HNV indicators need to be considered together for a net assessment of combined impacts.  e.g. cross impacts with water quality. To assess impacts at the programme level, ALL measures (i.e. also from Axes 1 and 3) have to be considered. The environmental impact indicators

17 122nd EAAE Seminar Additional Impact indicators as used by the EU Member States 718 additional specific impact indicators, identified by the MS. However only about a third of the additional indicators listed (i.e. 210) were real additional impact indicators in the MS. Economic indicators: e.g. Number of participants indicating the measure had a financially positive effect on their farm business; Number of new businesses which are still in existence two years after final funding; Number of supported new businesses which are still in existence two years after final funding. (Northern Ireland) “human capital” (measured by: promotion of competences development) (Niedersachsen and Bremen) Environmental indicators: e.g. changes in levels of phosphorus and pesticides to ammonium. Italian national water quality index (Molise) sequestration of greenhouse gases (GHG) by natural sinks (UK-England) 17

18 122nd EAAE Seminar Additional Impact indicators as used by the EU Member States II 18 ProgrammeAdditional IndicatorMeasurement Thematic Field HessenLife qualityattractive life environment (life quality)quality of life Mecklenburg VorpommernStabilize the population numberno measurement provideddemography Niedersachsen and BremenLifequality and governanceliving milieu and quality, social life, local identity: data collection difficult, mostly qualitative data collected (via public consultations) quality of life Niedersachsen and BremenLifequality and governancegovernance - improvement of regional competenciesquality of life Niedersachsen and BremenLifequality and governancegovernance - planned and implemented plans/proposals quality of life Rheinland-PfalzAttractive living environmentno measurement providedquality of life SaarlandPopulation trends: indicator to assess the prevention of the migration of population no measurement provideddemography SachsenImpacts on safety and recreation function of the forest (total, including private forest, including forest of the public sector) no measurement providedquality of life SachsenImplementation rate of regional conceptsno measurement providedLEADER Schleswig-HolsteinImprovement of living qualitylater collected based on consultation of concerned population quality of life BayernNumber of persons who benefit directly of the flood protection no measurement providedenvironment Bayernincrease of life qualitythe effects are to be investigated within special case studies quality of life Castilla y LeónNew LAGsno measurement providedLEADER Pais VascoStudy of quality of lifeno measurement providedquality of life Cantabria% age reduction of the farmer measurement provideddemography Corsica (but national priorities) Generation renewalNumber of farmers under 35 yers old related to the number of farmers over 55 demography Marchepopulation dynamicsresident population interested by programdemography Scotlandimprovement in community capacityno measurement providedLEADER

Download ppt "122nd EAAE Seminar Approaches for assessing the impacts of the Rural Development Programmes in the context of multiple intervening factors Findings of."

Similar presentations

Ads by Google