Monitoring Performance and Impact of Sustainable Land Management projects in Ethiopia World Bank Land and Poverty Conference March 2018 Daniel Ali, Klaus Deininger, Daniel Monchuk The World Bank

This study… Combines data from different sources to assess the effectiveness of soil and water conservation interventions in Ethiopia (under the Tana-Beles Integrated Water Resources Development (TBIWRD) Project 2009-15) Project M&E data on activities and outcomes Remote sensing data extracted from medium resolution satellite imagery

5 critical watersheds - 85,700 ha, 163 micro-watersheds – 526ha average, natural resource management investments

Main Activities/Interventions of the NRM Component of the Project Reported in M&E Output 1.2 - Soil and water conservation measures undertaken on cultivated lands Output 1.4 - Degraded land (hillside, grazing and forestry land) treated Output 1.6 - New area planted by community forestry and agro-forestry systems to stabilize landscape and produce fuel wood and timber

Micro-watershed average treated area (ha) by output activity 5 critical watersheds 85,700 ha 163 micro-watersheds with average area of 526 ha

Can NDVI (using LS7) pick-up meaningful change? areas in the 'y' as having been part of a tree planting activity

Processing imagery?? The old way… The new way… Download 11 years worth of LS7 imagery -> ~2 MONTHS from World Bank country office Batch routines Filter out clouds, shadows, etc. Merging raster files The new way… Google Earth Engine developer platform -> some javascript -> sit back, relax… -> 22 images/ year (16 day pass), 4 images span area of interest, 11 years, 250 MB, 4GB/day from country office

Three main seasons Daily rainfall pattern in the 5 hydro-stations located in the TBIWRD study areas Dry, pre-rainy and rainy seasons Seasonal vegetation indices are thus computed for analysis October 1-Jan 31: dry season; Feb1 – May 31: pre-rainy season; June-Sep: rainy season

Data for analysis ~30m pixel resolution Control – 5km buffer ~3 million pixels -> ~99 million obs in panel (11 years, 3 seasons/year) Remote sensed data: seasonal average of NDVI, SAVI, and LSWI are computed using L7 imagery Rainfall - Climate Hazards Group InfraRed Precipitation with Station data (CHIRPS) which incorporates 0.05° (~5km) resolution satellite imagery with ground-based weather station data to create gridded rainfall time series

Imagery-based outcome variables (No dense forest)   NDVI SAVI LSWI Control Treatment Dry season (Bega) 2005-09 0.384 0.388 0.226 0.110 0.120 2009-12 0.387 0.396 0.231 0.234 0.106 0.118 2012-16 0.363 0.376 0.211 0.216 0.078 0.094 Pre-rainy season (Belg) 0.206 0.214 0.123 0.126 -0.015 -0.004 0.202 0.121 0.124 -0.019 -0.007 0.235 0.249 0.141 0.147 -0.009 0.006 Rainy season (Kiremt) 0.476 0.489 0.297 0.305 0.232 0.253 0.529 0.532 0.330 0.331 0.263 0.273 0.464 0.477 0.291 0.299 0.223 0.239 NDVI (normalized difference vegetation index) ranges between -1 and 1: close to -1, water; around zero (-.1 to .1), barren land; values .2 to .5 represent crops, shrubs, and grass land, above 0.5 dense forest SAVI (soil adjusted vegetation index): adjusts for soil brightness when the vegetation cover is low…The lower the value, the lower the amount/cover of green vegetation LSWI (Land surface water index): ranges between -1 and 1: helps to monitor evapotranspiration and plant and soil water content – positive values for green vegetation and negative values for dried (brown) leaves

M&E: Daily water flow and sediment load at hydro station level Enkulal G/Mechaw Tikur wu Toma Flow in m3/s 2010 0.101 0.530 0.305 0.353 2011 0.089 0.380 0.071 0.425 2012 0.098 0.141 0.052 0.341 2013 0.082 0.234 0.064 0.460 2014 0.135 0.311 0.147 0.765 2015 0.104 0.201 0.240 0.749 Mean 0.102 0.300 0.516 Daily sediment load in t/d 1.731 3.460 2.942 3.104 0.408 1.858 0.757 2.210 0.677 0.728 0.476 1.646 0.619 0.930 0.294 2.049 0.632 1.079 0.390 3.490 0.563 0.852 1.368 3.416 0.772 1.485 1.038 2.653 - Stream flow or discharge: the volume of water that moves through a specific point in a stream during a given period of time Steep terrain: less time to soak into the ground and runoff will be greater; if level train or dense vegetation: plenty of time to soak into the ground Sediment load: the amount of solid load that a stream an carry – measured in metric ton per day passing a given location

Treatment Indicator Used M&E data to construct three treatment variables separately for cultivated land, degraded land, and new area planted as: the ratio of (cumulative) area of land treated as a share of total micro-watershed.

Impact of micro-watershed treatment on vegetation cover (95% confidence interval)

Impact of micro-watershed treatment on stream flows (95% confidence interval)

Impact of micro-watershed treatments on sediment load (95% confidence interval)

Summary Ex-post evaluation possible “Easy” to process imagery and scalable (if good intervention data available) Caveats: Lack of data on the cost of specific interventions to value the impact in economic terms  M&E Data Lack of information on the location or timing of interventions makes it difficult to assess impact say by slope or pre-project levels of land degradation  CAPI helps