Development Research Group Living Standards Measurement Study – Integrated Surveys on Agriculture: Main Features, Challenges and Next Steps Gero Carletto Development Research Group The World Bank February 27th, 2012

Outline The Living Standards Measurement Study The LSMS-ISA project Main features Progress to date Methodological validation/research Challenges Next steps

The LSMS Flagship initiative in DECRG since 1980 Evolution … Poverty monitoring and measurement: the “McNamara anecdote” Technical assistance, capacity building Back to the “roots”: “wholesale” research Focus on agriculture: LSMS-ISA

The LSMS-ISA project Started in early 2009 Tanzania pilot in mid-2008 Funding from the Bill and Melinda Gates Foundation, with additional funding from USAID, DFID and a number of other sources, including governments Working on four fronts: Household survey data collection Methodological validation/research/tool development Capacity Building Dissemination

The team Gero Carletto Luc Christiaensen (from Nov 2011) Kinnon Scott (on DAIS until Sept. 2012) Kathleen Beegle (WDR until June 2012) Diane Steele (LSMS Database coordinator) Talip Kilic Kristen Himelein (sampling) Mimi Oseni (50% with AFTAR) Alberto Zezza (50% with LDIA project) Siobhan Murray (GIS; part-time) Raka Banerjee (ETC; project coordinator) Jon Kastelic (formerly in Malawi; data processing) Prospere Backiny-Yetna (in Mali; formerly in Niger) Bjorn Campenhout (in Uganda, 50% with IFPRI) Colin Williams (formerly in Nigeria, now on STC)

Main Features 6+ year program (2009-2015) Panel household surveys with emphasis on agriculture in 7 Sub-Saharan African countries Sample: 3-5,000 households Population-based frame (Global Strategy) Representative at national- and few sub-national levels Tracking Movers Subsample of individual split-offs

Main Features (cont’d) Multi-faceted “integration” Multi-topic survey instrument Farm plus nonfarm, consumption, nutrition, inter alia Build on existing/planned surveys country ownership Sustainability … but large trade-offs Link to major initiatives National Strategy for the Development of Statistics (NSDS) Global Strategy to Improve Agricultural and Rural Statistics Improved links to other data sources GIS, censuses, surveys, etc.

Main Features (cont’d) Gender-disaggregated data Use of technology GPS for households and plots (area) Concurrent field-based data entry Computer Assisted Personal Interviews (CAPI) Open data access policy Micro-data publicly available within 12 months of data collection GPS data dissemination

Main Features (cont’d) Inter-institutional partnerships In-country Government (NSO, MoA, other line ministries) WB country offices Other development partners Thematic GPS and crop production: FAO (GS Action Plan) Livestock: WB-ARD/ILRI/FAO Food Security: WFP Empowerment: IFAD Agricultural Income: FAO/RIGA Climate Change: WB/ENV Fishery: WorldFish Centre Analysis: IFPRI/Harvest Choice, IFAD CGIAR: technology adoption

Surveys Tanzania National Panel Survey Uganda National Panel Survey Malawi Integrated Panel Household Survey Nigeria General Household Survey Panel: Niger Enquête National sur les Conditions de Vie Des Ménages Ethiopia Rural Socio-Economic Survey Mali Integrated Agricultural Survey

Survey Schedule Tanzania Uganda Malawi Nigeria Mali Ethiopia Niger 2008/09 2010/11 2012/13 Uganda 2009/10 2011/12 2013/14 Malawi Nigeria Mali 2014/15 Ethiopia Niger

Data release Tanzania Uganda Malawi Nigeria Niger Ethiopia Mali 2010 2011 2012 2013 2014 2015 Tanzania   X     X   X Uganda   X X   X Malawi   X Nigeria   O   O Niger   X X Ethiopia  X Mali   X   X

Project outputs Documented public-access microdata Sourcebooks, best practice guidelines Tools Computer Assisted Personal Interviews (CAPI) software ADePT-Agriculture and ADePT-Livestock Comparative Living Standards Project (CLSP) Research and analysis for policy and programming Methodological validation Research pillar of Action Plan of Global Strategy DFID grant

Experiments on Survey Methodology Identification process Alignment with Global Strategy Field experience Consultation Peer review Research topics Land area measurement Soil fertility Water resources Labor input in agriculture Continuous/extended harvest crops Non-Standard Units Livestock Post-harvest losses

Land Area Measurement Why is it important? Fundamental component of agricultural statistics Priority #1 of Global Strategy Current methods inaccurate or impractical What are the different methods available? Traversing (compass and rope method) P2/A Self-reporting GPS

Impact of GPS on IR Old controversy in development economics, with recent reprise Friends: Barrett (1996), Benjamin and Brandt (2002), … Binswanger, Pingali, … Foes: Bhalla and Roy (1988); Benjamin (1995); Collier and Dercon (2009), …\ Binswanger et al (1995); Eastwood et al. (2010). Possible explanations Factor market imperfections Omitted variables (land quality) Measurement errors Low correlation bet/w measurements (Udry and Goldstein, 1999) Recent findings: Lamb (2003) Factor market imperfections and land quality differences Barrett et al (2010) Lab soil test. Marginal impact of land quality. Measurement error? De Groote and Traore (2005) Systematic bias in reporting (small over-report) Objective: Test robustness of IR to land measurement error

Systematic bias in reporting

Systematic bias in reporting Acres Land Deciles (GPS)

Yields and farmsize

The models Plot level: Discrepancy in measurement HH head characteristics Plot size, squared Control for rounding Involvement in disputes, Slope, PSU dummies Household level: Farm profits/acre Farm size HH char’s (age, edu, gender) Inputs: Family labor (interacted with land) Hired labor Variable inputs Land quality (soil, slope, irrigation) PSU dummies

Main findings Household level: IR strengthened by GPS measurement Plot level: Small farmers over-report Household level: IR strengthened by GPS measurement Area Self-Reported Area GPS Log Land Size -0.62*** -0.83*** [0.000]

Smallholders are more efficient however you measure it

Experiments on Survey Methodology Research topics Land area measurement Soil fertility Water resources Labor input in agriculture Continuous/extended harvest crops Non-Standard Units Livestock Post-harvest losses

Soil Fertility Why is it important? Fundamental driver of productivity in Africa – remains a key unobserved variable for analysis What are the different methods available? Conventional soil analysis (CSA) Farmers’ subjective evaluations Spectral soil analysis (SSA)

Water Resources Why is it important? Agriculture in Africa is predominantly rainfed – water is a key input to production Large discrepancies across data sources Low resolution/idiosyncratic risk Lack of access to data What are the different methods available? Satellite imagery Weather stations Self-reporting Community rain gauges

Labor Inputs Why is it important? Labor inputs are fundamental to labor productivity measurement Very poorly measured What are the different methods available? Recall (6 month or 12 month, by activity, by demographics) Computer-assisted telephone interviews (CATI) Labor input diaries

Experiments on Survey Methodology Research topics Land area measurement Soil fertility Water resources Labor input in agriculture Continuous/extended harvest crops Non-Standard Units Livestock Post-harvest losses

Extended Harvest Crops Why is it important? Continuously / extended harvest crops are major staple crops in many African countries Inaccuracy of recall May extend across seasons What are the different methods available? Recall (6 month or 12 month) Crop cards (with crop card monitors) Ongoing work with the Uganda Bureau of Statistics CATI (data collection / supervision)

Diary vs. recall (Uganda) Compare recall and diary methods for crop production estimates (and consumption from own production) in Uganda Lack of gold standard “well-administered” diary? Crop cutting? Does it vary by crop type? Extended-harvest crops Cassava Banana

Frequencies of crops reported Diary vs. recall (Uganda) (cont’d) Frequencies of crops reported Recall Diary % Coffee 31.9 21.7 Maize 76.8 73.8 Cassava 58.8 82.1 Tomatoes 3.4 22.2 Avocado 2.8 25.8

Consumption plus sales Diary vs. recall (Uganda) (cont’d) Output values (usd) Recall Diary Consumption plus sales Ratios (1) (2) (3) (2)/(1) (3)/(1) Cash crops 48.6 27.2 57.1 0.56 1.18 Food crops seasonal 135.4 297.4 164.5 2.20 1.21 Food crops continuous 178.1 275.6 270.3 1.55 1.52 Fruit & Vegetables 10.8 14.9 43.4 1.37 4.00 Total 372.9 615.1 535.3 1.65 1.44

Experiments on Survey Methodology Research topics Land area measurement Soil fertility Water resources Labor input in agriculture Continuous/extended harvest crops Non-Standard Units Livestock Post-harvest losses

Non-Standard Units

Weight in kgs of a 50 kg sack Non-standard units (cont’d) Weight in kgs of a 50 kg sack Maize 50.0 Cassava 41.7 Sweet potato 36.2 Irish potato 42.6 Groundnut 44.2 Ground bean 43.2 Rice 56.2 Finger millet 50.5 Sorghum 49.6 Pearl millet Bean 77.6 Soyabean 53.1 Pigeonpea 57.1

Non-standard units (cont’d) State of crop … Maize: A 50 kg size sac weighs only 29.0 kilograms when it is filled with fresh maize. This translates into 14.8 kgs of maize grains. Cassava: A 50 kg size sac weighs 37.8 kilograms when it is filled with dried cassava. To attain these 37.8 kgs of dried cassava, one would have had to start with 108 kgs of fresh cassava. Rice: A 50 kg size sac weighs 38.5 kgs when filled with rice which has not been husked. These 38.5 kgs translate into 24.3 kgs of grain when husked.

Land area by zone (Nigeria) Non-standard units (cont’d) Land area by zone (Nigeria) Zone Specific Conversion Factors into Hectares Zone Conversion Factor Heaps Ridges Stands 1 0.00012 0.00270 0.00006 2 0.00016 0.00400 3 0.00011 0.00494 0.00004 4 0.00019 0.00230 5 0.00021 0.00013 6 0.00001 0.00041

Challenges Integration comes with trade-offs Data preparation and data release Time consuming and not enough “control” Resident advisor in bet/w years Dissemination of GPS information protocol Sample Too small? Representative at crop level? Livestock? Linking to Ag census and large Ag surveys CAPI More sustainable solution Frequency of surveys Moving to every other year Tracking Still a challenge in some countries

Next steps More tools? More countries ? More analysis Mobile phones Ag input/labor use, climate data; Listening to Africa More countries ? in Africa: Burkina Faso beyond Africa? More analysis Gender and Agriculture Agriculture and Nutrition Myths and Facts in African Agriculture

http://www.worldbank.org/lsms-isa

The End

Is women’s control of income important for child nutrition? >> Dependent Variable: Z-Score of Height-for-Age Definitions of Woman’s Share of Household Income V1 Assumption 100 to Head V2 Assumption 50/50 Split V3 Assumption a la HH V4 Preferred Child: Male -0.130 -0.129 -0.147 -0.186** Woman's Share of Household Income x Male Child -0.735 -0.070 -0.008 0.155*** Observations 2,522 R2 0.711 0.710 note: *** p<0.01, ** p<0.05, * p<0.1

The tale of the missing plot measurements Concurrent Data Entry The tale of the missing plot measurements High initial rates of missing gps data in months 1 & 2 Percentage of Plots coming from the field without measurements over the 12 months of field work. Background on Malawi Plot Measurement protocol;

Concurrent Date Entry (cont’d) The tale of the missing plot measurements Intervention - High rate of missing data observed and new instructions to field disseminated. In Month 2 we find that the frequency of missing measurement is too high. And adjust the protocol for plot measurement.

Concurrent Data Entry >> The tale of the missing plot measurements Substantial decrease in missing data. Because of revisit of households in month 4-6, part of the missing data was now captured. Icing on the cake. Part of the households surveyed were going to again be surveyed three months later. Thus, under the new protocol originally missing plot measurement were captured, decreasing the overall percentage of missing plot measurements and contributing to more accurate yield calculations. At the end of the survey under 1 percent of plots were going unmeasured.

CAPI “Comparative Assessment of Software Programs for the Development of Computer-Assisted Personal Interview (CAPI) Applications”, with the University of Maryland IRIS Center Implementation in LSMS-ISA partner countries relying on readily available commercial software products Plans to create a public, freely available CAPI software package >>

GPS: What we are collecting Household location with current devices coordinate precision is within 5-10m (atmospheric interference, vegetation canopy, buildings.. may effect precision ) Just a reminder of the GPS data that we are collecting with the LSMS-ISA surveys. With mapping grade precision a particular dwelling is easily identified using survey GPS coordinates

GPS: What we are collecting Plot area Plot location Plot outline

Dissemination Strategy NOT proposing to disseminate actual household GPS coordinates ARE proposing to disseminate modified EA center-points, offset to prevent identification of communities AND a set of geovariables generated using the true locations true household GPS locations maintained in the Statistics Office, to be used for continuation of the panel or at the discretion of the NSO Acceptable risk of disclosure?

Dissemination: geovariables Having georeferenced hh and plot locations enables integration with other datasets, making available large range of additional variables. 3 types of geovariables we would like to disseminate with the survey dataset are: Distance – mapping showing distance to major market – darker areas more remote lots of geophysical characteristics that would be relevant, particularly looking at variations in ag productivity, this just a subset 3. Time series – as we mentioned yesterday in shock module, information on timing of rains and rainfall anomalies can be useful in interpreting production statistics and other survey data. Where time series are available we can use use rainfall or vegetation response measures to enhance augment what is capture in the survey instruments Distance HH to Plot HH to Market HH to Major Road Environmental Climatology Landscape typology Soil Elevation Terrain Time Series Rainfall Vegetation Indices >>