1 1 A Sustainable Poverty Monitoring System for Policy Decisions Bjørn K. G. Wold, Astrid Mathiassen and Geir Øvensen Division for Development Cooperation, Statistics Norway IAOS October, 2008

2 Two parts: Describing the Household Survey System Model, including the Poverty Monitoring System; Case Malawi Testing the Poverty Model; Case Uganda

3 3 Part 1: The Household Survey System Model

4 Growing need for statistics for policy decisions Recent initiatives since Millenium Development Goals –Paris21: ”Scaling up” –World Bank: ”Better statistics for better results” Three major challenges remain –Design short questionnaire for fast monitoring of MDG and PRSP indicators –Method for easy and accurate measurement of money-metric poverty –Household survey system with annual core for MDG and PRSP, and a rotating program of specialized sector surveys

5 Suggested solution 1. Identify indicators to measure progress on MDG and PRSP 2. Household Budget Survey for initial poverty line 3. A poverty model for monitoring in non-HBS years 4. Annual, rotating sector surveys with common core 5. Household Survey Program to cover all topics in 5-10 years 6. Statistical tools such as seasonal adjustment for consistent trends, and small areas estimation 7. Fast and easily accessible results 8. Active dialogue between donors and to ensure that all agencies accept integration of ”their” surveys in program

6 Poverty monitoring based on ”light” household surveys Exploit statistical correlation only, not total consumption Find poverty indicators in HBS (other than expenditure) Include exactly the same indicators in the “light survey” Collect light survey data and apply the correlation found in HBS Estimate annual regional/ district poverty headcount from the “light survey”, including the inaccuracy Standard error for estimate at similar levels as in traditional consumption aggregate approach

7 The Malawi Poverty Prediction Sequence 2004Timeline: Consumption aggregate Poverty line District Headcount Selection based on statistical correlation and theory 15 Indicators IHS Poverty line Headcount Consumption aggregate Est. HC 15 Indicators Model evaluation Precision? Same question for each indicator as used in HBS WMS0 5 Est. HC WMS07 Nacal Est. HC WMS08 Est. HC WMS06 Est. HC IHS2

8 Malawi: Large data gaps if use HBS only! 1998: IHS1 Budget Survey, (with data problems) 2004: IHS2 2009(?): IHS 3 ? ? ? ? ? ? ? ? ?

9 Malawi II: Complement with Poverty Estimates from “light” Surveys! 1998: IHS1 Budget Survey, with data problems 2004: IHS2 Estimates from (light) WMS 2005, 2006, (?): IHS 3 ? ? ? ? ? ? ? ? ? Malawi Rural Urban ? ? ?

10 Part 2: Testing the Poverty model on Ugandan Surveys

11 Testing the poverty models’ predictive ability Test the predictive ability of the poverty model  Compare model’s poverty estimates relative to poverty estimated directly from consumption aggregates Use 7 comparable household expenditure surveys from Uganda from 1993 to 2006 –Comparable consumption aggregates and sufficiently number of (exactly) identical indicators –Calculate urban/rural poverty models from each survey –Cross-testing models from each survey onto the other surveys

12 Example: Pairwise testing from 1995 survey onto itself, and the 6 other surveys Expenditure Survey 1995 Model 1995 Cons.agg 1993 Cons.agg 1994 Cons.agg 1995 Cons.agg 1997 Cons.agg 2002 Cons.agg 2005

13 Uganda ”baseline” trend; Using traditional consumption aggregate approach only 85% rural  national ~ rural Falling headcount ratio, especially in late 90-ies Urban – rural poverty gap closing

14 Uganda: Comparing actual poverty level predictions from RURAL model Models capture most, but not all of reduction All models have similar patterns of changes Less capture of variability within trend  Biases related to factors specific for specific years? (e.g, omitted variables)

15 Uganda: Comparing actual poverty level predictions from RURAL model Models capture most, but not all of reduction All models have similar patterns of changes Less capture of variability within trend  Biases related to factors specific for specific years? (e.g, omitted variables)

16 Uganda: Comparing actual poverty level predictions from URBAN model Better capture of variability than rural Low poverty in base year  low urban predictions 1999 survey bad base for urban models Combination of long time elapsed and large fall in poverty seriously shake the model? (ref predictions)

17 Uganda: Take out two most problematic surveys in urban and rural models Predictions now much more in line with trend Also good predictions at sub-regional level Lower ability to capture sudden changes  Add new types of variables?

18 Conclusion Predictive ability on general trend proven, but: –Models”carry on” their base year poverty level –Difficult to capture sudden changes –Challenge with two individual surveys (survey issues)  If two HBS available, and both <10 years old, use average of both in predictions Possible improvements: Number and level of assets and locality-level explanatory variables Statistically, a second best solution  never perfect