Copyright 2010, The World Bank Group. All Rights Reserved. Reducing Non-Response Section A 1
Copyright 2010, The World Bank Group. All Rights Reserved. Non-Response Non-response is the failure to obtain measures on a unit of enumeration Non-response increases the potential for non- response bias. It occurs after the sampling step of a survey Non-response rates can help us assess the ACCURACY of our statistical projects 2
Copyright 2010, The World Bank Group. All Rights Reserved. Two Main Types of Nonresponse Error Unit Nonresponse –Failure to obtain data from unit of enumeration Item Nonresponse –Failure to collect one or more items on the questionnaire for a unit of enumeration 3
Copyright 2010, The World Bank Group. All Rights Reserved. Non-Response Measurement Statisticians in the United States are working together to develop a core set of unit non- response measures AAPOR - American Association for Public Opinion Research IHSNG - Interagency Household Survey Non- Response Group 4
Copyright 2010, The World Bank Group. All Rights Reserved. Uses of Nonresponse Data Overall indicator of data quality Indicates potential nonresponse bias Efforts to minimize nonrepsonse during data collection operations can indicate quality 5
Copyright 2010, The World Bank Group. All Rights Reserved. Causes of Item Nonresponse Break off an interview due to length and/or complexity Skip applicable questions due to confusion Refusal to answer sensitive questions Inability to provide information required 6
Copyright 2010, The World Bank Group. All Rights Reserved. Computing Item Nonresponse Only responding units are included in calculations The calculation per item only includes those units eligible to answer the item # eligible units answering an item Total # units eligible for the item Can calculate by item, blocks of items, total questionnaire, respondents or interviewers iv7
Copyright 2010, The World Bank Group. All Rights Reserved. Item Nonresponse Classification Criteria can be specified to classify item nonresponse outcomes for analysis –Break off = less than 100% last resort data –Break off = 100% last resort data + less than 50% –Partial = 100% last resort data % –Complete = 100% last resort data + more than 80% 8
Copyright 2010, The World Bank Group. All Rights Reserved. Unit Nonresponse Issues Identify unit of analysis before defining unit nonresponse Nonresponse effect is higher in multi-stage sampling Nonresponse increases in longitudinal surveys 9
Copyright 2010, The World Bank Group. All Rights Reserved. Causes of Unit Nonresponse Personal Interview Survey –Noncontact –Refusal/Inability Telephone Survey –Direct refusal –Indirect refusal Delivered Survey –Not received by intended recipient 10
Copyright 2010, The World Bank Group. All Rights Reserved. Calculation of Unit Nonresponse More complex than simple ratio calculation of interviewed units divided by total number of units in the sample –Partial interviews –Eligible and ineligible cases 11
Copyright 2010, The World Bank Group. All Rights Reserved. Importance of Classifying Nonresponse Nonresponse bias differs by reason for non participation Can use mixed methods to more accurately measure reasons for non-response Can identify trends over time 12
Copyright 2010, The World Bank Group. All Rights Reserved. Computing Nonresponse All cases should be reported by possible outcomes: –Interview –Noninterview –Unknown eligibility –Not eligible 13
Copyright 2010, The World Bank Group. All Rights Reserved. Methods of Calculation Unweighted response rates Weighted response rates Longitudinal calculations 14
Copyright 2010, The World Bank Group. All Rights Reserved. Calculations for Specific Purposes By noncontact versus refusal By interviewer By mode of data collection 15
Copyright 2010, The World Bank Group. All Rights Reserved. Measuring Nonresponse Bias Measurement of differences between respondents and nonrespondents can be obtained from: –Stratified survey sampling frame –Longitudinal survey data –Reinterview a subsample of nonrespondents 16
Copyright 2010, The World Bank Group. All Rights Reserved. Compensating for Nonresponse Weighting unit nonresponse data –Information is needed on nonrepondents –Effects of nonresponse adjustments should be researched 17
Copyright 2010, The World Bank Group. All Rights Reserved. Compensating for Nonresponse Imputation for item nonresponse –Omitting nonresponse items reduces smaple size –Weighting requires a different weight for each item 18
Copyright 2010, The World Bank Group. All Rights Reserved. Quality Indicators for Nonresponse Response/nonresponse rate Rates of refusals and noncontacts Item nonresponse rates Documentation of data collection procedures Special studies of nonresponse bias 19
Copyright 2010, The World Bank Group. All Rights Reserved. Section A Quiz 1.There’s a survey with a sample size of 500 units. You need to calculate the item nonresponse for an item. There were 300 answers out of 450 returns, what is the item nonresponse? 2.What do you need to know to calculate nonresponse bias? 3.How can you calculate unit nonresponse? 4.How can you compensate for nonresponse in your survey data? 20