Presentation on theme: "Analysis of Factors Related to Nonresponse for Landline and Cell Phone Surveys in China Yan Jiang School of Statistics, Renmin University of China Visiting."— Presentation transcript:
Analysis of Factors Related to Nonresponse for Landline and Cell Phone Surveys in China Yan Jiang School of Statistics, Renmin University of China Visiting Professor, Iowa State University, Department of Statistics and Center for Survey Statistics and Methodology ITSEW2010
6/2/2014 3 Introduction Surveys in China High nonresponse rates Low contact rates (RDD Sampling) High refusal rates Change of coverage Steady increase in cell phone frame Decrease in landline phone frame since 2006
6/2/2014 4 Amount of telephone users (Ten thousands) 2009 Cellphone Users: 747.38 million Landline Users: 697.02 million Overlap: unknown Total : 1335 million
Survey Objectives To evaluate the utility of selected methods (e.g., increased call effort) in telephone surveys of adults in China To identify factors related to nonresponse rates (e.g., interviewer voice quality, call effort) To study how the effect of these factors on nonresponse vary by sample frame (i.e., landline, cell) 6/2/2014 5
Survey Design Population: Adults in China (over 18 years old) Two-phase Sampling Design Phase 1: Dual frame sample of phone numbers to sample adults Phase 2: Sample of nonrespondents Data Collection Mode: CATI Phase 2: Modified methods 6/2/2014 6
Survey Design in First Phase Sample Size: 17,162 phone numbers Cell phone: 9678 Landline: 7484 Stratified RDD Sampling Strata: Provinces Allocation: proportional to Province HH pop. Standard call methods Up to 3 call attempts – usually 2 Ring for about 20 secs 6/2/2014 7
Survey Design in Second Phase Population: nonrespondents in first phase Sample size: 1549 Strata: province, frame source, noncontact and refusal numbers Effort to improve the response rate (both cell phone and landline survey) Up to 15 call attempts were made to contact Wait until at least one minute in each dial process 6/2/2014 8
Nonresponse Rate in Second Phase Survey landlinecellphone sample sizeratesample sizeRate response76343.6%78645.0% nonresponse98556.4%96155.0% non-contact44325.4%46726.7% refuse54231.0%49528.3%
6/2/2014 11 Number of call attempts in refusal sample (stratum) of Phase 2 Cumulated Refusal Conversion Rate of landline survey is higher than cell phone survey when the number of attempts is less than 5. Cumulated Refusal Conversion Rate increases rapidly up to about 6 call attempts. Landline Cell phone Number of call attempts
6/2/2014 12 logistic model for Refusal Conversion Dep. Variable=1, refusal unit convert to respondent landline survey cell phone survey Variable BSig. Variable BSig. Gender-.014.604Gender-.196.000 Young-.791.000Young-1.056.000 Middle-aged-.767.000Middle-aged-1.068.000 Education-.189.000Education.187.000 Income-.066.000Income-.090.000 City-.066.024City.156.000 East1.236.000East.941.000 Middle-.367.000Middle.984.000 Number of call attempts.146.000 Number of call attempts.097.000 Constant-1.621.000Constant-2.241.000 The relationship between above factors and refusal conversion rates differs for the landline survey and the cell phone survey.
6/2/2014 13 Number of call attempts in non- contact sample (stratum) of Phase 2 Cumulated Phase 2 Response Rate of landline survey is lower than cell phone survey in non-contact sample. Landline Cell phone Number of call attempts
6/2/2014 14 Distribution of the number seconds until the phone is answered On average,10 additional seconds per interview are required for the phone to be answered in the cell phone survey relative to the landline survey. Cell Phone Land line
6/2/2014 15 Phase 1 Interviewer Analysis Cooperation rate distribution of interviewers 0.700.600.500.400.300.20 Cooperation Rate 20 15 10 5 0 Frequency Mean = 0.375 Std. Dev. = 0.083 N = 108
Phase 1 Interviewer Analysis Which interviewer characteristics affect cooperation rates? Interviewer characteristics Skill level Vocal quality Control for respondent characteristics Gender, education, career, income, location of residence, sample frame 6/2/2014 16
Phase 1 Interviewer Analysis Hierarchical Generalized Linear Model Level 1 model (respondent level) Level 2 model : (interviewer level) i : respondent; j: interviewer; :refusal rate : variables of respondent : variables of interviewer
6/2/2014 18 Variable Definitions outcome of interview y =1, refusal; y =0, response respondent X gender=1, male; gender=0, female edu1: low education, edu2: midium education career1: student, career2: manager, career3: clerk, career4: self-employee income>=0 survey =1, landline ; survey =0, cell phone grade: =1, large city; grade=0, other Interviewer W W1: interview skill of interviewer W2: vocal characteristic of interviewer
Results Both the interviewers speaking skill and natural vocal characteristics have significant impacts on refusal rates. Both the demographic characteristics of respondents and interviewers have significant impacts on refusal rates. Survey type does not have significant effects on refusal rates in the model RespondentInterviewer
Conclusions 1. Dual frame sampling is necessary for reducing coverage bias for telephone surveys in China. The ineligible (e.g., non-working, business) rate in cell phone surveys appears to be lower than in landline surveys in China. 2. High non-response rates can be reduced by effective voice training of interviewers, more call attempts, and extending waiting time before giving up on the call. 3. The demographic characteristics and behaviors of cell phone users and landline telephone users are different, which may have implications for nonresponse bias. 4. Interviewers should wait longer for the respondent to answer the phone in the cell phone survey relative to the landline survey. 5. Refusals occur more quickly in the cell phone survey than in the landline survey.
Your consent to our cookies if you continue to use this website.