1. The European Statistical Training Programme (ESTP)

2. Chapter 2: The nonresponse problem
Handbook: chapter 1
What is nonresponse?
Causes of nonresponse
Response rates
Some international trends

3. Nonresponse Nonresponse
Elements that are selected in the sample, and that are also eligible for the survey, do not provide the required information.
Two types of nonresponse:
Unit nonresponse: a selected element does not provide any information at all.
Item nonresponse: a selected element does answer some questions, but not all of them.
Why is nonresponse a problem?
Smaller sample size.
Nonresponse bias due to selective nonresponse.

4. Selective nonresponse
Example: Dutch Housing Demand Survey 1981
Initial sample size: 82,849 persons.
Response: 71.2%.
Follow-up survey among nonrespondents:
Percentage of movers in response: 29.7%.
Percentage of movers in nonresponse: 12.8%.
Do you intend to move within 2 years
Response
Nonresponse
Yes
17,515
3,056 No
41,457
20,821
Total
58,972
23,877

5. Selective nonresponse in Dutch Surveys
Victimization Survey
People who are afraid to be home at night are less inclined to participate in the survey.
Mobility Survey
The more mobile people were harder to contact, and thus under-represented in the survey,
Labour Force Survey
Refusal rates are higher among the unemployed.
Election Survey
Participation rates are higher among voters. Voters are over-represented.

6. Selective nonresponse
Simulation
Estimation of the mean income in Samplonia.
Simulate 1,000 samples of size 50.
Case 1: No nonresponse.
Case 2: Nonresponse increases with income.
Conclusion: estimates are systematically too low in case of nonresponse.
Confidence intervals are wrong.
No nonresponse
Selective nonresponse

7. Causes of nonresponse (CAPI)
Contact? No
Eligible?
Yes
Nonresponse
Non-contact
Over-coverage No
Eligible?
Over-coverage
Yes No
Participates?
Nonresponse
Refusal
Yes No
Able?
Nonresponse
Not-able
Yes
Response

8. Causes of nonresponse Non-contact
Nobody home, phone not answered.
Make more contact attempts (up to 6).
Refusal
Can be permanent or temporary.
Role of interviewers important.
Use of incentives?
Not-able
Physical or mental problems.
Language problems.
Unprocessed
Interviewers are unable to handle their workload.

9. Response rates Definition Lynn et al. (2002)
Proportion of eligible elements in the sample for which a questionnaire has been completed:
Notation
nE = Number of eligible elements in the sample
nR = Number of eligible respondents
nE = nR + nNC + nRF + nNA
Initial sample size n = nE + nOC.
So nE = n – nOC.
Problem: over-coverage unknown for non-contacts.

10. Response rates Estimating the response rate
Assumption 1: all non-contacts are eligible:
Assumption 2: the proportion of eligibles among non-contacts is the same as the proportion of eligible among contacts:

11. Response rates - Example
Fieldwork results for the Dutch Integrated Survey on Household Living Conditions in 1998:
If all non-contacts are eligible, the response rate equals
If the proportion of eligibles among contacts and non-contacts is the same, the response rate is
Outcome
Frequency
Over-coverage
129
Response
24,008
Non-contact
2,093
Refusal
8,918
Not-able
1,151
Other nonresponse
3,132
Total
39,431

12. Response rates
Response percentages in some Dutch surveys

13. Response rates Some other response rate problems
Self-administered surveys (no interviewers):
It is unclear who is completing the questionnaire. Is the person eligible?
The cause of nonresponse is unclear.
Unequal probability sampling: How to compute the response rate if the sample is selected with unequal probabilities?
Unweighted response rate as a quality indicator of the performance of interviewers.
Weighted response rate as a quality indicator of estimates.
Household survey: How to define response/nonresponse of a household?

14. International trends
De Heer (1999) compared Labor Force Surveys for official statistical offices
Large difference in response rates and trends
Rates vary from around 60 (NL) to around 90 (USA) percent.
UK, USA stable pattern.
Finland, Sweden downward trend.
Belgium seems to be recovering.
The Netherlands world champion nonresponse.

15. International trends
Also a large differences in the composition of nonresponse in non-contacts and refusals
UK: non-contact decreasing whereas in most countries it is increasing.
Refusals mostly differ in rate, not so much in trend.

16. International trends Possible explanations towards differences:
Difference in general design
Belgium NL UK
Finland
Sweden
USA
Data collection mode
face-to-face
CAPI
CATI
Observational unit
household
person
Substitution
yes no
Call scheduling
Refusal conversion
Mandatory
no*

17. International trends
Stoop (2005) analyses data from the first round of the European Social Survey, in 2002/2003. In the European Social Survey:
One sampling design
Response target: 70%; non-contact target: 3%
Central fieldwork specifications
The results are very different:
The response rate varies from 33% (Switzerland) to 80% (Greece)
The non-contact rates varies from 2% (Poland) to 15% (France)
The refusal rate varies from 14% (Hungary) to 55% (Switzerland)
The Netherlands is doing well, with a response rate of 68%, non-contact of 3% and refusal rate of 24%.

18. International trends
Results of the first round of the ESS

19. International trends
The second round from the ESS in 2004/2005 showed and improvement of low response countries with respect to the first round (Stoop and Billiet, 2007):
Response rate in the European Social Survey in the first two rounds

20. International trends
Response rate and refusal/ (refusal+noncontact) in the European Social Survey Round 2
Relationship between the response rate and the composition of the nonresponse.
Refusal is more important than non-contact in most countries
No relationship between response rate and response composition