1. The European Statistical Training Programme (ESTP)

2. Chapter 14: Mixed-mode datacollection
Introduction
Mixed-mode designs
Mixed-mode in practice: the Dutch re-design

3. Introduction Every mode has its weaknesses and its strengths
CAPI: expensive, good quality data
CATI: cheaper, but not every person has a telephone
Mixing modes provides an opportunity to compensate for weaknesses of the individual modes
Interviewer
No interviewer
Paper
PAPI
Mail-survey
Laptop
CAPI
CASAQ
Telephone
CATI
Voice response
Internet
CAWI
Web-survey

4. Mixed-mode designs
A mixed-mode design consists of a combination of two or more data collection modes
Three possibilities:
Concurrent
Sequential
Choice to the respondent

5. Concurrent mixed-mode design
Sample
Mode 1
Mode 2 …
Mode m
The sample is divided in groups that are approached by different modes, but at the same time

6. Sequential mixed-mode design
Sample
Mode 1
Mode 2
Response
Nonresponse
All sample elements are approached by the same mode, but a different mode is used to follow-up the nonrespondents

7. Examples of mixed-mode designs
Safety Monitor 2006 and (n = 30,000, n = 3,600)
LFS 2005 (n=18,000, n=1,000)
Sample
Phone
F-to-F
Phone
F-to-F
Web
Response
Nonresponse
F-to-F
Response
Nonresponse
Web
Phone
F-to-F

8. Examples of mixed-mode designs
Informal Economy (n = 2,000, n = 2,000)
Phone
Web
Response
Nonresponse
F-to-F

9. Examples: Response rates and composition
Measure of representativity
Strategy 1
Strategy 2
Safety Monitor
68%
67%
81%
Informal Economy
57%
49%
77%
78%
LFS
62%
76%
79%

10. Choosing a mixed-mode design
Issues
Questionnaire
Trade-off between errors and costs: coverage, unit nonresponse and measurement errors
Subject of the survey
Constraints
Time
Costs
Logistics

11. Mixed-mode datacollection in practice: Dutch re-design
Project from
Aim: Reduce costs but maintain quality
Main ingredients of re-design:
Core questionnaire
Use of register information
Model based estimation
Quality framework
Parallel runs of old and new designs
Mixed-mode datacollection

12. Mixed-mode datacollection in practice: Dutch re-design
Web
response
nonresponse
CAPI
CATI

13. Mixed-mode datacollection in practice: Dutch re-design
Issues: +
Web
response
nonresponse
CAPI
CATI
nonresponse
response
nonresponse
response
14.13

14. Mixed-mode datacollection in practice: Dutch re-design
Ideally, data collection strategy tailored to different groups based on:
Costs
Quality
Logistics (systems)
Quality is a mixture of nonresponse bias, coverage- and measurement errors in the different modes.
Stability in the mixture of modes is important for planning of fieldwork, adjustment weighting and mode effects.

15. Mixed-mode datacollection in practice: Dutch re-design
Coverage in different modes: CAPI covers entire population. For Web and CATI there is undercoverage for:
Characteristic
CATI
Web
Size of hh
1 person
1, 2 persons
Mar. Status
Unmarried
Age
25 – 44
65+
Ethnicity
Non-natives
Urbanicity
Strong degree of urbanisation
Province plus large cities
Large cities
Income hh
Low
Low and middle
Average house value
0 – euro

16. Mixed-mode datacollection in practice: Dutch re-design
Response in different modes (Health Survey 2010)
Design
Subgroup
R-indicator CI
Old
0.804
(0.780 – 0.828)
New
0.796
(0.780 – 0.812)
Internet
0.786
(0.772 – 0.800)
Follow-up
0.820
(0.800 – 0.840)
CATI
0.833
(0.807 – 0.859)
CAPI
0.790
(0.758 – 0.822)

17. Mixed-mode datacollection in practice: Dutch re-design
Response in different modes (Health Survey 2010)
Conditional and unconditional partial R-indicators
Old design (CAPI) vs new design (total Mixed-Mode)
 CAPI
Unconditional
Conditional
Age
56.7
43.5
Urbanicity
43.9
20.6
Type of hh
55.4
31.7
House value
50.9
25.9
Job
11.1
13.4
Ethnicity
51.0
31.8
 Mixed-Mode
Onconditional
Conditional
Age
60.7
44.6
Urbanicity
45.5
18.5
Type of hh
58.1
30.3
House value
54.0
25.0
Job
1.4
1.5
Ethnicity
54.3
37.2
Het verschil tussen de onconditionele en de conditionele waarden in Tabel 1 is groot. Dat betekent dat er veel onderlinge samenhang is tussen de achtergrondkenmerken.

18. Mixed-mode datacollection in practice: Dutch re-design
For the new design: Internet response vs Internet nonresponse
For the new design: Follow-up response vs follow-up nonresponse
 Internet
Onconditional
Conditional
Age
55.0
39.8
Urbanicity
30.2
10.0
Type of hh
64.0
43.6
House value
63.2
32.5
Job
23.7
17.2
Ethnicity
58.0
41.8
 Follow-up
Onconditional
Conditional
Age
70.5
52.1
Urbanicity
43.0
25.5
Type of hh
34.6
18.4
House value
29.3
Job
27.4
6.7
Ethnicity
33.1
21.2