1. ESS Mixed mode experiment ESRA Conference, Ljubljana, July 2013 Alun Humphrey @alunhumphrey 18 July 2013

2. 1 Contents Background and experimental design Response rates Sample profile Completion online Costs

3. Background 1.

4. 3 Funders Co-funded by Economic & Social Research Council City University, London

5. 4 Key aims Feasibility of online ESS UK only has address-based sample frame No personal contact at address – letter only Need to carry out random selection Cost effectiveness of online ESS Need for two survey instruments (F2F/ web) How best to incentivise web completion Are there cost savings?

6. 5 Key issues Exploring online response and subsequent F2F response rates Assessing the impact of different value incentives Identifying best approach to carrying out random selection in absence of interviewer Exploring how people complete long interviews online

7. 6 The sequential design Online questionnaireFace-to-face follow-up

8. 7 Design Run alongside main European Social Survey Round 6 Sample of 2,000 addresses from Post Office Address File All sent letter plus £5 incentive Postal and email reminders if no response Key experimental elements Two methods of respondent selection tested (birthday vs web selection) Two conditional incentive amounts (£15 vs. £35) 500 addresses in 20 points then visited by interviewer who attempts F2F interview with £5 conditional incentive

9. 8 Contact strategy Pre-notification Letter Reminder letter Mailed invite to web survey with £5 incentive Interviewer visit 7 days 10 days 2 months

10. Response 2.

11. 10 Response rates Base: All eligible addresses

12. 11 Impact of incentive Base: All eligible addresses

13. 12 Selection method Base: All eligible addresses

14. Sample profile 3.

15. 14 Sample profile - gender Base: All respondents Male Female

16. 15 Sample profile - age Base: All respondents 15-2425-4445-6465+

17. 16 Sample profile – years in education Base: All respondents Less than 99-1213-15 16-1920 or more

18. 17 Sample profile – in work Base: All respondents

19. 18 Sample profile – household income Base: All respondents Low Mid HighRefusalDKNA

20. 19 Completion online 4.

21. 20 Web survey completion levels Base: All web respondents

22. 21 81% of responders completed the web questionnaire in one session. (80% fully completed and 1% partially) Base: All web respondents.

23. 22 Length of web sessions Median sessions length was 42 minutes But some took extra sessions to complete web survey so median total length was 52 minutes – very similar to F2F ESS R6

24. 23 Does random selection by respondents work? Eligible: Web respondents who reported more than 1 adult living in the household. 68% comply Of eligible respondents who provided information on household Only 40% of eligible provided information on household. We don’t know about the other 60%.

25. Costs 5.

26. 25 Costs

27. 26 Operational costs

28. 27 Operational costs

29. 28 Operational costs

30. 29 Operational costs

31. 30 Conclusions Higher incentive unnecessary Potential problems with person selection done by household Some profile differences by mode, but much reduced in combined web plus face-to-face sample Potential for cost savings using combined approach

32. If you want further information or would like to contact the author, Alun Humphrey Senior Research Director T. 020 7549 7043 E. Alun.Humphrey@natcen.ac.uk Visit us online, natcen.ac.uk Follow us on @alunhumphrey @natcen Thank you