1. HTW Berlin University of Applied Sciences
General Online Research Conference (GOR 17) March 2017
HTW Berlin University of Applied Sciences
Berlin, Germany
Maria Andreasson
Elias Markstedt
Johan Martinsson
Laboratory of Opinion Research, University of Gothenburg
Day of the week and time of the day for survey dispatch
Contact:
Andreasson, M, Markstedt, E, Martinsson, J “Day of the week and time of the day for survey dispatch.” General Online Research (GOR) Conference, Berlin.
This work is licensed under a Creative Commons Attribution 4.0 International License (

2. Day of the week and time of the day for survey dispatch Two large-scale randomized experiments
Maria Andreasson
Elias Markstedt
Johan Martinsson
Laboratory of Opinion Research (LORE)
University of Gothenburg

3. Laboratory of Opinion Research (LORE)
The Swedish Citizen Panel (SCP):
Probability (9,000) and
Nonprobability-based panels (ca 50,000)
Web survey experiments
Panel data
Survey methodology

4. Study 1
Q: Are click rates affected by the day-of- week respondents receive an invitation ?
Expectation: Week-end days are worse

5. Survey design Study 1: Day-of-week
11,300 pre-recruited members of the panel
Non-probability sample
Randomly assigned day-of-week for survey invitation (n=1,600/day)
Survey invitation 15–21 October 2014
Reminder after one week

6. Study 2
Q: Are click rates affected by when during the day respondents receive an invitation ?
Expectation: Mornings are better

7. Survey design Study 2: Time-of-day
47,000 pre-recruited members of the Citizen Panel
Nonprobability sample
Randomly assigned to receive a survey invitation on one of six time-of-day treatments
n=7,900 per group
Field period 27th November–3rd December 2014

8. Time-of-day Late evening 21.00 Early evening 18.00 Early morning:
07.00
Afternoon
14.20
Late morning
10.00
Lunch time
12.20

9. DV: Click rates Excluding email bounces
Clicking on survey link in invitation
Click rate after:
24 hours
6 days (1 day before reminder)
8 days (1 day after reminder)
14 days

10. RESULTS STUDY 1 DAY-OF-WEEK

11. Click rates (%)
Anova: 24 h p=.00, 6 days p=.43, 8 days p=.27, 14 days p=.16

12. ANOVAs by labor market status

13. ANOVAs by age

14. Click rate for individuals of different labor market status, by day-of-week
Workers are less likely to respond quickly (within the first 24 hours) to invitations they receive on a Saturday than to invitations they receive on a Monday (diff= -10%, t= -4.61, p= .000), Tuesday (diff= -7%, t= -3.27, p= .019), Wednesday (diff= -9%, t= -4.18, p= .001), Thursday (diff= -8%, t= -3.69, p= .004), and, at a 90% confidence level, on Friday (diff= -6%, t= , p= .095).
Workers are also less likely to respond quickly when they receive the invitation on a Sunday than on a Monday (diff= -6%, t= -3.07, p= .035).
Pensioners are less likely to respond quickly (within the first 24 hours) to invitations they receive on a Saturday than to invitations they receive on a Monday (90% confidence level) (diff= -12%, t= -2.90, p= .057), Wednesday (diff= -15%, t= -3.66, p= .005), and a Thursday (diff= -13%, t= -3.21, p= .023).
There are no significant differences in participation rate for unemployed/students.

15. Summary Study 1
Invitation day has significant effects on click rates after 24 hours
Saturday worst
Early days of the week best
All differences disappear within 6 days
When comparing different subgroups, effects are only seen for gainfully employed and pensioners, and for individuals between the age of 30-69

16. RESULTS STUDY 2 TIME-OF-DAY

17. Click rate by time-of-day
Table 4 shows the net participation rates split by the time respondents received the invitation . The first column shows the participation rate after 24 hours and the second shows the participation rate after 6 days. Only 1 of 15 group comparisons differ significantly from zero (99% confidence level). The 21:00 group had a 2 percentage points lower participation rate than the 12:20 group.
No differences are found after 6 days.

18. Click rate by labor market status and time-of-day
An analysis of participation rate in different labor market status groups shows that pensioners are less “flexible” than other groups (see Table 5). Pairwise comparison tests reveal that, on 90% confidence levels, these respondents are less likely to respond quickly, that is within the first 24 hours, to s they receive at 18:00 than to s they receive at 12:20 (diff= -4%, t= -2.67, p= .081), and to s they receive at 21:00 than to s they receive at 12:20 (diff= -5%, t= -2.83, p= .054).
There are no significant differences between treatment groups for workers or for unemployed/students.

19. Summary study 2
Click rate can vary within first 24 hours by time-of-day for invitation .
Looking at subgroups, more pronounced effects are seen for pensioners
Differences disappear after 6 days

20. Further studies Preference for receiving inviations
Matching and mismatching preference with day and hour
Small gains in the short run
Complicated

21. Conclusions
Minor differences in click rates when time-of-day and day-of-week for invitation is varied
Effects can only be seen 24 hours after survey invitation, not later
For pensioners and gainfully employed, click rates can be improved somewhat within the first 24 hours by sending invitations during weekday evenings
All differences disappear in the long run
Implications: there are small gains in varying the timing of invitation s to surveys overall

22. Thank you! You can read more at lore.gu.se