1 1 Web in mix-mode surveys in Norway Bengt Oscar Lagerstrøm Symposium on General Population Surveys on the web, London November 2011

2 Surveys in Norway on ”general” populations 2001 – Mode effects in a mail plus Internet designed Census. Haraldsen et al (2002) 2007 – Rent Market Survey 2006, Lagerstrøm (2007) 2008 – Rent Market Survey 2007, Lagerstrøm (2008) (2011 – Survey on Education 2011, Haraldsen et al (2012)) 2011 – CPI - Rent prices, Lagerstrøm (2012)

3 Experiments 1. Census Mode effects. Paper versus two types of web questionnaires 2. The Rent Market Survey (RMS) – From interviewer administrated to self administrated – effect on RR, bias and estimates 3. RMS – data quality and costs 4. RMS and CPI – panel issues

4 Hypothesis Is it essential that the respondent have the opportunity to choose in which mode they would like to participate? A common assumption appears to be that offering multiple modes of reporting makes the reporting task easier, which will lead to higher response and better data quality. More motivated respondents and better quality of data are the two main pillars in Statistics Norway’s new data collection strategy, and one way to get more motivated respondents would be to offer them to respond in a preferred mode. But, how should we present the different tools? One by one, all at once?....

5 Option of P or W vs. paper/web (RMS 2006) H (15,000) - option to respond on paper/web (telephone) P (1,000) – option to respond on paper (telephone) W (1,000) – with option to respond on web (telephone)

6 Estimated tenants by treatment (RMS 2006) HWPT Tenants, phase Tenants, phase

7 Response rates by phases (RMS 2007) Phase 1Phase 2Phase 3 Seq.Opt.Seq.Opt.Seq.Opt. Total (1 157)(1 799)(2 465)(2 639) (4 759)(4 861) Web (1 157)(472)(1 480)(631)(1 497)(718) Paper (1 327)(985)(2 008)(1 032)(2 121) Telephone (2 230)(2 022)

8 Bias by age groups (RMS 2007) P1P2 P3

9 Costs (NOK) (RMS 2007) P1P2P3Total Cseq Copt

10 Lessons learn from Census and RMS Paper seems easier to fill in than web questionnaire, but less errors in web questionnaires for less educated and older respondents Don’t give the respondents the options to choose, but change mode to ensure a satisfactory RR. Young respondents have a higher propensity to respond on web, especially men Estimates could be mode sensitive? Probably nothing two win in terms of money if you want same RR and bias structure

11 Our puzzle Motivation AvailabilityOpportunity

12 New experiments Change the wording in the advance letter – from ”please use the URL and password” to ”A interviewer will contact you, but you can still use the URL/password” Sending SMS with direct access to the web questionnaire

13 Mode propensity in a panel survey (RMS/CPI 2011) PaperTelephoneWebTotal Telephone Web