Implementing mixed mode questionnaire in FI-SILC Kaisa-Mari Okkonen Workshop on Best Practices for EU-SILC Revisions Warsaw 16.10.2018

Contents Background of mixed-mode collection in StatFin FI-SILC mixed-mode questionnaire project Description of current FI-SILC data collection and questionnaire Challenges and possibilities of implementing mixed-mode survey in FI-SILC Main principles of questionnaire design Main changes of FI-SILC questionnaire after redesign Way forward: Plans for pilot survey and what’s next

Background Strategy of Statistics Finland 2016–2019: ”All personal and household data collections will have a web survey alternative in 2018 the latest. Respondents are actively adviced to respond in web.” Strategic target not reached Building of data collection infrastructure delayed State of mixed-mode (web) data collection in StatFin Implemented in ICT survey, AES, Leisure Survey Pilots done in LFS, Consumer Survey, Travel Survey Parallel data collections in Consumer Survey, Travel Survey foreseen in 2019 Piloting in LFS continues in 2019

FI-SILC questionnaire redesign project 1/2017 – 6/2018 Goals Redesigning of questionnaire for both self-administered web survey and telephone Questionnaire programming (Blaise 5.2.5) Cognitive and usability testing of questionnaire Implementing the collection of household composition Designing the collection of paradata Data about answering process for questionnaire improvement: which device is used, how long pages are open, pages where respondents quit Out of scope of this project Designing the pilot survey (incl. decisions about sign-in) Ad hoc modules Changes in the SILC production process Translations of questionnaire (SV, EN)

Current state of FI-SILC data collection Combined data collection for national IDS and EU-SILC Registers are the source for most income data, much of demographic data and composition of dwelling unit (sample) Coherence of register and survey data CATI-mode 98 %, CAPI in 2 % CAPI: elderly persons, immigrants, persons with disabilities FI uses selected respondent model One person followed for 4 years Collection of most personal data for one person per household Large sample (~10,000 hhs) Every 3 years collection for HFCS is done for SILC sample Most data from registers, only few additional survey questions

Current structure of the questionnaire Household members 1st wave: Prefilled data about members of dwell. unit from registers 2nd+ wave: Prefilled data about hh members from previous wave Health, personal material deprivation and SR specific modules Collected only for selected respondent (16+) Module on labour and other activities Collected for all 16+ members, proxies are common, prefills Non-registered incomes Household level questions Transfers between hhs, income from interest, income from abroad… Children and child care Housing, housing costs + hh-level material deprivation Interviewer assistance required Loans, arrears, subjective questions about household’s financial situation

Implementing web questionnaire in FI-SILC: challenges (1/2) First hh level data collection in StatFin mixed-mode environment SILC combines both household and personal questions Who is contacted in the household? Are we able to reach each member personally? How is the use of proxies handled? Use of selected respondent in FI-SILC SR not always the hh respondent How is SR reached (esp. young selected respondents)? Contact and sign-in No interviewer persuading to answer Letter including passwords? Use of web bank sign-in? New portal? Use of registers in prefills Data security in case of errors in registers? Collection of PINs and other personal data for members added to the household?

Implementing web questionnaire in FI-SILC: challenges (2/2) Panel data collection Changes in household composition between collection waves Tracing of SR only, who is contacted, security of personal data of persons no longer hh members? ”Invisible work” by interviewers will disappear in self-administered survey Household’s eligibility to participate and reason for non-eligibility Explaining the household concept Reporting the reason for non-response, in web all are non-contacts? Interviewers can ”soften” sensitive questions (register prefills etc.) Blaise-comments by interviewer clarify answers given Complex questionnaire regarding contents and structure Answering on mobile devices

On the positive side… Web allows participation of households temporarily abroad during fieldwork period More flexilibilty for respondents, people with hearing or other disabilities If ”easy” households use web survey, more interviewer resources can be targeted at hard-to-reach households Also CATI questionnaire has been improved in the process Use of new insentives is possible E.g. using web responses to compare household with other households

Principles of resigning a mixed-mode questionnaire in FI-SILC Mixed-mode = CAWI + CATI + registers Separate questionnaires needed for each survey mode Web questionnaire designed to work on mobile devices Proxies allowed and identified in web mode Avoiding item non-response; routing of questions possible Need to include all relevant currently ”invisible” data in the questionnaire E.g. eligibility of household, most common interviewer comments The whole questionnaire needs a redesign Simplicity and prioritising: instructions are not read, questions are read poorly Usability: respondents are impatient Major redesign for hh composition, labour and other activities, housing

What has changed in the questionnaire? Overall structure of questionnaire (modules) and use of prefills remain the same Interview and web modes have different question formats Data about eligibility of hh in the target population collected in a more structured manner in the beginning of survey Most common interviewer comments are turned into questions E.g. missing electricity bills edited with information about energy consumption given in Blaise comment In the future, collected as a separate question for those who answer DK Checking of internal consistency of survey data improved Module on labour and other activities changed the most Completely redesigned: from thematic structure to narrative design (respondent’s perspective) Heavy routing and calculations based on calendars More data about overlapping activities Simple for respondents, more complex for producer

Next steps: pilot data collection Planning in 2019, data collection in early 2020 Partly funded with Grants Actions consist of e.g. Finalizing data collection tools, planning fieldwork strategy, sign-in options, data collection Planning of sampling, sample size and estimation, evaluation of mode effects, unit and item non-response etc. Planning of data processing, development of processes editing and imputing, constructing target variables Analysis of data, non-response analysis, calculation of weights and calibration Implementation in 2021?

Life after pilot? Decisions concerning production will be made after pilot The whole production process needs rebuilding Breaks in series anticipated concerning especially subjective questions (health, making ends meet, swb module) Alternatives for use of mixed-mode still on the table Web first for all? Interview for 1st wave, web for 2nd wave onwards? Web first for small households? Personal interviews still needed for targeted groups Improvement of questionnaire should be continuous Real time income register 2020 onwards (wages and benefits) Another round of questionnaire design required? IESS implementation Timing, modularisation, other changes

Kaisa-Mari Okkonen kaisa-mari.okkonen@stat.fi