1. Implementing mixed mode questionnaire in FI-SILC
Kaisa-Mari Okkonen
Workshop on Best Practices for EU-SILC Revisions
Warsaw

2. Contents Background of mixed-mode collection in StatFin
FI-SILC mixed-mode questionnaire project
Description of current FI-SILC data collection and questionnaire
Challenges and possibilities of implementing mixed-mode survey in FI-SILC
Main principles of questionnaire design
Main changes of FI-SILC questionnaire after redesign
Way forward: Plans for pilot survey and what’s next

3. Background Strategy of Statistics Finland 2016–2019:
”All personal and household data collections will have a web survey alternative in 2018 the latest. Respondents are actively adviced to respond in web.”
Strategic target not reached
Building of data collection infrastructure delayed
State of mixed-mode (web) data collection in StatFin
Implemented in ICT survey, AES, Leisure Survey
Pilots done in LFS, Consumer Survey, Travel Survey
Parallel data collections in Consumer Survey, Travel Survey foreseen in 2019
Piloting in LFS continues in 2019

4. FI-SILC questionnaire redesign project 1/2017 – 6/2018
Goals
Redesigning of questionnaire for both self-administered web survey and telephone
Questionnaire programming (Blaise 5.2.5)
Cognitive and usability testing of questionnaire
Implementing the collection of household composition
Designing the collection of paradata
Data about answering process for questionnaire improvement: which device is used, how long pages are open, pages where respondents quit
Out of scope of this project
Designing the pilot survey (incl. decisions about sign-in)
Ad hoc modules
Changes in the SILC production process
Translations of questionnaire (SV, EN)

5. Current state of FI-SILC data collection
Combined data collection for national IDS and EU-SILC
Registers are the source for most income data, much of demographic data and composition of dwelling unit (sample)
Coherence of register and survey data
CATI-mode 98 %, CAPI in 2 %
CAPI: elderly persons, immigrants, persons with disabilities
FI uses selected respondent model
One person followed for 4 years
Collection of most personal data for one person per household
Large sample (~10,000 hhs)
Every 3 years collection for HFCS is done for SILC sample
Most data from registers, only few additional survey questions

6. Current structure of the questionnaire
Household members
1st wave: Prefilled data about members of dwell. unit from registers
2nd+ wave: Prefilled data about hh members from previous wave
Health, personal material deprivation and SR specific modules
Collected only for selected respondent (16+)
Module on labour and other activities
Collected for all 16+ members, proxies are common, prefills
Non-registered incomes
Household level questions
Transfers between hhs, income from interest, income from abroad…
Children and child care
Housing, housing costs + hh-level material deprivation
Interviewer assistance required
Loans, arrears, subjective questions about household’s financial situation

7. Implementing web questionnaire in FI-SILC: challenges (1/2)
First hh level data collection in StatFin mixed-mode environment
SILC combines both household and personal questions
Who is contacted in the household? Are we able to reach each member personally? How is the use of proxies handled?
Use of selected respondent in FI-SILC
SR not always the hh respondent
How is SR reached (esp. young selected respondents)?
Contact and sign-in
No interviewer persuading to answer
Letter including passwords? Use of web bank sign-in? New portal?
Use of registers in prefills
Data security in case of errors in registers?
Collection of PINs and other personal data for members added to the household?

8. Implementing web questionnaire in FI-SILC: challenges (2/2)
Panel data collection
Changes in household composition between collection waves
Tracing of SR only, who is contacted, security of personal data of persons no longer hh members?
”Invisible work” by interviewers will disappear in self-administered survey
Household’s eligibility to participate and reason for non-eligibility
Explaining the household concept
Reporting the reason for non-response, in web all are non-contacts?
Interviewers can ”soften” sensitive questions (register prefills etc.)
Blaise-comments by interviewer clarify answers given
Complex questionnaire regarding contents and structure
Answering on mobile devices

9. On the positive side…
Web allows participation of households temporarily abroad during fieldwork period
More flexilibilty for respondents, people with hearing or other disabilities
If ”easy” households use web survey, more interviewer resources can be targeted at hard-to-reach households
Also CATI questionnaire has been improved in the process
Use of new insentives is possible
E.g. using web responses to compare household with other households

10. Principles of resigning a mixed-mode questionnaire in FI-SILC
Mixed-mode = CAWI + CATI + registers
Separate questionnaires needed for each survey mode
Web questionnaire designed to work on mobile devices
Proxies allowed and identified in web mode
Avoiding item non-response; routing of questions possible
Need to include all relevant currently ”invisible” data in the questionnaire
E.g. eligibility of household, most common interviewer comments
The whole questionnaire needs a redesign
Simplicity and prioritising: instructions are not read, questions are read poorly
Usability: respondents are impatient
Major redesign for hh composition, labour and other activities, housing

11. What has changed in the questionnaire?
Overall structure of questionnaire (modules) and use of prefills remain the same
Interview and web modes have different question formats
Data about eligibility of hh in the target population collected in a more structured manner in the beginning of survey
Most common interviewer comments are turned into questions
E.g. missing electricity bills edited with information about energy consumption given in Blaise comment
In the future, collected as a separate question for those who answer DK
Checking of internal consistency of survey data improved
Module on labour and other activities changed the most
Completely redesigned: from thematic structure to narrative design (respondent’s perspective)
Heavy routing and calculations based on calendars
More data about overlapping activities
Simple for respondents, more complex for producer

12. Next steps: pilot data collection
Planning in 2019, data collection in early 2020
Partly funded with Grants
Actions consist of e.g.
Finalizing data collection tools, planning fieldwork strategy, sign-in options, data collection
Planning of sampling, sample size and estimation, evaluation of mode effects, unit and item non-response etc.
Planning of data processing, development of processes editing and imputing, constructing target variables
Analysis of data, non-response analysis, calculation of weights and calibration
Implementation in 2021?

13. Life after pilot?
Decisions concerning production will be made after pilot
The whole production process needs rebuilding
Breaks in series anticipated concerning especially subjective questions (health, making ends meet, swb module)
Alternatives for use of mixed-mode still on the table
Web first for all?
Interview for 1st wave, web for 2nd wave onwards?
Web first for small households?
Personal interviews still needed for targeted groups
Improvement of questionnaire should be continuous
Real time income register 2020 onwards (wages and benefits)
Another round of questionnaire design required?
IESS implementation
Timing, modularisation, other changes

14. Kaisa-Mari Okkonen kaisa-mari.okkonen@stat.fi