Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet Statistics Netherlands Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands May 6, 2010

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Outline Driving forces Approach Maintaining quality Results and perspectives Conclusions

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Driving forces Efficiency Quality Flexibility Constraints: (European) quality / user satisfaction Solution for discontinuities

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Approach Means: More use of registrative data More use of CAWI and mixed mode designs Integration of surveys in one survey system “Re-use” of respondents Implementation of advanced estimation techniques

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Survey Model Optimal use of these means in integrated design for household surveys: Core module and topic modules Well-balanced use of mixed mode (CAWI - PAPI - CATI - CAPI)

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Core Questionnaire Household composition Labour status Health Religion Voting behaviour Occupation in employment Economic sector in employment Highest level of education completed Screening, telephone numbers, adresses

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Standard mixed mode strategy CAWI (+ PAPI) CATICAPI If tel.nr. availableIf not tel.nr. available 25 – 35 % ± 45%± 35% % % Refusal No contact ± 5%

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Mode effects Design principle: Unimode questionnaire design Selection effects Context effects Satisficing Interviewer effects

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Maintaining output quality Explain discontinuities for core variables Quality indicators (framework) Pilots and experiments Parallel designs Time series modelling

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Quality indicators Relevance Accuracy Timeliness Accessibility Comparability Coherence

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Results 2009: Core questionnaire in Health Survey LFS sample size reduction by using data from unemployment offices Structural time series estimators for monthly unemployment Small area estimators for sub-regional unemployment statistics

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Results 2010: Core questionnaire in LFS Mixed mode in LFS (CAPI – CATI) Mixed mode in Health Survey (CAWI – CATI – CAPI) Topic modules after Health Survey 2011/2012: CAWI in LFS More topic modules after core questionnaire

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Parallel designs

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Parallel designs

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Parallel designs

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Issues Unpredictibility CAWI results and impact on fieldwork planning in a mixed mode design Changing geographical distribution CAPI-samples Mode effects (methodology & best practices) Survey integration Creating uni-mode questionnaires Households/LFS and CAWI Flexibility in operation (methodology – survey case management system)

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands Conclusions Quality can be maintained Efficiency targets met Flexible in meeting new national and European needs

Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen May 6, 2010 Statistics Netherlands Challenges of redesigning household surveys and maintaining output quality Statistics Netherlands