provide information Development and Implementation of a Mixed-Mode Multipurpose Survey-Tool for Official Statistics Marc Plate Social Statistics Rome 15 May 2014

2 | 5 September 2015 Development of a new Survey-Tool Status Quo: CATI and CAPI using BLAISE, Mixed-Mode: Mainly alternating modes for Panels 160 CAPI Interviewer 60 CATI Interviewer Up to Interviews, in up to 20 Surveys per year New IT-System to reach these goals: Enable Multipurpose Surveys Enable different kinds of Mixed-Mode Designs Enable new mode “CAWI” Planned first use: Household Budget Survey 2014/15 then LFS 2016

3 | 5 September 2015 Enable Multipurpose Design Vision: Modularize the statistical production. Questionnaires are composed by combining modules Official Statistics have the need to design surveys more flexibel Q1 Q2 Q4 Q5 Q6 Q7 Module 1 Module 2 Module 3 eQuestionnaire … Module-Repository CAWI CATI CAPI PAPI

4 | 5 September 2015 The Module Repository - an Example Module: Expenditures for Education and Training Submodule : Housing Expenditures connected to formal education Submodule : Expenditures for other (non formal or occupational) education Submodule: Expenditures for formal education Submodule : Expenditures for occupational training Module: Formal Education Submodule: Highest level of education attained Module: Occupational Education Submodule: Type of occupational education Submodule: Place of education Module-Owner

5 | 5 September 2015 Enable Mixed-Mode Design Vision: Make use of “classic” and “new” ways of data collection (CATI, PAPI, CAPI, CAWI) and supply the possibility of combining modes in a concurrent, sequential or longitudinal mixed-mode design We need the best balance between standardized,automated and respondent oriented case-management We need a combination of modes to achieve highest data quality Centralized System for planning, conducting and monitoring field- phase

6 | 5 September 2015 Monitoring the Field Phase – an example ActivityPerformance IndicatorEvaluation CriteriaCorrective action Timing − median time per page/item − median time for completion per respondent − above a certain level − above or below a certain level − check wording, layout … of the item − consider removing the entry Undesired response patterns − rate of satisfycing patterns per screen/grid − rate of satisfycing patterns per respondent − above a certain level − above a certain level − check wording, layout … of screen/grid − consider removing entry Error/ prompt messaging − rate of error/prompt messages per item − rate of error/prompt messages per respondent − above a certain level − above a certain level − check wording, layout … of item − inspect answers of respondent Response corrections − rate of response corrections per item − above a certain level − check wording, layout … of the item Recruiting− structure of referals- some referals with frequency smaller than expected − check for errors, consider alternative recruiting approach Lozar-Manfreda & Belek 2013

7 | 5 September 2015 Websurvey – The Vision More and more Respondents raise the wish to answer a survey online Vision: Offer a web-questionnaire so well designed, that it works for the general population without the need of any personal contact. Websurvey On- and Offline Functionality Highest data security Webportal List of Questiontypes Innovative Plausibility Checks Innovative Explanatory notes Paradata Integration of Lookuplists

8 | 5 September 2015

9 | 5 September 2015 Multipurpose Design / Modularisation The Challenges Substantial Challenges Reasonable grouping of questions How to make links between modules … Organisational Challenges Reorganisation of responsibilities into modul-responsible, topic-responsible, questionnaire-responsible Persons. Strong communications between departments Module-responsible Person has to design „perfect“ questions Project Team of one survey is not reasponsible for all questions anymore How can modularity be accomplished without loosing sight of the overall context? What methodological implication arise, when questionnaires are designed part by part?

10 | 5 September 2015 Mixed-Mode Design / CAWI The Challenges Known methodological issues of Mixed-Mode desing, particularly Compareability between modes Break in time series How to get R to use web when we „only“ know adress and name CAWI: Does loosing the interviewer mean loosing quality assurance? (Losing control over who is answering, Losing means of collecting feedback on questions) „New“ Issue: Paradata Need for constant Evaluation of questionnaire for all modes.

11 | 5 September 2015 Please address queries to: Marc Plate Contact information: Guglgasse 13, 1110 Vienna phone: +43 (1) Thank you very much for your attention