Keystroke analysis and implications for fieldwork WP3.3 – task 2 Johanna Bristle (SHARE, MEA) Verena Halbherr (ESS, GESIS) DASISH Final Conference – Gothenburg, 28 th Nov 2014

Introduction 2  Tools produce paradata, e.g. Fieldwork Monitoring Application produces contact data (task 1 of WP3.3)  Goal of task 2:  Analysing one type of paradata, namely SHARE keystroke data and ESS time stamp data  Inform cross-national fieldwork & derive lessons learned  Definition:  Paradata = data about the process of data collection  Keystroke data = actions on a keyboard and time stamps for each action taken are recorded and extracted

Deliverable D3.7  Structure of deliverable  Description of data and preparation  Diagnostic of keystrokes  Outliers and distribution  Measuring interview length  Using keystrokes in a survey`s life-cycle  Questionnaire development  Fieldwork monitoring  Post-survey quality assessment  Available on the website of DASISH beginning of 2015  Today: comparing the two surveys 3 Focus of poster Focus of this presentation

Interview length  Concept of interview length  Perspective of questionnaire development vs. interviewer and respondent 4

Measurement error in time stamps  Paradata is far from error-free  Technical errors or breaks produce outliers or missing data  Reporting error if interviewer is involved (rounding) 5

Cross-national interview length 6 Notes: SHARE: own analysis on panel single households in wave 5 ESS: Analysis by Loosveldt & Beullen round 5, all respondents.

Conclusions 1.Computerized data collection avoids interviewer-induced measurement error  Data quality of paradata  data quality of survey data 2.Cross-national & cross-survey comparison  Beyond survey-specific reasons (topic)  „Local survey culture“ (language) Potential of paradata  Pretest: check quality of new items  During fieldwork: real-time monitoring and interventions  Post-survey analysis: survey methodology & enhancing survey items Harmonization in terms of documentation, dissemination and tool development produces useful para- and metadata 7

Further reading and references  DASISH Deliverable 3.7 (forthcoming). Keystroke analysis and implications for field work. It will be available on  Loosveldt, G. and K. Beullens (2013). "'How long will it take?' An analysis of interview length in the fifth round of the European Social Survey." Survey Research Methods 7(2):  Kreuter, F. (2013). Improving Surveys with Paradata: Analytic Uses of Process Information. Hoboken, New Jersey, John Wiley & Sons. 8