Presentation on theme: "Sources of Measurement Errors in Business Surveys Mojca Bavdaz."— Presentation transcript:
Sources of Measurement Errors in Business Surveys Mojca Bavdaz
Outline Measurement errors & their sources. Empirical study. Sources of measurement errors redefined. Implications.
Measurement errors Theoretical definition of a measurement error: –deviation of the observed survey value from the true value (Groves, 1991); –difference or discrepancy between the observed survey value and the true value (Hansen et al., 1951; Sukhatme and Sukhatme, 1970). Practical definition of a measurement error: –observational gap between the ideal measurement and the response obtained (Groves et al., 2004); –error which occurs at the time of data collection (Biemer and Lyberg, 2003).
Sources of measurement errors (1) In household surveys (e.g. Groves, 1989): –respondent; –interviewer; –instrument (the survey questionnaire); –mode of data collection.
Sources of measurement errors (2) In business surveys, the addition of the fifth source: –records (Ponikowski, Meily, 1989); –information system (Biemer, Fecso, 1995; Biemer, Lyberg, 2003); –organization (O'Brien, 2000).
Empirical study Business survey under study: –Quarterly Business Survey on Trade (SORS). Aim: –examination of actual response process. Research methods: –in-depth interview; –observation. Sample size (n=28). Challenges of implementation.
Three-dimensional model of business survey response
Sources of measurement error redefined Existing sourcesIdentified sources Respondent + Business participants Interviewers + Survey staff Survey questionnaire (Survey instrument) Survey instrument Mode of data collection + Survey characteristics Records/Information system/Organization Organizational context INTERACTIONS!
Implications Improved tools: –Three-dimensional model for comprehensive & systematic analysis of response process in business surveys. –Redefined sources of measurement errors for development of approaches to measurement error prevention & reduction.
Implications: Example 1 Self-administered business survey. No interviewer. But really no survey staff involved? One-time & rare contacts make huge impact. Training of (all) survey staff.
Implications: Example 2 Recurring business survey. No source of measurement error. But really no impact of recurrence? Learning curve effect & routine. Support & verification of first-time questionnaire completion (at business & respondent level).
Future research More research on specific aspects: –comprehension of economic concepts; –role of business participants & survey staff; –circulation of survey instrument; –impact of soft issues; –... Quantification through experiments. Generalization to non-business organizational surveys.
Sources of Measurement Errors in Business Surveys Mojca Bavdaz Mojca.Bavdaz@ef.uni-lj.si