The Future of Statistical Data Collection? Challenges and Opportunities Johan Erikson (Statistics Sweden) Gustav Haraldsen (Statistics Norway) Ger Snijkers (Statistics Netherlands) Seminar on New Frontiers for Statistical Data Collection 31 October – 2 November 2012, Geneva, Switzerland

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 2 Outline 1.Fundamental challenges 2.How do we meet these challenges? 3.Are we prepared? Traditional survey perspectives: -Management -Methodology Beyond the traditional survey framework 4. Conclusions

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 3 1.Fundamental challenges Official Statistics in a changing world! Three main challenges: 1.A shift in the balance of power between survey organisations and respondents 2. New competitors may make Official Statistics redundant 3.Globalisation of the economy

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 4 A shift in the balance of power Dillman et al. (2009): “Surveys are now respondent driven, rather than driven primarily by the needs of survey organizations.”

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 5 A shift in the balance of power Traditionally – data collectors in control -Responding to survey was seen as a duty – not any more -The change can be seen clearly by studying the attitude of respondents in different age groups Business organisations focus more on production and competiveness The means of communicating are changing Respondents are getting harder to reach, and can control who they want to be reached by Conclusion: We need to adapt to respondents more and more in order to get their cooperation using all kinds of comm. means.

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 6 New competitors Groves (2011) - “Organic vs. designed data”: “We’re entering a world where data will be the cheapest commodity around, simply because society has created systems that automatically track transactions of all sorts.” E.g. transactions on social media: 3.2 million new entries/day, in the Netherlands

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 7 New competitors Data are everywhere: “Big Data” Conferences New competitors use these data to produce new statistics/information in real time: -E.g. Google’s stock prices index (GOOG) This challenges the way traditional (NSI) statistics are produced: -founded in survey methodology, with a focus on quality The new information maybe directed at different users Conclusion: What is our position in the information market? Can we produce timely, relevant and competitive stats?

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 8 Globalisation UNECE (2011) - The Impact on globalization on National Accounts: “The increasingly global nature of economic transactions and arrangements presents a challenge to the application of national accounts concepts and the use of data collection and compilation systems for measuring developments in the domestic economy.”

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 9 Globalisation Traditional indicators that show a society’s development may no longer be valid The concepts and units (needed to compile these indicators) are becoming irrelevant to globalised businesses: -If the concepts are irrelevant to providers, data will be hard to get -Offices in any part of the world may have to be contacted Conclusion: The definitions, measurement and compilation of traditional indicators needs to be reviewed The concepts and units need to be revised, The data collection methods and procedures need to be adapted in international perspective (relates 1st challenge)

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 10 Data Collection Strategy - 3 steps: 1. Re-use of available dataData sharing & data warehousing 2. Use of new registers and other secondary sources Traditional government-based registers Information from private businesses “Organic/Big Data” sources 3.Primary data collection: 1.EDI technologies, like XBRL 2.Web surveys 3.traditional modes: paper, CATI, CAPI Using new communication technologies Reciprocity: report back to respondents Multi-source designs Mixed-mode designs 2.What is done? The answer to these challenges so far

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 11 3.Is this sufficient? Are we prepared re: challenges? To answer this question we looked at the internal processes in our Statistical Offices What are we actually doing …? What is our short-term focus? The Quality Diamond: (Haraldsen; in: Snijkers, Haraldsen, Jones & Willimack, 2013, Wiley) Integrates four quality perspectives on statistics:

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 12 Specifications Coverage Sampling Nonresponse Measurement Processing Relevance Accuracy Timeliness Punctuality Accessibility Clarity Comparability Coherence (Eurostat Quality Dimensions, 2011) Question order effects Mode effects Cross-cultural effects Costs Technology Ethics Timeliness PRB The Quality Diamond Management perspective Methodology perspective

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 13 Non-response 6Managing customer demands and expectations Mixed-mode effects Data Collection Strategy effects 1 Costs 2 Timeliness 3 Flexibility of process planning 4 Innovation of process 5 Response Burden 7 Planning staff 8 Systems and tools 9 Culture The Quality Diamond: The Management Perspective

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 14 Non-response Measurement Coverage Sampling Accuracy Mixed-mode effects Questionnaire effects Data Collection Strategy effects Response Burden Innovation of data collection methods The Quality Diamond: The Methodology Perspective

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 15 Differences in focus: Management (mostly) interested in becoming better (more efficient) within traditional survey framework. Short-term challenges, while running the surveys: -Standardisation and integration of processes -Innovating the current processes within the Data Collection Strategy, like planning mixed-mode designs Methodology driven by quality considerations: -Reducing survey design effects and Total Survey Error -Survey process constraints not much considered The two perspectives hardly overlap

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 16 Beyond the traditional survey framework Short-term focus – based on traditional framework  We need to go beyond that framework: Considering challenge 1: balance of power The end of probability sampling? -High non-response levels and low representativeness: responsive designs, web panels, post-survey adjustments -Use of registers: no sampling involved; non-sampling errors? Towards multi-source/mixed-mode designsThe end of standardised questionnaires/data collection? -From collecting data to collecting {data & metadata} using data capture methods (like XBRL) -From standardised interviewing to conversational interviewing and data mining, using new technologies (smart phones, Skype)

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 17 Beyond the traditional survey framework Official data requirements and concepts – based on stove-pipe approach: traditional indicators, separate surveys  We need to go beyond that: The end of traditional indicators to measure societal developments? Considering challenge 3: Globalisation -What information do we need? -How do we measure that information? The end of official statistics? Considering challenge 2: New competitors -What will be our position in the information market?

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva So, are we prepared? Standardisation and integration of data collection processes are a step in the right direction … but not enough Data Collection Strategies towards multi-source/mixed- mode designs are a step in the right direction … but not enough  Managers and methodologists need to collaborate, having common short-term and long-term focuses: a communication perspective!  We need to go beyond the traditional survey framework … and act now! The future is already here!