Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 1 Topic (v): Editing based on results Discussants: Maria M. Garcia / US Census Bureau Daniel Kilchmann / SFSO

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 2 Introduction This topic covers issues concerning –Macro editing (1 paper) –Selective editing (4 papers) –Post-editing (1 paper)

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 3 Summary Macro-editing –WP.30 (USA) - Macro-editing based on time series Selective editing –WP.31, WP.32, WP.33 (Sweden) - Local and global score functions, case studies, ‘complex’ selective editing considering domains of study –WP.35 - Selective editing in a census Post-editing –WP.34 (Finland) – Case studies related to editing after dissemination of results

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 4 Topic (v): Editing based on results Presentations

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 5 Macro-editing: Main developments WP.30 (USA) –New tools for macro-editing applied to petroleum monthly product supplied survey data. –Provide point and interval forecasts of next month estimates using econometric time series. (1) base model using trend and seasonal indicators (2) ARMA model (3) base model plus exogenous variables –Forecast intervals are compared with preliminary survey aggregates to identify potential outliers.

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 6 Selective Editing: Main developments WP.31 (Sweden) –Formalization of selective editing methodologies summarizing local scores into a unified global score function using the concept of distance. –Different strategies based on the error structure Errors left after selective editing should be as small as possible in terms of MSE and/or bias. Keep the effect of errors under control

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 7 Selective Editing: Main developments WP.32, WP.33 (Sweden) and WP.35 (Spain) –WP. 32 – Studies on several separate surveys to analyze and test selective editing methodologies with the aim of evaluating feasibility of general selective editing tools. –WP.33 - Development of general IT tools for selective editing including adjustments to capture survey-specific needs. –WP.35 – Performance of selective editing based on administrative data.

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 8 Post-editing: Main developments WP.34 (Finland) –Description of difficulties met when proceeding to a post-editing stage after “clean” microdata has been released.

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 9 Macro-editing: Points for discussion WP.30 (USA) –What is the procedure if the survey estimate is outside the forecast interval (localization of the outlier)? –Is it possible to take the variability of exogenous variables, which may be predictions, into account in the models? –Are there cost benefits to the agency with savings that can be redirected to other survey activities? Is there any measure of increased data quality? –These tools are applied to monthly data. Do you get some insight from the macro-editing application into possible procedures that could be enhanced to improve micro-editing in the next cycles?

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 10 Selective Editing: Points for Discussion WP.31, WP.32, WP.33 (Sweden) and WP.35 (Spain) –Global score function: Minkowski‘s distance type vs. multivariate score functions (Mahalanobis distance type)? –Editing may increase the bias: how to deal with that? –How can we take into account and control for multivariate relations among variables when developing/implementing score functions?

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 11 Selective editing: Points for discussion –Selective editing leads to efficiency gains: impact on data quality? –Most statistics may benefit from the use of selective editing: general mandatory guidelines? –How to choose optimal parameters? Are the optimal parameter values dependent on the method that has been used to flag the observations? Do we need to continuously monitor for parameter adjustments? –Is it possible to design a selective editing strategy that is robust against changes in the survey organization and/or modifications of the survey design?

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 12 Selective editing: Points for discussion –Do you consider including all domains of study in selective editing realistic or is there a risk of masking? –How is the IT-tool progressing? Could you already test if the importance parameters are efficient enough to manage a huge amount of publication cells of domains of study in selective editing? make a sensitivity analysis using different parameters?

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 13 Selective editing: Points for discussion –How to model “expected” values needed for computing suspicion and/or impact? How best to choose domains for computation of “expected” values in order to achieve relevancy and accuracy? What is the minimum number of observations needed to compute these “expected” values in each domain? –When incorporating the use of administrative data in selective editing, may high scores also reveal possible errors in the administrative data?

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 14 Post-editing: Points for discussion WP.34 (Finland) –How can the published results be maintained if post- editing is applied? –Can post-editing of cross-sectional surveys fill the gaps for longitudinal analysis? –How can we reconcile “clean” microdata with new inconsistencies found after its released while using data for purposes not envisioned at the time it was originally collected?

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 15 General Points for discussion Designing an overall editing strategy –To what extent should national institutes incorporate selective editing, automatic editing, and macro-editing in their overall editing strategies? For what kind of surveys (i.e. short vs. long term, sample surveys vs. administrative data)? –When designing editing strategies for a particular survey, how can we take into account the fact that the microdata may be used by other users and for different purposes?