1. Topic (iii): Macro Editing Methods Paula Mason and Maria Garcia (USA) UNECE Work Session on Statistical Data Editing Ljubljana, Slovenia, 9-11 May 2011

2. Topic (iii): Introduction This topic covers issues concerning macro editing and selective editing  Macro editing Key Invited paper – Australia Invited papers – Netherlands, New Zealand, Canada (2)  Selective editing Key Invited paper – Spain Invited papers – Sweden, UK 2

3. 3 Topic (iii): Introduction  Macro-editing – WP.13 – significance editing framework for macro editing – WP.14 – development of a macro editing tool – WP.15, WP.16, WP.17 – macro editing in an overall editing strategy  Selective editing – WP.18 – theoretical framework for selective editing – WP.19, WP.20 – selective editing using software tools developed in Sweden, and applied by Sweden and the United Kingdom

4. Topic (iii): Macro Editing Methods Enjoy the presentations! 4

5. Topic (iii): Macro Editing Methods Summary of main developments and points for discussion 5

6. Macro editing: Main developments  WP.13 (Australia) – Added macro editing strategies to existing significance editing framework – Scores based on predicting impact on outputs – Target macro editing effort at different hierarchical levels – Incorporate sensitivity measures to address swamping and masking 6

7. Macro editing: Main developments  WP.14 (Netherlands) – Software for developing custom macro editing tools accessed by scripts – Functionalities include aggregation techniques, data visualization, dynamic filters, data correction and recalculation. 7

8. Macro editing: Main developments  WP.15 (New Zealand) – Incorporate macro editing in an overall editing strategy – Increased use of automatic micro edits – Prioritize using expected effects on the outputs – Developed quality indicators – Report efficiency gains 8

9. Macro editing: Main developments Canada – Common survey framework for business surveys (two papers)  WP.16 – Iterative process – Rolling estimates model and common editing strategy – Elimination of manual intervention until after estimates are available – Allocation of resources based on macro quality indicators and micro level scores 9

10. Macro editing: Main developments (Continued)  WP.17 – Shared, generic corporate strategies, methodologies, and common metadata framework – Methodology for top down approach – Methodology for measuring quality and measures for quality – Score functions to measure impact 10

11. Selective editing: Main developments  WP. 18 (Spain) – Theoretical framework for selective editing as an optimization problem – Minimize expected workload subject to minimal expected error on the aggregates – Linear constraints – computationally easier, suitable when timeliness is an issue – Quadratic constraints – wider error bounds, more units are marked for review 11

12. Selective editing: Main developments SELEKT tools at both Statistics Sweden and ONS –Scores based on suspicion, potential impact on the outputs –Need “expected” values, final data from previous cycle  WP.19 (Sweden) –Prioritize using expected effects on the outputs –“Expected “ values using time series or cross-sectional data –Different levels of data edited concurren tly 12

13. Selective editing: Main developments  WP.20 (UK) – Selective editing as part of an overall efficient editing strategy – Assess impact on quality of changes to edit rules prior to using SELEKT – Suspicion based on traditional edit rules or test variables 13

14. Points for discussion Using a software tool and/or scores for guiding macro editing operations and/or selective editing has benefits: standardizes review process, can be used for several surveys, and provides overall cost benefits. – How are agencies incorporating cost/resources savings into the survey process? – How are agencies planning on maintaining these tools/systems given the complexities of the metadata, constraints, variable mappings, expectation models, and hierarchies as surveys and output requirements evolve (particularly business surveys)? 14

15. Points for discussion (Continued) –What is the effect on other survey activities? –How is the overall macro editing and/or selective editing process contributing to the overall data quality? –How can the effect on data quality be measured? 15

16. Points for discussion When macro editing and/or selective editing tools are applied to periodic survey data, subject matter experts may acquire further knowledge about the survey from the macro editing and/or selective editing operations: –How can this knowledge be used to improve the survey process? –How can we incorporate this knowledge to get insight into how to reduce errors and/or enhance micro editing for the next cycle? 16

17. Points for discussion In both macro editing and selective editing scores there is the need for estimates of anticipated values. –How to model “expected” values needed for computing measures of suspicion and/or impact? –How do we choose the appropriate domains for computation of “expected” values in order to achieve relevancy and accuracy? –What is the minimum number of observations needed to compute these “expected” values within each domain? 17

18. Points for discussion (Continued) –How do we separate model errors for expected values from response errors (for either aggregate expected values or micro expected values) in a production environment? –Are there concerns about potential bias under certain variable distributions that may result from a collection of non-influential units that will not be addressed by selective editing? 18

19. Point for discussion Most statistics may benefit from the use of macro editing and/or selective editing. – What are the agencies specifications for a set of general mandatory guidelines? 19

20. Point for discussion When designing an overall editing strategy, – To what extent should agencies incorporate selective editing and/or macro-editing in their overall editing strategies? – For what kind of data are these strategies suitable? – How can we take into account the fact final data may be used by other users and for different purposes? 20

21. 21 Thank you for your attention! Paula and Maria