SLICE 1.5: A software framework for automatic edit and imputation Ton de Waal Statistics Netherlands UN/ECE Work Session on Statistical Data Editing, 16-18.

Slides:



Advertisements
Similar presentations
Data Quality Assurance and Dissemination International Workshop on Energy Statistics Aguascalientes, Mexico.
Advertisements

Integrated Data Editing and Imputation Ton de Waal Department of Methodology Voorburg Statistics Netherlands ICES III conference, Montréal June 19, 2007.
Data Imputation United Nations Statistics Division (UNSD) 16 March 2011 Santiago, Chile.
CBS-SSB STATISTICS NETHERLANDS – STATISTICS NORWAY Work Session on Statistical Data Editing Ljubljana, Slovenia, 9-11 May 2011 Jeroen Pannekoek and Li-Chun.
Metadata to Support the Survey Life Cycle Alice Born, Statistics Canada Joint UNECE/Eurostat/OECD Work Session on Statistical Metadata (METIS) Geneva,
UNECE Work Session on Statistical Data Editing Vienna April 2008 Topic ii – Editing Administrative Data and Combined Sources.
1 Editing Administrative Data and Combined Data Sources Introduction.
IMSE Week 18 White Box or Structural Testing Reading:Sommerville (4th edition) ch 22 orPressman (4th edition) ch 16.
C++ for Engineers and Scientists Third Edition
Editing of mixed source data for turnover statistics Jeffrey Hoogland (SN) Work Session on Statistical Data Editing (Ljubljana, Slovenia, 9-11 May 2011)
Edit and Imputation of the 2011 Abu Dhabi Census Glenn Hui and Hanan AlDarmaki Statistics Centre - Abu Dhabi UNECE CES Work Session on Statistical Data.
E&I of administrative data used for producing business statistics Vera Costa, Frances Krsinich, Rudi Van der Mescht 2008 UNECE Work Session on Statistical.
United Nations Economic Commission for Europe Statistical Division Applying the GSBPM to Business Register Management Steven Vale UNECE
Role of editing and imputation in integration of sources for structural business statistics Svein Gåsemyr, Statistics Norway Svein Nordbotten, University.
Eurostat Statistical Data Editing and Imputation.
Combining administrative and survey data: potential benefits and impact on editing and imputation for a structural business survey UNECE Work Session on.
The Edit Anders Norberg, Statistics Sweden (SCB) Work Session on Statistical Data Editing Ljubljana, Slovenia, 9-11 May 2011.
DATA MINING FINAL REPORT Vipin Saini M 許博淞 M 陳昀志 M
Automatic Editing with Hard and Soft Edits – Some First Experiences Sander Scholtus Sevinç Göksen (Statistics Netherlands)
New and Emerging Methods Maria Garcia and Ton de Waal UN/ECE Work Session on Statistical Data Editing, May 2005, Ottawa.
Topic (ii): New and Emerging Methods Maria Garcia (USA) Jeroen Pannekoek (Netherlands) UNECE Work Session on Statistical Data Editing Paris, France,
Sander Scholtus and Leon Willenborg Editing and Imputation in the Memobust Handbook.
Review Session #1. Outline Inequalities Exponents Absolute Value.
Recommended Practices for Editing and Imputation in the European Statistical System: the EDIMBUS Project* Orietta Luzi (Istat, Italy) Ton De Waal (Statistics.
HOT TOPIC: How to read an Earning Statement. Earnings Statement: Earnings Statement: Hours & Earnings BIWEEKLY PAYCHECK (STUDENT & CLASSIFIED): MONTHLY.
Topic (vi): New and Emerging Methods Topic organizer: Maria Garcia (USA) UNECE Work Session on Statistical Data Editing Oslo, Norway, September 2012.
Calibrated imputation of numerical data under linear edit restrictions Jeroen Pannekoek Natalie Shlomo Ton de Waal.
1 Example: Solution of Quadratic Equations We want to solve for real values of x, for given values of a, b, and c. The value of x can be determined from.
CBS-SSB STATISTICS NETHERLANDS – STATISTICS NORWAY Work Session on Statistical Data Editing Oslo, Norway, September 2012 Jeroen Pannekoek and Li-Chun.
United Nations Economic Commission for Europe Statistical Division High-Level Group Achievements and Plans Steven Vale UNECE
Implicit Linear Inequality Edits Generation and Error Localization in the SPEER Edit System Maria Garcia U.S. Census Bureau UNECE Work Session on Statistical.
Paolo Valente - UNECE Statistical Division Slide 1 Technology for census data coding, editing and imputation Paolo Valente (UNECE) UNECE Workshop on Census.
“Teach A Level Maths” Vol. 1: AS Core Modules
1 Numbers & Basic Algebra – Math 103 Math, Statistics & Physics.
1.7 “Absolute Value” Absolute Value is always positive!! 7 = 7-7 = 7 **When solving equations or inequalities, you MUST set up 2 separate problems, one.
[title of your research project] [author] INST 381 Fall 2015.
Material Safety Data Sheets Interpreting and Understanding Information on a MSDS.
Evaluating the Quality of Editing and Imputation: the Simulation Approach M. Di Zio, U. Guarnera, O. Luzi, A. Manzari ISTAT – Italian Statistical Institute.
Software Engineering1  Verification: The software should conform to its specification  Validation: The software should do what the user really requires.
Software Engineering Saeed Akhtar The University of Lahore.
Multivariate selective editing via mixture models: first applications to Italian structural business surveys Orietta Luzi, Guarnera U., Silvestri F., Buglielli.
New and Emerging Methods UN/ECE Work Session on Statistical Data Editing Vienna April 21-23, 2008.
Generic Statistical Data Editing Models (GSDEMs) Workshop on the Modernisation of Official Statistics The Hague, 24 November 2015.
7.5 Solving Systems of Inequalities by Graphing Steps Intersecting Regions Separate Regions.
INFO 4470/ILRLE 4470 Visualization Tools and Data Quality John M. Abowd and Lars Vilhuber March 16, 2011.
Automatic Editing Data. A New Version of DIA System Prepared by J.M. Gomez Presented by D.Lorca National Statistical Institute of Spain.
R&D Satellite Accounts in the Netherlands 2008 World Congress on National Accounts and Economic Performance Measures for Nations May 12-17, 2008 Washington.
Ljubljana, 11 Mai 2011UNECE Work session on SDE Topic (vii) New and emerging methods 1 Topic (vii): New and emerging methods Discussion Discussants: Rudi.
Ljubljana, 11 Mai 2011UNECE Work session on SDE Topic (vii) New and emerging methods 1 Topic (vii): New and emerging methods Introduction Session organizers:
Integer Review If you are combining two numbers with: SAME SIGNS YOU ADDTAKE THE SIGN OF THE LARGER NUMBER DIFFERENT SIGNS YOU SUBTRACT TAKE THE SIGN.
Math 71A 4.3 – Equations and Inequalities Involving Absolute Value 1.
Theme (v): Managing change
The user as data detective
6.3 Compound Inequalities
Mathsercise-C Ready? Inequalities Here we go!.
OASUS Spring or Fall YYYY
An R package for selective editing based on a latent class model
United States Department of Agriculture
Solving Linear Systems by Linear Combinations
Data Preparation and Description
Data validation in Statistical Office of the Republic of Serbia
Jeroen Pannekoek, Sander Scholtus and Mark van der Loo
Complete Bellringer
Automatic Editing with Soft Edits
Methodological questions raised by the combined use of administrative and survey data for the French structural business statistics Work session on statistical.
Inequalities: Listing numbers with division
Jeroen Pannekoek, Mark van der Loo and Bart van den Broek
New and Emerging Methods
W-4 View Employees can view their current W-4 on record.
Hands-on GSIM Mauro Scanu ISTAT
Presentation transcript:

SLICE 1.5: A software framework for automatic edit and imputation Ton de Waal Statistics Netherlands UN/ECE Work Session on Statistical Data Editing, May 2005, Ottawa

SLICE 1.5 Several related modules for automatic edit and imputation –Blaise parser: reads and interprets Blaise edit rules –Cherry Pie: localises errors –Module for highly contaminated records: localises errors in records with many erroneous values –Imputation module: imputes missing and erroneous data –AdaptValues: adjusts imputed values so all edits become satisfied

SLICE 1.5  Modules can be used in combination or separately  SLICE 1.5 can handle - categorical, continuous and integer data - positive and negative values - complex edits with inequalities, equations, IF THEN ELSE statements, AND/OR/NOT operators, e.g. IF ((Tax on wages > 0) AND (Activity code = “Chemical Industry”)) THEN ((Number of employees  100) OR (Size  {“Medium”, “Large”}))