S T A T I S T I K A U S T R I A Quality Assessment of register-based Statistics A Quality Framework Manuela LENK Directorate Population Statistics Register based census © STATISTIK AUSTRIA I n f o r m a t i o n e n Wir bewegen

S T A T I S T I K A U S T R I A  Switch from traditional to register-based census in Austria 2011  Compared to other countries transition time is relatively short  Census Test was carried out for 2006  Seven base registers and several comparision registers  Registers are combined using unique identifiers (bPK)  Gathering experience for the register-based census Introduction 2

S T A T I S T I K A U S T R I A  Quality is influenced by (according to the quality report as requested by Eurostat)  Quality of the administrative data source  Existence of an unique key or rather the availability to link a person  Comparision between survey and administrative data  Item imputation  Consequence  Establishing a quality framework for the register based census 2011 – cooperation with WU Vienna Quality Issues in Census Test 3

S T A T I S T I K A U S T R I A  Quality assessment of statistics based on administrative data  Quality Indicator for  Raw Data – data obtained from administrative sources  Each attribute in each register  Whole register (data source)  Census 2011  Attributes of the Final Data Pool Objectives 4

S T A T I S T I K A U S T R I A  Assessment of quality is derived from hyperdimensions (HD) for each attribute in each register.  Documentation  Pre-Processing  External Sources  Imputation Hyperdimensions 5

S T A T I S T I K A U S T R I A  Documentation  Focus is on processes taking place before the data is transfered to Statistics Austria  Data treatment at the source maintaining the register  Reliability of the data source  Quality criteria  Relevance of attributes for the source (e.g. legal foundation)  Availability for certain reference dates  Compatible definitons  Registers are benchmarked using a perfect pseudo register  Informations are extracted from the meta database Hyperdimensions (1) 6

S T A T I S T I K A U S T R I A  Pre-Processing  Process of Data editing from Raw Data to the Edited Data  Analysis of Raw Data  Error corrections  Recoding  Plausibility checks  Quality criteria  Proportion of missing unique keys  Proportion of values out of domain  Item non-response  Automated generation out of database Hyperdimensions (2) 7

S T A T I S T I K A U S T R I A  External Source  Checks accuracy of the data  record linkage with survey data (e.g. Labour Force Survey)  Quality criteria  Consistency of attribute values  For some attributes no external source for comparision exists, thus it is being replaced by  Expert interviews  Rating of accuracy – expert knowledge obtained from working experience  Subjective – should be considered when allocating the weight for this hyperdimension Hyperdimensions (3) 8

S T A T I S T I K A U S T R I A  Each hyperdimension receives a weight in accordance with its relevance for the project.  Within these hyperdimensions quality criteria are determined and quality indicators are derived for each hyperdimension (HD D, HD P, HD E ).  After weighting these dimensions (v D, v P, v E ) we get an aggregated quality indicator q Quality Indicators (1) 9

S T A T I S T I K A U S T R I A  Finally we are able to derive a quality indicator q ij for each attribute in each register  We could derive an aggregated quality indicator for each register too (weighted row sum). Quality Indicators (2) 10

S T A T I S T I K A U S T R I A  Register-based census  Quality assessment within the Census Database  Combination of existing registers to a new data base.  Quality assessment of the Final Data Pool  Census Database including item imputations. Application on linked data sets 11

S T A T I S T I K A U S T R I A  Unique Attributes  attribute exists exactly once in one register and is transfered directly to the CD  q ij = q  j = q Ψj linklink  Multiple Attributes  attribute exists in more than one register. Information from different sources are combined in the CD using certain decision rules – e.g. majority principle (Sex)  Quality indicators for this attribute from different registers are combined to an overall quality indicator q  j (e.g Dempster-Shafer Theory)  Additional check using external data source HD E leads to a corrected quality indicator q Ψ j linklink  Derived Attributes  several attributes are linked to a new attribute  Quality indicator q  j is generated by using the quality indicators of the input attributes  Additional check using external data source HD E leads to a corrected quality indicator q Ψ j linklink Assessment of the Census Database (CD) 12

S T A T I S T I K A U S T R I A  Census Database including item imputations  Hyperdimension Imputation (HD I )  Quality indicators depend on methods of imputation  Shall be generated with appropriate valuation methods  The weight for this hyperdimension is approximated by the proportion of imputation Quality assessment of the Final Data Pool 13

S T A T I S T I K A U S T R I A Process-oriented Quality Framework back back 14

S T A T I S T I K A U S T R I A Conclusion  Framework should act as a road map  Next milestones are  Establishing check lists (questionnaires)  Specify how to derive the quality indicators for each hyperdimension and how to aggregate them  General approach  Application to other projects is possible 15

S T A T I S T I K A U S T R I A  Additional Information  Quality Framework  Fiedler/Schwerer/Berka/Humer/Moser: Quality Assessment for Register based Statistics in Austria  Register-based Census  Outlook 16