European Conference on Quality in Official Statistics Session 26: Quality Issues in Census « Rome, 10 July 2008 « Quality Assurance and Control Programme in the 2011 Population and Housing Censuses in Portugal « 1 Carlos Dias Statistics Portugal

2 « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal « Summary  Introduction;  Control of the processes;  Post Enumeration Survey;  Conclusions. «

3 « « 1. Introduction Due to dimension and complexity, Census are surely the largest statistical operations in the world. Statistical units enumerated in 2001 Census in Portugal:  Buildings ( );  Dwellings ( );  Households ( );  Persons ( ). Fieldwork activities involved around people. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal } 74,3 millions of pages (A4 format): more than 200 tonnes of paper

4 « «  Introduction In Census operations, the correct planning, organization and resource management of its different stages (since preparation to execution and dissemination) is the key factor for its success. It is also important to define: measures of quality control to minimize the sprouting of errors; measures of evaluation and quantification of errors to check the data production. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

5 « « 1. Introduction In this scope, quality control and evaluation are fundamental tasks and are ways to assure that final results will respect the principles of statistical information:  Relevance;  Accuracy;  Timeliness;  Accessibility;  Comparability;  Coherence. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

6 « «  Introduction Challenge to the 2011 Census To assure the implementation of a quality assurance and control programme of both processes and data, in order to guarantee that the productive process is able to detect any inconsistencies and errors in time and for the final results to tend towards unquestionable quality. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

7 « «  Introduction Profiting by the success of the model adopted in 2001, the programme to be implemented in 2011, will be based on two great areas:  Control of the processes;  Post Enumeration Survey. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

8 « « 2. Control of the processes The implementation and results of the Quality Assurance and Control Programme will be oriented and audited by a highly respected university institution in the statistical area. The control of the processes will include implementation of mechanisms for checking and control applied to some key points of the productive process, so that the quality of the data collection and treatment are ensured and under permanent control. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

9 « « 2. Control of the processes For 2011 Census is planned the implementation of evaluation procedures as well as the quality control in the following sub- processes: 2.1. Selection, training and technical evaluation; 2.2. Fieldwork control; 2.3. e-Census system; 2.4. Data processing. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

10 « « 2. Control of the processes 2.1. S election, training and technical evaluation of regional and local participants;  Selection tests for training candidates;  Definition of the training programme;  Technical models for the evaluation of acquired knowledge;  Models for the control of practical training;  Control the training period. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

11 « « 2. Control of the processes 2.2. Fieldwork control Delivery and collection of questionnaires – as a nuclear and decisive task for guaranteeing the Census quality, we foresee the implementation of two types of control instruments:  Individual control - verification and evaluation of the performance of each person involved during the Census process. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

12 « « 2. Control of the processes 2.2. Control of the fieldwork  Alert Indicators System – will control information quality during the enumeration that allows correction of eventual inconsistencies. It consists of monitoring just in time some results (during the enumeration phase) by comparing them with the expected values, and having as reference other information sources (e.g. administrative registers). Whenever deviations are detected it is activated an additional checking procedure in the fieldwork.  Information system on persons;  Information system on dwellings; « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

13 « « 2. Control of the processes 2.3. e-Census System I n 2011, Statistics Portugal plan to implement, for the first time, an on-line response system. Control and evaluation of the quality applied to e- Census system (Internet return of forms) – evaluation instruments will be developed, taking into consideration the coverage checking on the 3 alternatives: web response versus paper questionnaires versus non response. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

14 « « 2. Control of the processes 2.4. Data processing It will be developed instruments allowing evaluation of quality related to:  ICR (Intelligent Character Recognition) processes;  Automatic codification system;  Correction system. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

15 « « 3. Post Enumeration Survey (PES) The PES will start soon after the end of enumeration phase, in an autonomous way and oriented to a sample of dwellings. The PES will not have impact in the 2011 Census databases, and will allow measuring the census data quality, by quantifying errors (both of coverage and content). As happened in 2001, the sample extraction and control will be oriented to an external entity’s responsibility to ensure that the results are properly audited and to strengthen the credibility of this survey. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

16 « « 3. Post Enumeration Survey (PES) The PES is the true instrument of the final measurement of the quality of the 2011 Census, and for this reason, special care should be taken with it, both in preparation and in carrying it out. Objectives:  Evaluation of errors in gross and net coverage of each statistical unit observed;  Evaluation of content errors in variables and their respective modalities for each statistical unit. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

17 « « 3. Post Enumeration Survey (PES) - Main methodology characteristics of the PES  The sampling used in this survey will be multi-stage and will represent every one of the statistical units and the main characteristics of each, at the NUTS 2 level (7 regions);  The collection of data for the 2nd enumeration will be carried out after the collection of the data for the first enumeration is concluded, by a different person and one who is unknown to the person who carried out the first enumeration;  Whereas the evaluation of coverage includes all of the statistical units enumerated in the same area, the content evaluation only includes those statistical units enumerated in both. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

18 « « 3. Post Enumeration Survey – Coverage evaluation « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

19 « « 3. Post Enumeration Survey – Content errors « Content errors  Wrong or inconsistent declarations  Omission of information due to bad performance of the enumerator or incorrect interpretation of the questionnaire  Unfamiliarity, confusion and resistance relatively to the answer  Processing of the information: edition and imputation of missing data Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

20 « « 3. Post Enumeration Survey – Content evaluation « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

21 « « 3. Post Enumeration Survey – Content evaluation « Based on a matrix of this type, the content errors of each variable should be calculated by the formula m=number of modalities of the variable; n= number of answers to the variable in Census and PES Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

22 « « 3. Post Enumeration Survey – Content evaluation « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

23 « « 4. Conclusions Program of control and evaluation of the quality of the Censuses involves complex challenges that will need customized solutions. To assure the implementation of a quality assurance and control programme of both processes and data, in order to guarantee that the productive process is able to detect any inconsistencies and errors in time and for the final results to tend towards unquestionable quality. The implementation and results of the Quality Assurance and Control Programme will be oriented and audited by a highly respected university institution in the statistical area. The sample extraction and control will be oriented to another external entity’s responsibility to ensure that the results are properly audited and to strengthen the credibility of this survey. « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal

24 « « Thank you for your attention « Quality Assurance and Control Programme in the 2011 Censuses Population and Housing in Portugal