United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Overview of Census Evaluation and Selected Methods Pres. 1

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Errors  Omissions: missing housing units, households and/or persons during census enumeration  When you miss the whole housing unit, it means all households and persons living in the housing unit will also be missed  Major causes of omissions are: failure to cover whole land area of a country in creating EAs;  Mistakes made by enumerators in canvassing assigned areas

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Errors (contd.) Omissions contd.  The latter problem may be due to unclear boundaries of EAs, faulty maps or coverage error during the pre-census listing exercise  Canvassing errors can be caused by ambiguous definitions of EAs, faulty maps, and laxity of some enumerators  In addition, omissions within EAs can result because all or some of the members of the household were not present at the time of enumeration

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Errors (contd.) Omission contd.  Proxy respondents can inadvertently or deliberately omit some members of a household Duplications  Occur when persons households or housing units are counted more than once  Reasons for duplications include: Overlapping of enumerator’s assignments owing to errors done during pre-census listing and delineation Failure by enumerators to clearly identify boundaries

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Errors (contd.) Duplication contd.  In practice, the number of omissions usually exceeds the number of duplicates  This implies that it is common, for countries to have net under-counts Erroneous Inclusions  This includes housing units, households and persons canvassed in a census while they should have not been or were canvassed in a wrong place

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Errors (contd.) Erroneous Inclusions contd.  An example, is entering in census forms people who died before the census date Gross error  This is the sum of duplications, erroneous inclusions and omissions Net error  This is the difference between over-counts and under- counts

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Errors (contd.) Net error contd.  In measuring net coverage error we note that omissions result in underestimating the population while duplications and erroneous inclusions result in overstatement of the population  Thus net census under-count exists when number of omissions exceeds the number of duplicates and erroneous enumerations  Net census over-count is the converse

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Why evaluate ?  To provide users with some measures of quality of census data to help them interpret the results  To identify types and sources of error in order to assist the planning of future censuses  To serve as a basis for constructing a best estimate of census aggregates, such as total population, or to provide census results adjusted  But…not to criticize the census takers !!

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Data Sources for Census Evaluation Single source of data:  An example is one census whose results can be used for demographic analysis  Interpenetration studies ( see definition below) which are used to evaluate the implementation procedures of a particular census  Multiple sources of data:  In order to do matching studies, there is need to compare two independent sources of data e.g. Census and PES

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Data Sources for Census Evaluation (contd.) Record checks  Census records are matched with a sample of records e.g. from a vital registration system  The relevant respondents to the census questionnaire are traced to the time synchronized with the census Comparison with other surveys  In theory any probability sample of households or individuals can be used to measure coverage and content error in a census if they have identical items using same concepts and definitions

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Data Sources for Census Evaluation (contd.)  Non-matching studies:  Demographic analysis based on two censuses  Census results compared with administrative records and  Census results compared to existing survey results

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Evaluation Methods Demographic Analysis  Results from a census may be compared with data from other demographic systems such as vital registration systems  For example, the cohort component method of demographic analysis uses successive censuses including  life-table survival rates  age-specific rates  age-specific fertility rates and  estimates of international migration

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Evaluation Methods (contd.) Demographic Analysis contd.  Population can be projected forward to the reference date of the second census based on estimated levels of and age schedules of fertility, mortality  The expected population is then compared to the enumerated population in the current census  Yet another method is the comparison of age distributions of successive censuses

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Evaluation Methods (contd.)  Demographic Analysis contd.  Also the cohort survival method which is a regression method can be used, thus, population counts by age from two censuses and deaths by age during the inter-censal period are used to estimate coverage rate  For an overall assessment of quality:  an age pyramid is a standard method  stable population analysis can be undertaken as long as assumptions pertaining to constant fertility and mortality and no migration are met, for countries with declining mortality a quasi-stable model may be appropriate

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Evaluation Methods (contd.) Interpenetrating studies  Method involves drawing subsamples, selected in an identical manner, from the census frame  Each subsample should be capable of providing valid estimates of population parameters  Assignment of personnel (i.e. enumerators, coders, data entry staff, etc.) is done randomly  The method helps to provide an appraisal of the quality of census information and procedures

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Evaluation Methods (contd.) Record checks  Census records are matched with a sample of records from identification systems such as the vital registration system  Sources include: Previous census Birth registrations School enrolment Citizen registration card Immigration registers etc.

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Evaluation Methods (contd.) Record checks contd.  Both coverage and content errors could be measured through the above comparisons To evaluate coverage efficiently the following preconditions are essential:  A large proportion of census population should be covered in record system  The census and record system should be independent from each other  There should be sufficient information in records

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Evaluation Methods (contd.) To evaluate content efficiently the following preconditions are essential:  The record system should contain some relevant items covered in the census such as age, sex, education, relationship, marital status etc.  Definitions of items should be identical between the census and the record system Countries that have used record checks include: Demark, Finland, Norway, Sweden, Taiwan and Canada

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Evaluation Methods (contd.) Comparison with existing household surveys  In theory any probability sample of households or persons can be used to evaluate coverage and content error in a census if:  They have identical items with same concepts and definitions  They are independent from the census  Must have been conducted close to the census date  There should be sufficient identification information to facilitate accurate matching

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Evaluation Methods (contd.) Overview of Post Enumeration Survey (PES)  This is a complete re-enumeration of a representative sample of census population and matching each individual enumerated in a PES with information from the census  An independent probability household survey whose broad objective is to: determine the magnitude of coverage and content error  Coverage error refers to people missed in the census or erroneously included

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Census Evaluation Methods (contd.) Overview of Post Enumeration Survey (PES) contd.  Content error evaluates the response quality of selected questions in a census  Its results can also be used to evaluate the reliability of some characteristics reported in the census  For some countries the results of PES can be used to adjust some census results  Facilitates better interpretation of census results  More discussion of PES is the focus of this workshop

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Strengths and weaknesses of evaluation methods  Single source:  Methods that depend on a single source data provide less insight into the magnitude and types of errors in the census data  The merit is that the methods using such sources do not require additional data to be collected  No need for sophisticated matching although this is also a limitation  They provide a general impression of quality of the census data

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Strengths and weaknesses of evaluation methods (contd.) Demographic Analysis:  Advantage – no additional data is needed to be collected to perform the analysis  Less costly  In statistical offices with sufficient numbers of demographers there is no need for additional staff to do the technical analysis  On the negative side these methods provide less insight into the different contributions of component errors to total error in the census  Quality of sources (Vital Statistics…)

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Strengths and weaknesses of evaluation methods (contd.) Matching methods:  It provide separate estimates of coverage and content error  Prospects of evaluating more characteristics compared to what can be done with non-matching studies Challenges  Calls for high level technical skills including managerial  Matching is expensive

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Strengths and weaknesses of evaluation methods (contd.) Non-matching studies:  Review census results at aggregate rather than unit level i.e. provides only estimates of net census error  Evaluates very limited characteristics such as sex and age distributions Merit  They are relatively cheap compared to matching studies

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Strengths and weaknesses of evaluation methods (contd.) Interpenetrating studies  Gives good idea of different contribution of component errors to total census error  Helps to identify operational stages that contribute to census error, thus identifying procedural limitation in a census  Demerits include:  That it is an expensive operation demanding many field staff, intensive training and close supervision  Relatively complex in designing and implementation

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Strengths and weaknesses of evaluation methods (contd.) Post enumeration survey Merits:  Its results can be used to independently evaluate census coverage and content error, including reliability of selected characteristics collected in a census  Incorporates matching of individuals or units between the census and PES  Its results are generally more reliable than those of the census i.e. it justification for evaluation

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Strengths and weaknesses of evaluation methods (contd.) Post enumeration survey Challenges:  Requires highly skilled field and professional staff  Matching is complex  As it is supposed to be carried out immediately after the census at times there is lack of adequate funds to implement the PES exercise

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción, Paraguay, 3-7 August 2009 Thank You!