First meeting of the Technical Cooperation Group for the Population and Housing Censuses in South East Europe Vienna, March 2010 POST-ENUMERATION SURVEYS Introductory notes by Gábor Rózsa, key expert of ICON Institute

Theoretical basics: (According to ISO 8404:) The term quality means the „Totality of characteristics of an entity that bear on its ability to satisfy stated or implied needs.” Factors of quality in statistics (Eurostat – Standard Quality Report)

relevance, i.e. measuring the compliance of statistics (data, definitions, classifications) with the actual and potential users’ needs; accuracy, i.e. the closeness between the value finally retained (after collection, editing, imputation, estimation, etc) and the true (but unknown) population value; completeness

timeliness and punctuality, accessibility and clarity, comparability, coherence, cost and burden (not really a statistical factor of quality, still influencing quality). TCG-Vienna

National Statistics Code of Practice Protocol on Quality Management (ONS, 2004): „Knowledge of National Statistics will be sustained by quality measures, regularly published... Producers of National Statistics will provide accessible information on the quality of their outputs using the quality dimensions defined for the European Statistical System.”

Measuring quality in population censuses No large scale enumeration exists without errors Errors may occur in any phase of a census (missed or erroneously included addresses or persons, questions misunderstood, intended or unintended non-responses, coding, data processing) Systematic or random errors

Main aims of assessing reliability of the data: To provide data users with information on the accuracy of data and so possibly the limitations of their usability: Professionally, the identification of the sources of errors aims at finding out more accurate, relevant and economical solutions for the next census; Questions to be answered: in which phase of work did the error occur; missing lists, wrongly formulated questions, unwell-trained field staff, subjective elements of coding, computer programs? Are there population groups more subject to specific errors than others?

Procedures differing in data needs and types of their results, in line with the following options: one source of data only (the census itself) or several data sources used for checking quality; if several sources, whether there is a comparison at individual records level; check of coverage or reliability of answers; gross or net error; comparison of collected data or after data processing?

Post-enumeration survey (PES) Not simply a re-interview after census data collection, but (ideally), as much independent survey from the census as possible, i.e.: Not the same enumerating and supervising staff in the two …; Possibly not exactly with the same forms, but containing the most important census topics;

PES to be carried out on a sample that should be representative in itself of the country and its (major) population groups; Separate (but rather close) in time; Comparison of records of households and persons PES (ctd.)

PES in the IPA beneficiary countries Albania: It has been foreseen to conduct for the first time a post enumeration survey. Apart from planning this survey, INSTAT is having discussion with the competent Ministries to have the Civil Register, and to put side by side the data from the two sources.

PES in the IPA beneficiary countries (ctd.) Bosnia and Herzegovina: Pilot Census will be prepared and carried out in the scope of IPA Within the Pilot Census, quality of updated enumeration areas maps will be checked. After the Pilot Census, a Post Enumeration Survey (PES) will be carried out on the selected sample. The questionnaire will include a limited number of questions. The training of statisticians for the main survey will be provided through PES.

PES in the IPA beneficiary countries (ctd.) Croatia: PES is planned (in the Census law) after the 2011 Census; it would be conducted on a sample of 300 enumeration areas; two-stage sample – 1.5% sample size (including about 25,000 households and housing units and about 70,000 persons). Kosovo: has done some relevant documentation.

PES in the IPA beneficiary countries (ctd.) Former Yugoslav Republic of Macedonia: Post-enumeration survey for statistical control of the coverage and quality of Census data will be conducted from 16 th to 22 nd April The Post-enumeration survey shall be conducted on randomly selected enumeration districts and municipalities and shall cover 0.5% of the total population in the Republic of Macedonia.

PES in the IPA beneficiary countries (ctd.) Montenegro: PES survey will be conducted just after the final census, as we planned with draft law. (It was not tested with the pilot census.) Turkey: No decision yet about this issue.

PES in the IPA beneficiary countries (ctd.) Serbia: Post-enumeration survey (PES) will be conducted immediately after the Census 2011 on a 0.5% random sample of EAs. Coverage errors of population, households and dwellings will be estimated and quality of answers evaluated. Pilot PES 2009 (PPES) was conducted after the Pilot Census. Results of PPES are valuable input for preparation of the PES organisation, instruments, field work, data entry, editing and tabulation.