1. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning and Management United Nations Statistics Division

2. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Reference Material for Presentation 1.Handbook on Census Management for Population and Housing Censuses, United Nations Publication, Sales No. E.00.XVII.15.Rev.1, New York, 2001 2.Principles and Recommendations for Population and Housing Censuses, Revision 2 (2008) -Both are available on the United Nations Statistics Division web-site: http://unstats.un.org/unsd/demographic/sources/census/census3.htm.

3. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Overview of Presentation  Census Planning  Quality Assurance  Evaluation

4. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 PART I: Census Planning – What is it?  Census planning is the core process linking the different phases of the census cycle: Preparation Field operations Processing Dissemination Evaluation  Census planning is the most critical process to conducting a successful census

5. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning (contd).  Aim of the planning process is to ensure that: Each phase is properly resourced and organized The output of each phase is of sufficient quality for all subsequent phases All dependencies between the different phases are identified  Due to long duration of census cycle, planning should not remain static but should be dynamic and continuous …and should be flexible enough to take into account changes that occur along the way….

6. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning (contd.)  Each phase of the census cycle is dependent on a preceding phase:  The quality of the output from each phase has a direct effect on the success of the next phase

7. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning (contd.)  Issues that require careful consideration when planning a census: √a) Specifying the role of the census √b) Specifying the role of Government √c) Setting goals √d) Developing a project plan √e) Monitoring the project plan √f) Developing a budget √g) Drafting an administrative report

8. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning- a) Specifying the role of the census Issues to consider  The census should be considered as part of the larger National Statistical Programme  The prime role of a census is usually to provide an accurate count of the total population for each of the administrative regions of a country  The key strength of a census is the ability to provide data for small geographic areas and for small population groups  When data are not required at this level of detail, other statistical methodologies more cost-effective than censuses should be adopted (e.g., sample surveys)

9. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning- a) Specifying the role of the census  The census serves other important roles as well: Essential for public administration in ensuring equity in distribution of wealth and government services in the areas of education, health, etc Used for delineation of electoral districts at national and local levels, and for provision of basic headcount for elections Serves as sampling frame for entire national household survey program Generates population benchmarks for use by rest of national statistical system (e.g., by household sample surveys), and allows for production of inter-census projections Census results used as a basis for research and analysis to inform evidence-based policy making

10. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – b) Specifying the role of Government To provide:  The legal framework for the census which covers: authority of NSO, topics to be included, citizens’ obligations, confidentiality protection, role of other organizations, etc.  Funding for the census  Logistical support for the census

11. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – c) Setting goals  Needs and requirements of stakeholders should be used to establish the census goals, taking into account:  Costs  Data quality considerations  Logistical implications  Need to maintain public cooperation and confidence

12. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – c) Setting goals  Census goals generally revolve around: (a) Topics on which to collect data (b) Confidentiality (c) Timeliness of data release (d) Data quality (e) The nature of the output (g) The total cost of the census  Goals interact with one another. So priorities need to be set as there could be trade-offs, e.g., between what topics can be collected and costs  Once goals are established, they should be communicated to the staff, and appropriate strategies should be devised

13. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – d) Developing a project plan  Once goals have been established and strategies identified to implement them, more detailed planning begins  A census is a large project broken down into a series of related projects that are dependant on one another

14. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – d) Developing a project plan  To understand the interdependencies of the project plan, there is a need to develop a framework with a hierarchical structure  Projects (planning, preparation, field operations, etc.) o Phases (Field mapping, training, evaluation, etc.)  Activities (Enumeration area design, map production, etc)  Tasks (Review previous census methods, procedures and outcomes; Prepare enumeration area design manual)  The plan should also include milestones (Specific points in time at which key outcomes are expected - to measure each project’s progress)

15. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – d) Developing a project plan  Some issues associated with each activity and task that need to be taken into account √Goals √Timing √Resources √Risk management  Planning process involves identification of issues and adopting a consistent approach in dealing with them

16. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – d) Developing a project plan: Risk management  Project plans should also deal with risk management  Risks are all possible events that could occur and have a negative impact on the success of the census  Risks with significant likelihood should be managed explicitly by developing fully detailed plans parallel to the census plan  Risk management is essential because of the importance of the census and the fact that it is an infrequent exercise  The success or failure of the census may depend on the implementation of the plans associated with these risks if they occur

17. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – e) Monitoring the project plan  Developing a good census project plan is important, but not sufficient  The project plan must be monitored closely, and feedback delivered to all levels of management  Results should be reviewed on a regular basis  Most important components to track Time for completing a task Resource (personnel) usage per task Cost per task Milestones: met or off-track

18. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – e) Monitoring the project plan: Gantt Charts Useful tool for planning and monitoring: the Gantt chart

19. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – e) Monitoring the project plan: Gantt Charts  All deviations from project schedule, cost and resources should be analysed and the impact assessed.  In particular, special attention should be given to: Slippage of critical tasks leading up to milestones; Critically late tasks, indicating that the estimated time to complete is later than the planned finish date; An over-commitment of resources in the remainder of the project; Too many tasks appearing to be “nearly complete” (“99% complete” syndrome)  Re-baselining: If it is estimated that a task cannot be completed in the planned time, then the task must be extended on the Gantt chart, and other adjustments made accordingly.

20. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – f) Developing a budget  Census budget must be planned well in advance and cover all known activities  Sufficient resources must be allocated to each phase  Essential to take into account that the census budget is highly cyclical (peaks during enumeration and processing) so expenditures may be “lumpy”  Resource requirements for the dissemination phase need to be realistically assessed - to make sure funds don’t “run out” by the end of the census cycle  Funds allocated and used effectively on planning and preparation can result in savings in all other phases, namely in enumeration and processing operations

21. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – f) Developing a budget  Monitoring the census budget is fundamental  Monitor regularly (quarterly or even monthly) expenditures against funding for each project  Estimates of expenditures for all years of the census cycle should be prepared in advance and reviewed yearly. This allows for timely identification of possible shortfalls to permit appropriate remedial measures.

22. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – g) Drafting an administrative report  What is it? -It’s the census “historical memory”: a report where all census experiences (good and bad) are recorded -It is not a detailed description of the process (documentation)  What is its purpose? -To retain as much as possible the skills and knowledge acquired in developing the census, and use them at the time of the next census  Why is it needed? -Because after census results are released, most census staff usually move to other duties -Because rationale for decisions made during the census process are often forgotten in the years following

23. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – g) Drafting an administrative report  When should it be prepared? -It should commence as soon as the decision to take a census is made and should continue through to the end of the census process  What should it look like? -The structure of the report could be similar to the structure of the project plan or could be modified somewhat – creating group tasks or sub-tasks when appropriate -A record should be kept of resources used (staff years and funds) and of changes to the planned schedule of activities (what changes and why they were made)

24. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Census Planning – g) Drafting an administrative report  How should it used? -To plan subsequent censuses or other large scale statistical activities (i.e., lessons learned for future activities) -To transfer knowledge and share experiences with other countries -To provide feedback to relevant donors and development partners

25. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 PART II: Quality Assurance  Due to the size and complexity of census operations, it is likely that errors may arise at any stage of the census  To minimize and control errors, it is good practice to devote a part of the budget to quality assurance and control programs

26. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 About Quality Assurance  The objective of quality assurance is to provide standards and controls so that decisions can be taken quickly to correct or change census operations as they take place  There is no single quality assurance system that can be applied to all censuses. They need to be tailored specifically for each census and for each stage of the census operations

27. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Dimensions of Quality Assurance  Attributes of quality include: Major Attributes √relevance √cost √timeliness √data accuracy Other Attributes √accessibility √interpretability √coherence  Achieving a quality outcome is essentially about balancing relevance, cost, timeliness, and accuracy. For example, timeliness is typically involved in a trade-off against accuracy.  Relevance is usually determined early when topics and output are determined

28. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Measuring Quality  The 4 major attributes – relevance, cost, timeliness and data accuracy are inter-linked. Data of higher accuracy can be obtained for higher cost, while acceptable timeliness increases the relevance of the data  Deficiencies in quality are usually the results of deficiencies in the process rather than individual actions of the staff. For example, insufficient training of staff may cause them to generate systematic errors that could have been avoided  Key to achieving a quality outcome is to regularly measure the cost, timeliness and accuracy so that the process can be improved - using the Quality Assurance Circle (next slide)  Quality is relative and based on what is acceptable, rather than an absolute concept of achieving perfection

29. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Measure Quality Implement Corrective Action Identify Root Causes of Problem Identify Most Important Quality Problem Quality Assurance Circle

30. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Quality Circle Dependency Chart

31. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Quality Assurance – Some Fundamentals  Staff (of all levels) engaged in the processes are in a good position to identify problems and suggest improvements  Quality therefore relies on: Active encouragement by management to involve staff in identifying and resolving quality issues Established and documented procedures Systems to monitor outcomes  Managers play a key role in achieving quality by: Establishing a culture of focusing on quality Giving staff responsibilities that allow them to achieve Ensuring that staff understand the philosophy of quality Providing the opportunity for staff to contribute

32. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Quality Control versus Quality Assurance  Quality Control: means finding errors and fixing them Relies on ability to find most errors Can add significant cost Fixing errors can itself introduce other errors Puts the responsibility in the hands of the inspector  Quality Assurance (Continuous Quality Improvement): has an emphasis on improving the process rather than just fixing the errors Recognizes there will be errors in the process Aims to improve the process as it proceeds Gives staff a responsibility in improving the process

33. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Examples of Activities to Assist in Assuring Quality  Testing the form design -Involving the public to learn how they understand the questions being asked (e.g., focus groups) -Involving stakeholders to ensure the results obtained are as expected -Involving the data processing team to ensure the form works with the processing systems -Involving the subject matter specialists to ensure the questions are being answered as expected  Testing the field operations -How are the procedures implemented? -Does the training provide staff with sufficient skills? -What suggestions do staff have to improve the process?  Testing the data processing -Do the systems perform as expected? -How well do the documented procedures work? -How accurate is the data obtained during the test?  It is acceptable for tests to fail - the purpose of testing is to learn and improve!!

34. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008  All aspects of the census program should be evaluated – this includes all operations of the census not just the accuracy (coverage and content) of resulting census data.  Evaluation may be considered as the last stage of the census cycle or the first step in the next census cycle  Evaluation of the coverage and content of the census data can be undertaken through: Post-enumeration surveys (mainly to measure extent of under or over-coverage of the population) Comparing the census results with similar data from other sources such as surveys and previous census, and by using analytical methods (mainly to assess the degree of errors in the content and to analyze the basic distributional properties) PART III: Evaluation

35. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Evaluation of Census Operations  Evaluation of operations may include every aspect of the work from initial planning and user consultation through to final dissemination and analysis  Changes to any census process may have an impact on final data quality  Changes to the enumeration process and processing systems (coding, edits, etc.) should be closely monitored  Strategies to monitor identified points of weaknesses should be developed through the evaluation  The results of evaluation should be disseminated widely to data users

36. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Evaluation of Coverage  This aims at estimating the extent to which people have been missed or duplicated  The best way of estimating undercount or over-count is through a post-enumeration survey (PES)  Undercount may result from missing households due to difficult access, people in transit, highly mobile persons, inaccurate mapping, misunderstanding census instructions, etc.  Over-count may result from duplication in overlapping enumeration areas, long enumeration periods, misinterpretation of coverage instructions, etc.  Usually more people are missed than double counted

37. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Evaluation of Coverage – Post-Enumeration Surveys  The PES should be independent from the census, as it provides an independent validation of the census count  The PES must be representative of the whole country and of all population groups, and should be conducted as close as possible in time to the census, without interfering with the census  The census/PES record matching greatly depends on accurate recording of names and addresses during both exercises. Other matching characteristics include age, sex, birthplace and relationships

38. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Evaluation of Data Accuracy  Benchmarks can be established for each variable to assess the contributions of non-response, editing and imputation to the overall data quality.  Tables can be created comparing non-response rates between censuses. The tables can show responses before and after processing (edits and imputations) for each variable. This will show the impact of the processing system on the original responses.  Other data sets (e.g., labour force survey) may give indicative information – for example, benchmarking the census on the size of the labour force

39. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 Evaluation of Data Accuracy  In-depth analysis should include comparisons between censuses and other data from surveys or administrative sources. The other sources need to be based on compatible standards, having comparable concepts and definitions - and should be recent  In many countries, the quality of census data on births and deaths can only be investigated through indirect methods since vital registration systems are sometimes deficient or absent. The indirect methods may include study of the age-sex distributions, stable population analysis, etc.

40. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data editing Doha, State of Qatar, 18-22 May 2008 THANK YOU!