Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Results Generated from the questionnaire disseminated prior to the workshop

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 The objective of the questionnaire To better understand data processing activities at the country level To invite country experiences with the goal of providing a forum for further collaboration on the effective use of techniques and methods in data processing To support the development and management of the workshop and future activities To understand what information and technical training is needed on the use of specific daata processing methods

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Data Capture: Methods for census/survey data capture Common methods used for census survey data capture were: manual data entry OMR ORC/ICR Several countries are interested advancing efficiency through the use of PDA’s and have expressed that they plan to use the for the next census round

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Data Capture: Scanners and features used by countries: Fujitsu w/Twain software driver (Egypt) OPScan7 & OPScan10: 500 sheets/hr w/validation OR 5000 sheet per/hr no validation used with scantools software (Kuwait) Kodak ds Digital Science Scanner to 85 ppm (depending on resolution, orientation, feeding, etc.) (Morocco)

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Data Capture: outsourcing of processes With concern to manual data entry, the data capture process is not outsourced. Methods included the use of a database management system such as Oracle along with CSPro where data entered, edited and coded in-house. With concern to OMR & OCR/ICR the data capture process is often outsourced.

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Data Capture: Planned data capture method for next census round Some countries are undecided as of which method to choose OMR/OCR/ICR is planned for use and outsourced (e.g. Iraq) PDA’s are also proposed for use (e.g. Kuwait, Palestine, Qatar)

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Data Capture: Archiving methods and policies used for storing forms Many countries use electronic means for the storage of forms. Some countries store forms both electronically and in hardcopy format. Several countries have laws requiring the storage of forms for a given time ranging from ~10 to ~60 years. Issues raised in the storage of hardcopy forms are that they take up space and may be damaged after a given time period

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Data Editing: Coding for Major Classifications of Occupations All offices use coding for major classifications of occupations, industry and education. Occupation- most use ISCO with several countries using nationally specific systems also Industry- most use ISIC with several countries using nationally specific systems also Education- Most countries use ISCED with several countries using nationally specific systems also Nationality was also mentioned as a major classification in which coding is used.

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Data Editing: Manual or Automated Coding Coding is done manually in most cases with some countries using both manual and automated methods. When automated, the software is developed in house (e.g. Egypt) or through a commercial produced such as Oracle (Lebanon) or developed by a private contractor and configured further by NSO staff (e.g. Morocco) All countries have an editing system as a part of the census/survey processing The dominant error detection systems expressed within the questionnaire were validity check consistency check

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Data Editing: Editing consists of two main stages (Iraq) Stage 1: during data entry sessions validation rules are used within the data entry program; Stage 2: after data entry process completed using additional validation rules by different processes like merging files in order to find out & resolve outliers In many cases manual methods for imputation are used with the following software CSPro, IMPS, SPSS, Oracle. Countries create automated routines using statistical software tools such as SPSS and STATA and batch editing programs attached with the data entry program (CSPro batch editing tool).

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Staff and Training Syria Process: Full-time: Part-time: Data capture ,000 Data coding Error detection Imputation 200 1,000 Qatar Process: Part-time: Data capture 100 Data coding 120 Error detection 120 Imputation 120 Palestine Process: Full-time: Data capture 100 Data coding 120 Error detection 200 Imputation 10 Lebanon Process: Full-time: Data capture Data coding 50 Error detection 50 Imputation - Jordan Process: Full-time: Part-time: Ad- hoc: Data capture , Data coding Error detection Kuwait Process: Full-time: Part-time: Data capture Data coding - 30 Error detection 10 30

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Cont… Training for each step ranges widely across countries Data Capture ~5 days to 2 months Data Coding ~5 days to 2 months Error Detection ~5 days to 2 months Imputation ~ 1 Day Example: Manual Data capture - 1 week Data coding - 3 days Error detection - 3 days Imputation - 1 day Example: OMR Data capture - 12 days Data coding - 5 days Error detection - 5 days Example: PDA Data capture - 14 days (for Buildings & Housing Units Census) & 14 days (for Population Census) Data coding - 7 days Error detection - 7days

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Quality control procedure country examples in relation to the various steps of data processing: Egypt: Training is given to data capture/data entry staff to help better understand questionnaire and fieldwork approaches Jordan: Design special data entry programs for error detection; tables of frequencies for reviewing results, their consistency and accuracy Lebanon: Specially trained team for verification and performance tests with the objective of establishing quality standards of a desired level for each operation (i.e. manual editing, coding, data entry and computer editing) Palestine: Performing checks which are added to editing applications according to the feedback of specific subject matter departments Qatar: Double keying; verification of data keyed from paper are some of the quality control procedures implemented

Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04–08 February 2008 Thank You END