Presentation is loading. Please wait.

Presentation is loading. Please wait.

Data Processing of the 2010 Population and Housing Census 15-19 September 2008, Bangkok, Thailand National Statistical Office, Thailand.

Published byBrianna Thornton Modified over 8 years ago

Similar presentations

Presentation on theme: "Data Processing of the 2010 Population and Housing Census 15-19 September 2008, Bangkok, Thailand National Statistical Office, Thailand."— Presentation transcript:

1 Data Processing of the 2010 Population and Housing Census 15-19 September 2008, Bangkok, Thailand National Statistical Office, Thailand

2  Hardware & Software of ICR System  TELEform / ABBYY Functions  Step of ICR System in NSO  Specific questionnaires for ICR System CONTENT DATA CAPTURING

3 NSO was firstly used ICR System to process the Population Census questionnaires in 2000 by scanning the 16 million households (16 million Forms) which spent only 8 months to process the raw data instead of 18 months by using Key in Data System. ICR for The Population Census 2000 DATA CAPTURING

4 TELEform Hardware & Software System in 2000 TELEform Hardware System TELEform Software System  NetServer for TELEform Server (1)  NetServer for Database Server (1)  Reader Modules Workstations (21)  Verifier Modules Workstations (55)  Scanner Control Workstations (6)  Scanner Fujitsu M4099D (6) TELEform 6.2 Elite Enterprise Edition Components :  TELEform Designer  TELEform Reader  TELEform Verifier DATA CAPTURING

5 ICR System in NSO (Thailand) can be divided into 2 parts : ICR System in 2003  TELEform Software System  ABBYY Software System DATA CAPTURING

6 NSO hired ABBYY Software to process about 25% of The Agricultural Census 2003 questionnaires that were totally 5.8 million households (24 million forms). ICR for The Agricultural Census 2003 DATA CAPTURING

7 TELEform Hardware & Software System in 2003 TELEform Hardware System TELEform Software System TELEform 6.2 Elite Enterprise Edition Components :  TELEform Designer  TELEform Reader  TELEform Verifier  NetServer for TELEform Server (1)  NetServer for Database Server (1)  Reader Modules Workstations (21)  Verifier Modules Workstations (30)  Scanner Control Workstations (6)  Scanner Fujitsu M4099D (6) DATA CAPTURING

8 ABBYY Hardware & Software System in 2003 ABBYY Hardware System ABBYY Software System ABBYY FormReader 6.0 Enterprise Edition Components:  Form Design  Administration Station  Recognition Station  Correction Station  IBM Server X Series 225 (1)  Correction Station (1)  Verifier Modules Workstations (25)  Scanner Control Workstations (4)  Scanner Fujitsu M4099D (4)  Storageflex LT707 (1) DATA CAPTURING

9 TELEform & ABBYY Functions

10 TELEform / ABBYY Designer Function To create template form by fix field boxes on questionnaire. Questionnaire Template DATA CAPTURING

11  To evaluate the questionnaires  Export the corrected questionnaires to a data file  Send the unclear questionnaires to TELEform/ABBYY Verifier Function for correcting and transferring the corrected questionnaires to a data file  Store scanned images TELEform Reader / ABBYY Administration Function DATA CAPTURING

12  To correct questionnaires that contain mismarked or illegible fields  The corrected questionnaires are automatically exported to a data file TELEform / ABBYY Verifier Function DATA CAPTURING

13 Scanning speed support A7 to A3 paper sizes  Simplex is provided 90 papers / minute. (A4 portrait)  Duplex is provided 180 images / minute.(A4) NSO questionnaires projects are mostly printed with A3 (297 x 420 mm.) paper sizes. Functions Speed FunctionsEstimated Speed (sheets/minute) Scanner 45 Reader17 Verifier 5 DATA CAPTURING

14 Step of ICR System in NSO

15 Scan and Forms Distribution : The questionnaires are scanned in each Block / Village and created Multi Page Image Files. Step of ICR System in NSO Forms Evaluation : The questionnaire images are evaluated. The corrected questionnaires which skipped Verifier Workstations and directly exported to Database server. DATA CAPTURING

16 Forms Verification : The unclear questionnaires are needed to review and corrected it in Verifier Workstations before transferring to Database server. Step of ICR System in NSO (cont’) Data Export :  Link a data file from Database server to IBM Mainframe System  Store Scanned image files to CD. DATA CAPTURING

17 Scan & Forms Distribution Questionnaire Scan Image File DATA CAPTURING

18 Forms Verification and Data Export Verify Storage (Images files) Export data for processing CD DATA CAPTURING

19 Input – Output of ICR System Questionnaire ICR DATA CAPTURING Ascii files Image files

20 ICR Linkage System ABBYY Software Transfers Storage (HD 880 GB) Mainframe HP Serve r COMPAQ Server Processing (Editing & Reporting) Scanners 4 unit IBM Serve r controller PC 6 unit PC 4 unit controller 30 unit Verifications 21 unit Readers Correction 1 unit station 25 unit Verifications - Administration - Export - Recognition CD Questionnaires - Backup Data - Software - Database S Scanners 6 unit Questionnaires TELEform Software

21 DATA CAPTURING Specific Questionnaires for ICR System

22 Specific questionnaires for ICR System  The questionnaires must be designed and printed in quality of paper, specific colour answer field boxes (blue, green, red)  To record the questionnaires should be used at least 2HB pencil  To distribute and collect as well as return questionnaires should be done with caution. DATA CAPTURING

23 ICR Benefits  Reduce Cost  Reduce Time  Efficient Data Capture  Increase Data Accuracy DATA CAPTURING

24  Strictly designed questionnaires : Paper, size, color, figure and answer field boxes  Record questionnaires should be fixed pencil and handwriting  Distribution and return questionnaires should be carful Major Problems Encountered in 2000 Census DATA CAPTURING

25 DATA EDITING EDITING & TABULATION

26 DATA PROCESSING STEP Machine Edit Listing Error? Checking Error & List Data Comparing with questionnaire images Table Checking Yes No Questionnaires ICR Accept? NoYes Validate Data (Manual, Cold deck, Hot deck) Tabulation & Report Tab/Report

27  Possible code: Check in each field which the out-of-range fields values is shown in asterisk (*) code.  Validity: Check characteristics of the message structure. DATA PROCESSING STEP Editing Process

28  Consistency:  Imputation: Automatic editing programs. DATA PROCESSING STEP Editing Process (cont’) Check inconsistent values within record and across record. Messages are shown the related conditional codes. All error is printed in continuous paper forms to be considerated and validated by subject matter until no messages error found.

29  Tabulation: DATA PROCESSING STEP Tabulation Process Report summary data which can be processed after data completely cleaned for subject matter to analyze the results of output.

30 DATA PROCESSING STEP Mainframe : IBM Multiprise 2000 Model 206 1.1 Operating System - OS/390 v.2 release 8 1.2 Compiler - PL/I 1.3 Statistic Program - Base SAS 1.4 Application Development Tools - Performance Reporter for OS/390

31 DATA PROCESSING STEP Personal Computer (PC) 2.1 Operating System - Windows XP 2.2 Package - MS Office 2003 - MS Studio v.6 (Visual FoxPro) - SPSS - CSPro 3.3

32 THANK YOU FOR YOUR ATTENTION

Download ppt "Data Processing of the 2010 Population and Housing Census 15-19 September 2008, Bangkok, Thailand National Statistical Office, Thailand."

Similar presentations

MICS4 Data Processing Workshop Multiple Indicator Cluster Surveys Data Processing Workshop Overview of Data Processing System.

MICS4 Data Processing Workshop Multiple Indicator Cluster Surveys Data Processing Workshop Overview of Data Processing System.
Slide 1Slide Slide 1 International Conference on Establishment Surveys III Montreal June 18-21, 2007 United States Department of Agriculture National Agricultural.

Slide 1Slide Slide 1 International Conference on Establishment Surveys III Montreal June 18-21, 2007 United States Department of Agriculture National Agricultural.
Managing data using CSPro

Managing data using CSPro
Harvard Center for Population and Development Studies1 Census Editing and the Art of Motorcycle Maintenance Michael J. Levin Center for Population and.

Harvard Center for Population and Development Studies1 Census Editing and the Art of Motorcycle Maintenance Michael J. Levin Center for Population and.
INTRODUCTION ABOUT OMR. INDEX  Concept/Definition  Form Design  Scanners & Software  Storage  Accuracy  OMR Advantages  Commercial Suppliers.

INTRODUCTION ABOUT OMR. INDEX  Concept/Definition  Form Design  Scanners & Software  Storage  Accuracy  OMR Advantages  Commercial Suppliers.
Commercial Data Processing Lesson 3: Data Validation.

Commercial Data Processing Lesson 3: Data Validation.
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data.

UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data.
AUTOMATIC DATA CAPTURE  a term to describe technologies which aim to immediately identify data with 100 percent accuracy.

AUTOMATIC DATA CAPTURE  a term to describe technologies which aim to immediately identify data with 100 percent accuracy.
Brief Overview of Data Processing of Afghanistan Household Listing, Pilot Census Results, Population and Housing Census and NRVA Survey Brief Overview.

Brief Overview of Data Processing of Afghanistan Household Listing, Pilot Census Results, Population and Housing Census and NRVA Survey Brief Overview.
The 8 th ECO National Focal Points on Economic Research and Statistics ( 21-22 April 2011, Baku, Azerbaijan) Country Report of the I.R. Iran Statistical.

The 8 th ECO National Focal Points on Economic Research and Statistics ( April 2011, Baku, Azerbaijan) Country Report of the I.R. Iran Statistical.
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data.

UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data.
UNSD Census Workshop Day 2 - Session 6 Data Capture: Optical Mark Recognition Andy Tye – International Manager DRS are Worldwide specialists in data capture.

UNSD Census Workshop Day 2 - Session 6 Data Capture: Optical Mark Recognition Andy Tye – International Manager DRS are Worldwide specialists in data capture.
Census Data Capture Challenge Intelligent Document Capture Solution UNSD Workshop - Minsk Dec 2008 Amir Angel Director of Government Projects.

Census Data Capture Challenge Intelligent Document Capture Solution UNSD Workshop - Minsk Dec 2008 Amir Angel Director of Government Projects.
National Statistical Office, Thailand 2-6 December 2013, Hanoi, Viet Nam Census Evaluation.

National Statistical Office, Thailand 2-6 December 2013, Hanoi, Viet Nam Census Evaluation.
UNSD Census Workshop Day 2 - Session 6 Data Capture: Optical Mark Recognition Andy Tye – International Manager DRS are Worldwide specialists in Census.

UNSD Census Workshop Day 2 - Session 6 Data Capture: Optical Mark Recognition Andy Tye – International Manager DRS are Worldwide specialists in Census.
Manual Data Processing of Census Data 2004 Population and Housing Census Statistics Sierra Leone Thekeka Moses Conteh Sierra Leone.

Manual Data Processing of Census Data 2004 Population and Housing Census Statistics Sierra Leone Thekeka Moses Conteh Sierra Leone.
The Core Welfare Indicators Questionnaire: A CWIQ Option for Monitoring Poverty Reduction Strategies.

The Core Welfare Indicators Questionnaire: A CWIQ Option for Monitoring Poverty Reduction Strategies.
Copyright 2010, The World Bank Group. All Rights Reserved. PROCESSING, Part 1 Data capture, editing, imputation and tabulation Quality assurance for census.

Copyright 2010, The World Bank Group. All Rights Reserved. PROCESSING, Part 1 Data capture, editing, imputation and tabulation Quality assurance for census.
1 Use of scanning technology for data capture ICR System (Intelligent Character Recognition) Information and Communication Technology Center National Statistical.

1 Use of scanning technology for data capture ICR System (Intelligent Character Recognition) Information and Communication Technology Center National Statistical.
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data.

UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data.

Similar presentations

Ads by Google