1. Data Capture Process Stages
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

2. Overview Objective Major Process Stages Factors/Considerations
Document Scanning operations
Recognizing operations
Verifying operations
Coding Assistance
Factors/Considerations
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008 2

3. Objective
To provide an overview of the major process stages associated with optical data capture and quality assurance considerations
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

4. Major Process Stages Document Scanning Recognizing Verifying
Scanner Speeds are dependent on process chosen
Recognizing
Recognizing is dependent on the sophistication of the recognition engine
Automatic Electronic Verification
Major Process Stages
Verifying
Non-Successful Electronic Verification
prepare data in a form suitable for entry into computer
Coding Assistance
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

5. Document Scanning Stage
Key feature: scanning speed
Scanning speed will be determined by:
Quality of the scanner machines
Size of non-drop out color
Paper quality, cleanness & weight
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

6. Recognizing Stage Accuracy of interpretation will be determined by:
The recognizing process is to interpret images
Accuracy of interpretation will be determined by:
Recognition engine/memory dictionary;
Configuration threshold
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008 6

7. Verifying Stage
Processing can be in geographic order or in random order:
Automatic electronic verification
Non successful electronic verification: Need to compare the value of the interpreted image with the real image of the form.
Image manipulation
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008 7

8. Verifying Stage (cont.)
Image Manipulation:
Electronic questionnaires can be sent to specialist operators then back to the original operator if necessary (in some cases, the same questionnaire can be worked on simultaneously by two or more persons)
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

9. Coding Assistance Stage
Process in which census questionnaire entries are assigned numerical and/ or alphanumeric values
Objective is to prepare data in a form suitable for entry into computer
Done by setting up possible responses to each question in the census questionnaire
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

10. Factors to be considered
Questionnaire Design & Preparation
Data Collection & Processing Considerations
Field Operation
Staff Training
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

11. Thank You
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

12. Additional material
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

13. Questionnaire Design & Preparation
Form Design Advise
Consider the number items to be included in a form
Pre-print codes near the place where the box for ticks are located
Considering the speed of the data capture process - it is advisable to use marks or “ticks” as much as possible
Define drop out color properly; use registration marks (allows for quicker recognition)
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

14. Questionnaire Design & Preparation
Form Design Advise
Maintain consistent pattern in which the information to be collected will be located
Do not disturb the visibility of the ticks and marks with titles, labels or instructions
Avoid putting "answers" of one field to another page of the questions;
Avoid using open ended questions
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008 14

15. Questionnaire Design & Preparation
How to Obtain Good Results of Scanning
Select adequate paper quality
Select a reliable printing press
Use appropriate ink, considering drop out color (for the questionnaires paper heavier than 80 grams per square meter can help avoid paper crashes in scanner)
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008 15

16. Data Collection & Processing Considerations
Field Operation
Field Operators should have basic knowledge of the data capture process chosen
Staff Training
A set-up of required training for staff will ensure quality and effectiveness of the data captured
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

17. Field Operation Considerations
Reasons of Error-Reading of OCR:
Bad condition of the form because of dirt, folded, crumple, etc
Unnecessary lines of characters such as points, decorative strokes, hooks, etc
Checking the questionnaires for completeness and consistencies
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

18. Training for Processing Staff
Installation and set-up break-down of equipment (e.g. hardware and software)
Basic software knowledge
Scanner operating procedures
Troubleshooting (e.g. solutions to common problems/issues)
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008

19. Control steps
Control steps should be taken if the information image is partial or no information to assure the quality of generated files
Value Checking Steps
Control for Blank
Missing Questionnaire
Value Checking Steps: Verify that the information captured is the same with the questionnaire
Control for Blank: If the information is blank, what type of control must be taken
Missing Questionnaire; Make sure that the entire and all questionnaires are scanned completely, no missing and no duplication as well
Therefore control procedures including to produce control tables to compare with manual work
UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving Bangkok, Thailand, September 2008