1. Uganda – October 2009 Census Data Collection & Processing John Gomersall

2. Agenda  Census data collection – Enumeration  Processing – Data Capture phase Decisions on the methods for both

3. Chronologically, enumeration comes before data capture & we will consider them in that order. However, decisions on one will inevitably influence the other. Most obvious example of this is the design of the Questionnaire. What is possible & what is practical? - cost, infrastructure, skills

4. Choose a Method for enumeration Options Field based Remote

5. Choose a Method for enumeration Options Field based EnumeratorSelf Remote Internet / Telephone

6. Choose a Method for enumeration Options Field based Enumerator PaperPDA Self Remote Internet / Telephone 1 ST decision made !!

7. Data capture Enumeration Questionnaire Design Including: Census Preparations CSPro Census Results

8. Data capture ? Not so good news …… No panacea ! Good news …… technology can help & the choices of technology – are relatively few. (Not to be confused with the potential number of suppliers of these technologies - which may be many !)

9. Choose a Method for enumeration Data Capture Methods - Sub Saharan Africa Field based Enumerator Paper PenPencilPen & Pencil Second decision ???

10. Choose a Method for Data Capture Data Capture Methods Sub Saharan Africa Field based / Enumerator / Paper Checking / Coding (?) / Organising Data Capture KeyingScanning + Keying

11. Considerations:  Timeframe for producing results  Anticipated condition of Questionnaires  Form preparation requirements  Accuracy ?? If Questionnaires are really bad, &/or sufficient low cost keying capacity is available and manageable to meet acceptable timescales ……… keying may be the best method ! Keying or Scanning + Keying ?

12. Choose a Method for Data Capture Data Capture Methods Sub Saharan Africa Field based / Enumerator / Paper Checking / Coding (?) / Organising Data Capture Keying Timescales / Questionnaire “Preparation” OK Scanning + Keying Decision made !!!

13. Scanning + Keying Options 4

14. Scanning + Keying Options 3

15. ICR OMR

16. Main Pro Characteristics:  OMR   Quickest, simplest, most accurate data capture method for well completed forms   Enumerator only needs to mark with HB Pencil + Eraser   Appropriate level of technology for current IT resources   Cost competitive solution – known / one time  ICR   Questionnaire design & Printing  Potential for technology transfer 5

17. Ethiopia 2007

18. Sudan 2008

19. MALAWI 2008

20. Scanning + Keying Options 2

21. The alternatives:

22. In both cases, an image of the form will be captured, and the information which has been filled on the questionnaire will be automatically collected. using OMR (Optical Mark Reading) & / or ICR (Intelligent Character Recognition) in either case, this will be followed by: in either case, this will be followed by: validation, and key correction of “irregularities” validation, and key correction of “irregularities”

24. So, scanning is not the worry !!

25. So what is the worry ? The worries are that: having captured the images - there may be problems with extracting the data from those images, and following from this integrity of the data – is the form in the correct EA; is all data for a household together. So what are the considerations – how do we minimise the worry? So what are the considerations – how do we minimise the worry?

26. Data Capture Methods Sub Saharan Africa Field based Paper EnumeratorData Capture KeyingScanning + Keying Scanning Extracting the data

27. Priorities:  Identify an appropriate data capture solution – hardware + software.

28. Identify an appropriate data capture solution for you. That may be:  Procuring Tools – if you have experienced in-house software development expertise, or  Partnering with an experienced supplier with a customised, tested software product, with effective on-site / remote support. (Kenya – with USBC, Ethiopia, Sudan, Malawi – with DRS, Egypt, Morocco) Basically, –the software needs to work? –this means that in addition to OMR / ICR recognition – does it provide elements for batch control, logic validation, key correction, quality control? –Will the required support & guidance be available? There are partners out there who have the technology!

29. Priorites  Implement effective processes & logistics  Identify an appropriate data capture solution.

30. Implement effective processes & logistics: Enumerator Training & Job definition Supervisor Training & Job definition Questionnaire flow Checking & Batching – by EA Preparation for Scanning – Separation Define scanning checks In good time - process a comprehensive Pilot

31. Priorities 1.Make it easy for the enumerator to do a good job 2.Implement effective processes & logistics 3.Identify an appropriate data capture software product and supplier.

32. 3. IT 2. CSO 1. ENUMERATORS & SUPERVISORS

33. Make it easy for the enumerator to do a good job. Unfortunate realities !  Scanning cannot make a “bad form” good  Bad forms will make any scanning technology look ineffective The message to remember ! The main concern should be the training of Enumerators and Supervisors to provide Questionnaires in good condition & which have been well completed --- then the scanning will take care of itself !

34. Questionnaire Considerations: Paper – Environment SizeScanningDesign Barcode/unique identifier Ease of completion The single most important factor for accurate data capture is to make sure ‘the forms are filled in correctly & are returned in good condition’ Data Capture

35. Considerations – Data types Data Capture Tick box data Constrained handwritten data Unconstrained handwritten data Barcode & Unique printed number

36.   It is easier to train 30,000 enumerators to mark in pencil   than it is to train them to correctly write, & accurately position characters – in ink! 5 Data Capture Methods - Sub Saharan Africa Field based Enumerator Paper PenPencilPen & Pencil

37. Priorities 1.Make it easy for the enumerator to do a good job 2.Implement effective processes & logistics 3.Identify an appropriate data capture software product and supplier.

38. Data Capture Methods Sub Saharan Africa Field based Paper EnumeratorData Capture KeyingScanning + Keying Scanning Extracting the data Validation & Key Correction

39. Validation & Key Correction I The scanning process will capture what the enumerators have filled in on the Questionnaires – mistakes included! The purpose of running the validation is to identify where the enumerator has not completed the Questionnaire according to the rules set out in the Enumerator’s manual, and to flag those errors for potential key correction. These errors will include those questions that are: expected to have been filled in but are blank partially marked multi marked

40. & a key correction screen something like this Operator enters “2” as it can be clearly seen that the correct age is 27

41. Validation & Key Correction II   Following validation and verification/correction, all captured data will be exported to CSPro, or other processing software. The end of the beginning !

42. Data Capture Summary/Recommendations –Choose the method that best suits your organisation/infrastructure and is the best fit to your requirements –If choosing a new method plan well ahead, consult with others & test with a pilot –Plan for your staffing and logistics –If you wish to use your data spatially you will need to geo-reference your data –Choose a partner with experience and one that will support your process

43. Thank you for listening