1. Nitai Chakraborty Professor, Department of Statistics, Biostatistics & Informatics, University of Dhaka 1

2. Data Quality Assurance : Concept & coverage Components of data quality assurance steps undertaken in Demographic & Health Surveys (DHS) during field implementation & data processing 2

3. Quality assurance could be understood as total quality management paradigm that examines the survey process at each step of survey design and implementation to improve the output of the survey in terms of its relevance, accuracy, coherence and comparability. The major areas of survey where “ Quality assurance “ protocol should be strictly administered are:  Selection of survey institution  Sampling design & sample size  Designing of data collection instruments & testing the instruments  Recruitment & training of field staff  Data quality control during field implementation & data entry process 3

4. The components of data quality assurance steps in most of DHS surveys during field implementation are: Supervision & monitoring of field work through “ Quality control” teams Monitoring data quality & performance of field staff through Field -Check tables Double entry verification Secondary editing 4

5. All DHS Surveys employed Quality control teams to work in the field for the entire duration of the field work, circulating among all teams, to ensure that :  The field interviewers are observed closely by the senior staff during first few days of field work and give back their immediate feedback to interviewers.  Thoroughly edited all completed questionnaires within a day of the interview or at least before the team leaves the sample cluster. Supervisors and field editors ensure that all questionnaires are thoroughly scrutinized and all errors are tactfully discussed with the interviewer.  In most DHS surveys, one of the supervisor’s responsibilities is to conduct re- interviews with approximately 5 percent of the households covered in the survey. The purpose of the re-interviews is to ensure that interviewers are visiting the selected households and that they do not intentionally leave out eligible household members or misreport their ages so as to reduce their workload. 5

6. Field-check tables are one way of monitoring data quality while the field work is still in progress. All DHS surveys including BMMS used the Census and Survey Processing System (CSPro) software package for data processing. Field-check tables on important aspect of data quality are produced regularly using CSPro data processing application. Use of the field-check tables is crucial especially during early stages of fieldwork when the option remains to retraining of personnel, modify procedures or re-interview problem clusters Each table focuses on an important aspect of data quality These Field check tables are run by the supervisors every week starting after entering the first batch of cluster data. As fieldwork progresses and becomes more settled and routine, the checks become bi-weekly 6

7. Table FC-1: Household response rate: Monitors the performance of interview teams in terms of non-response to the household questionnaire. The supervisor should be informed and remedial action is needed if a team or interviewer shows an exceptional pattern of non-response Table FC-2: Eligible Women per Household One way for interviewers to reduce their workload is to deliberately omit eligible women from the household or to estimate their ages to be either above or below the cutoff ages for eligibility (15-49). Table FC-2 monitors the number of eligible women per household. 7

8. Monitoring data quality with Field -Check tables (cntd. ) Table FC-1 Household response rates Percent distribution of sampled households by result of household interview and household response rate, by interviewer team Result of household interview Team Comple ted (1) HH presen t, no resp. (2) House - hold absent (3) Post- pone d (4) Refuse d (5) Dwelli ng vacant (6) Dwellin g destroy ed (7) Dwelli ng not found (8) Othe r (9)Total Num ber House- hold respon se rate (%)* Team 197.00.00.50.61.30.40.1 0.0100.032598.0 Team 296.51.00.01.00.90.00.50.10.0100.036597.0 Team 387.72.30.03.06.00.10.20.70.0100.034788.0 Team 498.20.2 0.70.20.00.30.00.2100.035298.9 All teams94.80.90.21.32.10.10.30.20.1100.0 138995.4 * HH response rate = (1) / (1+2+4+5+8) * 100 8

9. Monitoring data quality with Field -Check tables (cntd.) Table FC-2 Eligible women per household Mean number of de facto eligible women per household, according to interviewer team Team UrbanRural Number of completed households Number of eligible women in those HHs Mean number of eligible women per HH Target not met Number of completed households Number of eligible women in those HHs Mean number of eligible women per HH Target not met Team 146651.4186730.85 Team 21722431.412142251.05 Team 31391581.14 821191.45 Team 41972361.201201120.93 Team 51161311.13 1611901.18 All teams6708331.24 6637191.08 Note: Number of women expected per HH is country-specific and defined in the sample design (it usually differs by urban/rural areas). The target is the minimum mean number of de facto eligible women per HH expected, and should be 80% of what was expected at the time of sample design. Thus, if 1.2 women per HH was estimated at the time of sample design, teams should be finding a minimum of 0.96 women per HH. Country-managers should provide data processors with the country-specific targets 9

10. Table FC-3 Age Displacement Examines whether interviewers are intentionally displacing the age of young women from the eligible range (15 and over) to an ineligible age (14 and under). An Age Ratio less than 100 indicates a deficit of woman 15 years old compared with those 14 and 16 years old and might indicate intentional displacement Table FC-4 Similar to FC-3 this table whether interviewers are displacing women over the age eligibility boundary, i.e. from ages less than 50 years old to ages 50 and over. An Age Ratio less than 100 indicates a deficit of women 49 years old compared with those 48 and 50 years old and might indicate intentional displacement 10

11. 11 Table FC-3 Age displacement Number of all women 12-18 years listed in the household schedule by single years of age and age ratio 15/14 according to interviewer team Age of women Team12131415161718Total Age ratio (women 15/ women 14) Targe t not met Team 11011 8879640.73 Team 211 1298107680.750.78 Team 312 111311 9791.18- Team 41216135687670.38 All teams455047353336322780.74 Note: Target is an age ratio of women age 15 / women age 14 > 0.8 * All women = de facto + de jure

12. T ABLE FC-5: Birth Displacement Some interviewers intentionally displace the birth dates of children from the fourth or fifth year to the sixth year before the year of the survey, so as to decrease the length and difficulty of their assigned interviewing task. This practice seriously undermines the quality of the data. Field-check Table 5 measures the performance of interviewers regarding displacement of births from calendar years after the cutoff date ( say January 2004) to before the cutoff date. If significant displacement has occurred, the birth year ratio will be found much lower than 100, which is the observed ratio when a smooth change in the number of births is observed from the year before the cutoff (2003) to the year after the cutoff (2005). 12

13. 13 FC-5 Birth displacement Number of births since [2004] by year of birth and birth year ratio [2004/2003], according to interviewer team (based on births of all women) Team Year of birth Total Birth year ratio (2004/200 3) Target not met 2000200120022003200420052006200720082009 Missi ng Team 148465247353836 35150 388 0.74 Team 2403747453841332737121 358 0.84- Team 3364741502831303531131 343 0.56 Team 4454351 2649334328141 384 0.51 All teams169173191193127159132141131543 1,473 0.660.72 Target=0.8

14. T ABLE FC-7: Completeness of Date/Age Information for Births One of the main objectives of the survey is to estimate mortality rates for different age groups of children. This is why data are collected on the age at death of deceased children. Interviewers are required to record at least an approximate age at death for all deceased children. Field-check Table 7 monitors the performance of interviewers regarding birth date completeness. The table is divided into two parts, one for surviving and one for deceased children, since information about deceased children is typically less complete. 14

15. 15 Table FC-7L Birth date reporting: Living children Percent distribution of surviving births by completeness of date/age information by interviewer team LIVING CHILDREN Completeness of reporting Team Year and month of birth given Year and age Year of birth only Age onlyOther No dataTotalNumber Team 194.94.00.40.70.0 100.0450 Team 296.91.50.4 0.2100.0453 Team 395.82.20.02.00.0 100.0449 Team 472.511.26.74.52.22.9100.0448 All teams90.14.71.9 0.70.8100.01800

16. 16 Table FC-7D Birth date reporting: Deceased children Percent distribution of non-surviving births by completeness of date information by interviewer team DEAD CHILDREN Completeness of reporting Team Month and year givenYear only Month onlyNo dataTotalNumber Team 188.010.00.02.0100.050 Team 295.74.30.0 100.047 Team 394.15.90.0 100.051 Team 469.219.21.99.7100.052 All teams86.510.00.53.0100.0200

17. T ABLE FC-8: Heaping on age at death A common problem in the collection of data on age at death is “heaping” at 12 months of age i.e. a large number of deaths are reported at 12 months relative to the number reported at months 9, 10, and 11, or at months 13, 14, and 15. Such heaping can result in the underestimation of the infant mortality rate (based on deaths in months 0-11) and overestimation of the child mortality rate (based on deaths in months 12-23 and years 2-4). Heaping of deaths at 12 months of age is the result of two frequently encountered interviewing situations. The first situation occurs when respondents report age at death as "one year", even though the death may have occurred at 10 months, 16 months, etc. Some interviewers will record "1 year" (incorrectly) or (also incorrect) simply convert "1 year" to 12 months and record that without probing. The second situation in which heaping occurs is when a respondent initially reports that she does not the know the age but, when encouraged to recall the age, reports in terms of a preferred number of months (i.e., 12 rather than 11 or 13). 17

18. 18 FC-8: Age at death heaping Number of deaths in the 15 years preceding the survey occurring at 8-16 months of age by reported months of age at death (including age at death reported as "one year") and 12 months ratio, according to interviewer team. (Includes deaths for which a calendar period of death could not be assigned because of missing date of birth information. Deaths lacking age at death are not included. Based on births of all women) Team Age at death (in months) Total 8-16 months (including "1 year") 12 months ratio (including "1 year")* Targe t not met 8 m. 9 m. 10 m. 11 m. 12 months 13 m. 14 m. 15 m. 16 m. 12 m. Reported as “1 year” Team 1104443432 21371.7 Team 22012524521 32561.4 - Team 38126371100 24533.1 Team 4108432532 43441.4 - Total48361912162585 11101901.9 * 12 months ratio = (deaths at 12 months + deaths reported at "1 year") / ((all deaths 8-16 m. + deaths reported at "1 year") / 9)

19. Table FC-9 : Underreporting of Infant deaths Underreporting of births and deceased children seriously undermines data quality. This table is useful in determining whether gross underreporting of infant deaths is occurring. However, there is no certain way to determine whether an individual interviewer or team is omitting births of deceased children, because sampling fluctuations and genuine regional differences can produce differences among teams and individuals that are unrelated to data quality. Generally, if the neonatal to infant mortality ratio falls below 0.45, or is significantly lower in one or more teams relative to the others, then omission of neonatal deaths is suspected. Also, if the infant deaths to total birth ratio is substantially lower in one or more teams than in other teams, then omission in infant deaths is suspected. 19

20. 20 Table FC-9 Infant mortality Number of births in the 15 years before the survey by survival status and age at death (for those who died), the ratio of neonatal deaths (< 1 mo.) to all infant deaths (< 12 mos.), and the ratio of infant deaths to all births, according to interviewer team. All birthsRatios Age at death in months for children who died* Still alive (6) Total births (7)=(5+ 6) Ratio of neonatal to infant (1)/(1+2) Ratio of infant deaths to total births per thousand (1+2)/(7) Team <1 (1) 1-11 (2) 12+** (3) Missing (4) Total (5) Team 1713154395646030.350.033 Team 22518200635335960.580.072 Team 32415191595476060.620.064 Team 42516170585375950.610.069 All teams8162715219218124000.570.060 * Includes deaths without a birth date given. ** Includes deaths incorrectly recorded at “ 1 year ”

21. Table FC-14: Maternal Mortality Module Although the DHS Maternal Mortality Module is an optional “add- on” to the survey, it is widely used. Field-check Table 14 provides data on three key elements of the module: (1) the proportion of a respondent’s sisters who have died but for whom no age at death is given; (2) the proportion of sisters who died at age 12 or older but for whom there is no information as to whether the death should be classified as a pregnancy-related death; and (3) the proportion of sisters who died at age 12 and above for whom the timing of the death in terms of number of years ago is missing. The target values for the three elements are <2%, 0%, and 0%, respectively 21

22. 22 FC-14 Maternal mortality module Number of deceased sisters and number and percentage for whom age at death is missing; number of sisters who died at age 12 years and above and number and percentage for whom information on death during pregnancy, during delivery or after delivery is missing, and for whom information on timing of death ("years ago") is missing, according to interviewer team Team All deceased sisters Sisters who died at age 12+ Numb er Missing age at death Targe t not met in Colu mn (3) Numb er Missing information on death during pregnancy, delivery, or after delivery Targ et not met in Col um n (6) Missing information on "years ago" of death Targe t not met in Colu mn (8) Numbe r% % % (1)(2) (3)=(2 /1)(4)(5) (6)=(5/ 4)(7)(8)=(7/4) Team 144781.8-14500.0-10.7 Team 2572142.4 17510.6 00.0- Team 3491326.5 15121.3 10.7 Team 464740.6-21500.0-0 - All teams2,157582.72.368630.4 20.30.2

23. Double entry verification  CSPro supports both dependent and independent verification (double keying) to ensure the accuracy of the data entry operation.  For each PSU the is usually entered twice in separate computers. Data entry supervisor uses CSPro menu to run verification program. This program takes as input the original main data file and the verification data file, and scans them for differences.  When differences are found, they are output to a file and printed out by the supervisor Each difference is checked against the questionnaire to determine which is the correct entry. Both the verification and main data entry files are corrected as necessary, and the verification program re-run to ensure that no differences remain 23

24. Secondary editing After the raw data is backed up, the supervisor run the secondary editing application built in CSPro data entry system. The data entry application in CSPro can be designed in such a way that it will be able to identify any inconsistency at the time of entering the data. However, there are some inconsistencies that require a more in-depth analysis, as well as the attention of subject-matter specialists or senior staff to properly resolve them. DHS chooses to make these checks after data entry is complete in a secondary editing application. The result of running this application is an output reporting the inconsistencies identified. The office editors & subject-matter specialists analyze the messages and decide whether data needs to be changed or not according to “Secondary Editing Guidelines” manual. 24

25. EVENT table As part of the secondary editing program, the DHS software compiles an event table. The event table facilitates the consistency checking of the date and interval responses in the questionnaire. It allows up to 30 entries, beginning with the woman's date of birth, followed by the date of her first marriage/union, the birth dates of each of her children from the birth history, the date of sterilization (if appropriate), the date of conception of a current pregnancy (if appropriate) and completed date of interview Resolving inconsistencies in the responses, particularly those involving date and interval information, requires a detailed understanding of the nature and overall objectives of the survey questionnaire as well as the interrelationships among specific questions. Senior staff &subject matter specialist are employed to resolve the inconsistencies. 25

26. The crucial events for a woman for which consistency checks usually carried out include: Date of respondent's birth Date of first union Date of birth of each child Date of sterilization Date of conception of current pregnancy(date of interview minus months of current pregnancy Date of interview Age of the respondent Age of child at last birthday (if child alive) Age of child at death (if child died)) Time since last period Duration of use of current method (calendar Col1/) Duration of amenorrhea (live births in last five years) Duration of abstinence (live births in last five years)) Time since last sexual intercourse (if ever had sex)) Age at first sexual intercourse (if ever had sex) 26

27. Thank You 27