MICS4 Survey Design Workshop Multiple Indicator Cluster Surveys Survey Design Workshop Survey Quality Control

MICS4 Survey Design Workshop Objectives Identify factors affecting the accuracy and reliability of survey data How to prevent and correct errors The essential role of supervision in the field and providing feedback to the team

MICS4 Survey Design Workshop Overview of presentation Preparation for fieldwork Data collection Field supervision Data processing

MICS4 Survey Design Workshop Overview of presentation Preparation for fieldwork Data collection Field supervision Data processing

MICS4 Survey Design Workshop Preparation for fieldwork Questionnaire –Adaptation (relevant, tailored to needs) –Translation (for uniformity) –Pretest (good flow, capture relevant information) Sample design and size –For reliability, accuracy, feasibility Interviewers –Selection (systematic process, have surplus) –Training (trainers, length, practice)

MICS4 Survey Design Workshop Overview of presentation Preparation for fieldwork Data collection Field supervision Data processing

MICS4 Survey Design Workshop Data Collection Logistics & discipline Implementation of sample Monitoring of fieldwork

MICS4 Survey Design Workshop Data Collection: Material & equipment (questionnaires/forms) Transport Accommodation Payments to field staff Discipline & reasonable expectations Keep the morale of the troops Logistics & discipline

MICS4 Survey Design Workshop Data Collection: Implement. of sample Non-response: failure to obtain information for selected households, eligible women, or children A potentially serious bias that can be minimized Interviewers will need to make return visits to households (call-backs)

MICS4 Survey Design Workshop 1.Interviewer is unable to do the selected household 2.Interviewer unable to meet eligible respondents 3.Respondent refuses to be interviewed Types of non-response: Data Collection: Implement. of sample

MICS4 Survey Design Workshop 1.Interviewer is unable to do the selected household. Why? Types of non-response: –Structure not found –Occupied structure inaccessible –Structure non-residential, vacant, or demolished Data Collection: Implement. of sample

MICS4 Survey Design Workshop Data Collection: Implement. of sample 2.Interviewer unable to meet eligible respondents. Why? Types of non-response: –No one at home at time of call –Respondent temporarily absent Will need call-backs

MICS4 Survey Design Workshop Call-backs Interview that is not completed requires a call- back or follow-up visit Three call-backs required (different times and days) Supervisors and interviewers keep track using control sheets Requires good tracking of work to ensure that all planned interviews are completed before leaving cluster

MICS4 Survey Design Workshop Data Collection: Implement. of sample 3.Respondent refuses to be interviewed. What to do? Types of non-response: –Approach respondent from her point of view –Postponed interview to another day –Have field editor carry out the interview The number of refusals should be closely monitored. Need to find out reasons for frequent refusals.

MICS4 Survey Design Workshop Data Collection: Monitoring field work Training is a continuous process Observation and supervision throughout the fieldwork are a part of the training Team supervisors and field editors play very important roles in continuing this training and in ensuring the quality of MICS4 data

MICS4 Survey Design Workshop Data Collection: Monitoring field work Fieldwork control sheets Direct observation of interviews Review of completed questionnaires (editing) Spot-checking households

MICS4 Survey Design Workshop Fieldwork Control Sheets Interviewers work is monitored and evaluated by keeping accurate record of assignments Both supervisors and interviewers have control forms to maintain These forms should be returned to the director of field operations along with the completed questionnaires Interviewer is responsible for ensuring that control sheet is up-to-date

MICS4 Survey Design Workshop Observing Interviews, part 1 To evaluate and improve interviewer performance To look for errors and misconceptions that cannot be detected through editing Why? Precise but inaccurate answers Who? The supervisors or field editors (females only for womens questionnaire)

MICS4 Survey Design Workshop Observing Interviews, part 2 How often? 5-6 interviews per week, more at start of fieldwork How? Just take notes without disrupting the interview What to do after? Supervisors speaks to interviewers

MICS4 Survey Design Workshop Editing Questionnaires in the Field Fundamental to survey quality Need to ensure: –Accurate and complete information in each questionnaire –Correct count of questionnaires

MICS4 Survey Design Workshop Editing Questionnaires in the Field Done daily by field editor; supervisor can assist Finish all questionnaires before leaving cluster Provides an opportunity to identify and correct mistakes before leaving cluster Results and errors are discussed with interviewers

MICS4 Survey Design Workshop Systematic Spot Checking How to spot-check household composition? –Supervisors should complete columns 2-8 in the HH questionnaire and compare with that of the interviewer –Check about 5% of households (5-6 per week) –All team members must be spot-checked; provide feedback if necessary

MICS4 Survey Design Workshop Evaluating Interviewers Performance Daily discuss the quality of interviewers work Point out mistakes discovered during observation of interviews or noticed during editing Discuss examples of actual mistakes, but be careful not to embarrass individual interviewers. Re-read relevant sections from the Interviewer's Manual with the team to resolve problems. Encourage the interviewers to talk about any situations they encountered in the field Discuss whether situations are handled properly, and how to do it in the future

MICS4 Survey Design Workshop Overview of presentation Preparation for fieldwork Data collection Field supervision Data processing

MICS4 Survey Design Workshop Field supervision Staff from survey headquarters –Senior staff –Quality control team Purpose –Visit teams to observe interviews, review work –Re-visit selected clusters; spot-check households Very important during first few weeks of fieldwork! (Possible to start teams in one location?)

MICS4 Survey Design Workshop Overview of presentation Preparation for fieldwork Data collection Field supervision Data processing

MICS4 Survey Design Workshop Data Processing Integral aspect of survey quality (DP programmes, double-entry, secondary editing) Problems detected in DP should be communicated back to teams (phone calls and visits) Field-check tables Observations from reviewed questionnaires

MICS4 Survey Design Workshop GENERAL INSTRUCTION TO EDIT QUESTIONNAIRES According to MICS Manual Under no circumstances should we make up an answer