Presentation on theme: "SURVEY QUALITY CONTROL. Objectives § Identify factors affecting the accuracy, validity and reliability of survey data § How to prevent and correct errors,"— Presentation transcript:
SURVEY QUALITY CONTROL
Objectives § Identify factors affecting the accuracy, validity and reliability of survey data § How to prevent and correct errors, § How to avoid common causes of mis-reporting § The essential role of supervision and team interaction in the field
Main Components of Survey Quality!! §Supervision of interviewers §Non-response §Pending interviews §Fieldwork control sheets §Monitoring interviewer performance l Spot checking l Observing interviews §Editing of questionnaire §Evaluating interviewers §Returning questionnaires to central office
Supervision of Interviewers §Training is a continuous process. §Observation and supervision throughout the fieldwork are a part of the training §Team supervisors and field editors play very important roles in continuing this training and in ensuring the quality of MICS3 data.
Non Response §One of the most serious problems for MICS §No response is failure to obtain information for selected households, eligible women, or children. §A serious bias to be minimized by supervisors and editors §Interviewers will need to make return visits to households in the evening or on the weekends
Types of Non Response No response may be classified into three basic types: –Type 1: the interviewer is unable to locate the selected household; –Type 2: the interviewer is unable to locate the eligible woman or the caretaker of the children under five for the individual interview; –Type 3: the respondent refuses to be interviewed.
TYPE 1 §The interviewer is unable to locate the household –Occupied structure inaccessible –Structure not found. –Structure non-residential, vacant, or demolished.
TYPE 2 §The interviewer is unable to locate the respondent for an individual interview No one home at time of call Respondent temporarily absent
TYPE 3 §The respondent refuses to be interviewed §Number of refusals should be closely monitored §Reasons for frequent refusals §What to do? Approach respondent from her point of view Postpone interview to another day. Have field editor carry out the interview
Handling Pending Interviews §Pending interview-interview that was not completed and requires a call back §Who keeps track of pending interviews and how? Supervisors and interviewers via control sheets §Planning and completing call backs for pending interviews before leaving the cluster §Requires a good division of work to ensure that all planned interviews are completed
Maintaining Fieldwork Control Sheets §The work of interviewers is monitored and evaluated by keeping accurate record of assignments §Both supervisors and interviewers have control forms to maintain. §These forms should be returned to the director of field operations along with the completed questionnaires §Interviewer is responsible for ensuring that control sheet is up-to-date y giving it back to the supervisor
Systematic Spot Checking of Household Composition §Spot checking is used to improve data quality by identify common problems and poor skills §How to spot-check household composition? l Supervisors should complete again columns 2-8 in the HH questionnaire and compare with that of the interviewer l Conduct about 5% of households (5-6 per week) l All team members must be spot-checked. l If problems exist, the supervisor should review the household schedule with interviewer, review the MICS3 manual with the team l Take more serious actions if problem persists with individual interviewer
Monitoring Interviewer Performance §How? By observing interviews and carrying out field editing §When to observe interviews? Throughout field work but more frequently at the beginning of the survey and again toward the end
Observing Interviews §To evaluate and improve interviewer performance §To look for errors and misconceptions that cannot be detected through editing §Why? Precise but inaccurate answers §Who? The supervisors but only by female supervisors for the womens questionnaire § How often? 5-6 interviews per week § When? At any time during the day §How to observe and interview? without disrupting the interview, taking notes about the observed issues §What to do after? Supervisors speaks to interviewers to provided the needed feedback
Household Visits and Individual Interviews At the end of each day field editor should: §First, review the Household and Individual Questionnaires §Second, using the questionnaires, verify the information about the results of the interview §Check that all the households or dwellings are listed on the Interviewer's Assignment Sheet and agrees with those selected from household listing form or map for cluster §There can never be fewer household questionnaires than selected households or dwellings, but there can be more households questionnaire than selected ones
Evaluating Interviewers Performance §Daily discuss the quality of interviewers work §Point out mistakes discovered during observation of interviews or noticed during editing §Discuss examples of actual mistakes, but be careful not to embarrass individual interviewers. §Re-read relevant sections from the Interviewer's Manual with the team to resolve problems. §Encourage the interviewers to talk about any situations they encountered in the field §Discuss whether situations are handled properly, and how to do it in the future
Editing Questionnaires in the Field §To ensure that women and children are not omitted from the Household Questionnaire §To ensure that womans and childrens dates of birth are properly recorded §As explained before this is the work of the field editor §Where and When? Throughout fieldwork but specially at the beginning and end of fieldwork §Who can help? The supervisor can assist §Results and errors are discussed with interviewers §When necessary, interviewers may need to contact respondents to correct errors or inconsistencies §Disciplinary actions should be considered and used
Editing Questionnaires in the Office §A computer program checks each questionnaire and print out a listing of all errors found §If the errors are major ones, an entire questionnaire may be omitted from the analysis. §Errors and inconsistencies observed in the office can not be corrected only imputed (!) §Feedback to the field teams can be provided on errors found during office editing
GENERAL INSTRUCTION TO EDIT QUESTIONNAIRES According to MICS3 Manual Under no circumstances should we make up an answer
Returning Questionnaires to the Central Office Checking the Household Structure §Include all individual questionnaires into the household Questionnaires. §Verify if more than one individual questionnaire (women or children under five) pertain to the household? §Check household questionnaires for the number of eligible women and children, and their corresponding line numbers §Verify number of Individual Questionnaires, and that identification lines are correct and consistent between questionnaires §Follow procedures of communication with central office §For incomplete interviews, include blank questionnaires with only the identification information and the result code
Forwarding Questionnaires to the Central Office §Instructions from the director of field operations §Packaging and protection of questionnaires §Confidentiality of data in questionnaires §Proper instruction developed and implemented for transportation of questionnaire to avoid their loss or information