Survey Methodology Interviewing EPID 626 Lecture 9

Reference Many elements for this lecture were taken from : Fowler, Floyd J. and Thomas W. Mangione. Standardized Survey Interviewing. Applied Social Research Methods Series, Vol 18. Sage Publications, (In addition to the course text)

Interviewer Roles Locate and enlist the cooperation of selected respondents Train and motivate respondents Ask questions, record answers, and probe incomplete answers

Pitfalls along the way Interviewers can have detrimental effects on survey validity and reliability….

Interviewers as a source of error When they do not read questions as worded When they probe directively When they bias answers by the way they relate to respondents When they record answers inaccurately

Reading questions as worded Conservative estimates based on several observational studies suggest that interviewers change question wording at a rate of 20-40% Why? –Poor question design –Attempt to make the interview more conversational

Example A: Should Communists be forbidden to speak in public places in the US? B: Should Communists be allowed to speak in public places in the US? Comparable samples % said yes to A 50% said no to B

Probing Required when the initial reading of the question does not provide a satisfactory answer Usually begin by repeating the question Follow with non-directive probes: –How do you mean that? –Tell me more about that –Anything else?

Probing (2) Problem may be that respondent does not understand a term or concept in the question –If a definition is provided with the question, interviewer should reread the definition –If no definition is provided, the respondent should answer the question with the interpretation that seems best to them

Relating to respondents Interviewer should not share personal information about life situations, views, or values Interviewer should be careful about feedback to respondents so as not to imply any evaluation or judgement

Recording answers No interviewer judgement No interviewer summaries No interviewer effects Open-ended answers should be recorded verbatim

Detecting interviewer-related error Direct observation of interviewers –On-site observations –Recording interviews Validating survey answers –Check against available records

Detecting interviewer-related error (2) Associating interviewers with the answers they obtain –Model the data with interviewer as a predictor variable –Compare the results of one interviewer to the rest of the interviewers Requires that interviewers are assigned representative subsamples of the whole sample

Position overview Requires good reading and writing skills Usually part-time work Often requires flexible hours (evenings, weekends) May require mobility, transport Demographics?

Interviewer training Procedures for contacting respondents and introducing the study Conventions that are used in the design of the survey with respect to wording and skip instructions, so that interviewers can ask the questions in a consistent and standardized way

Interviewer training (2) Procedures for probing inadequate answers in a non-directive way Procedures for recording answers to open-ended and closed questions Rules and guidelines for handling the interpersonal aspects of the interview in a non-biasing way

Interviewer training (3) Purpose of the survey –How much detail?

Potential errors due to interviewing problems Samples lose credibility and are likely to be biased if interviewers do not do a good job of enlisting respondent cooperation

Potential errors due to interviewing problems (2) The precision of survey estimates will be reduced, and there will be more error around estimates to the extent that interviewers are inconsistent in ways that influence the data

Potential errors due to interviewing problems (3) Answers may be systematically inaccurate or biased to the extent that interviewers fail to train and motivate respondents appropriately or fail to establish an appropriate interpersonal setting for reporting what is required