Survey Methodology Survey Instruments (1) EPID 626 Lecture 7

What is a survey instrument? For interviews: A script for interviewers, including introductions, instructions, and questions For self-administered surveys: A questionnaire

Essential components for developing a survey instrument Statement of purposes –What do you want to accomplish with the survey? List of the variables to be measured –Group into logical categories Draft analysis plan –What are your dependent, independent, check variables?

Preliminary question design steps Focus groups Draft questions Cognitive laboratory interviews

Focus groups Objective: To compare the reality about which respondents will be answering questions with the abstract concepts embedded in the study objectives (Fowler, 1995) Usually 6-8 people per group; have a couple of groups

Focus groups(2) Try to get participants who are similar to your survey population Talk about perceptions, experiences, and feelings related to survey subject

Drafting questions Look for standard questions from other surveys –Benefit from their experience –Enhance comparability of your survey with previous surveys Write good questions (see next lecture)

Cognitive laboratory interviews Respondents are brought into a laboratory setting –May be videotaped –Interviews conducted by cognitive psychologist or experienced investigator Goal: To get information about how the respondent understood the questions and about the way they answered them

Layout of survey instrument Make it clear and simple Usually begin with straightforward questions, put complicated or sensitive questions in the middle or at the end

Interviews Should have everything scripted, including introductions, instructions, transitions, definitions, and explanations Be very explicit about skip instructions Differentiate instructions and optional wording from questions –Usually done with all caps and parentheses, respectively

Self-administered Questionnaire should be self- explanatory (minimal instructions needed) Limit to closed questions Use short questions with consistent formats Minimize skips; make them very clear

Field pretesting Goal: Find out how well the data collection protocols and survey instruments work under realistic conditions

Interviews Usually interview respondents who are similar to your survey population Interviewers note ways to improve survey instruments

Quantitative methods for pretesting Ask interviewers to fill out a rating for each question –Is it easy to read as worded? –Do respondents understand the question in a consistent way? –Can respondents answer the question accurately?

Taping and behavior coding –Trained coders review the recordings and evaluate the question and answer process in a consistent way Do the interviewers read the question as worded? Do the respondents ask for clarification? Do the respondents initially give an inadequate answer that requires interviewer probing? Quantitative methods for pretesting(2)

Self-administered Have respondents (who are similar to your survey population) fill out the survey and then discuss –Were the instructions clear? –Were the questions clear? –Were there any problems understanding what kind of answers were expected?