Questionnaire Design

“It is not every question that deserves an answer.” Publius Syrus (Roman, 1st century B.C.)

Ask a Sensitive Question, Get a Sensitive Answer Survey researchers believe that the responses that people give are valid. Care must be taken with sensitive questions. Researchers must take care in asking relevant questions in ways that produce the most truthful results.

Basic Considerations in Questionnaire Design Questionnaire design is one of the most critical stages in the survey research process. A questionnaire (survey) is only as good as the questions it asks — ask a bad question, get bad results. The questions must meet the basic criteria of relevance and accuracy.

What Should Be Asked? Questionnaire Relevancy Questionnaire Accuracy All information collected should address a research question that helps the decision maker in solving a current marketing problem. Questionnaire Accuracy The information is valid; it faithfully represents reality. Questionnaires should use simple, understandable, unbiased, unambiguous, & nonirritating words. Questionnaire design should facilitate recall & motivate respondents to cooperate. Proper question wording & sequencing to avoid confusion & biased answers.

Major Decisions in Questionnaire Design What should be asked? How should each question be phrased? In what sequence should the questions be arranged? What questionnaire layout will best serve the research objectives? How should the questionnaire be pretested? Does the questionnaire need to be revised?

Phrasing Questions Open-ended questions Fixed-alternative questions Aka essay questions, short-answer questions Fixed-alternative questions Aka closed or closed-ended questions

Open-Ended Response Questions Pose some problem & ask respondents to answer in their own words. Advantages: Particularly beneficial in exploratory research, especially when the range of responses is not known. Identify which words & phrases people spontaneously give. Valuable at the beginning of an interview. Disadvantages: High cost of administering open-ended response questions. The possibility that interviewer bias will influence the answer. Bias introduced by articulate individuals’ longer answers.

Example of Open-Ended Response Question

Fixed-alternative Questions Questions in which respondents are given specific, limited-alternative responses & asked to choose the one closest to their own viewpoint. Advantages: Require less interviewer skill Take less time to answer Are easier for the respondent to answer Provides comparability of answers Disadvantages: Researcher may be unaware of all potential responses Tendency of respondents to choose more prestigious or socially acceptable alternative

Example of a Fixed-Alternative Question

Types of Fixed-Alternative Questions Simple-dichotomy (dichotomous) Question Requires the respondent to choose one of two alternatives (e.g., yes or no). Example: Did you make any calls with your home (landline) phone during the 7 days? _____ Yes _____ No Determinant-Choice (multiple-choice) Question Requires the respondent to choose one response from among multiple alternatives (e.g., A, B, or C).

Types of Fixed-Alternative Questions (con’t.) Frequency-determination Question Asks for an answer about general frequency of occurrence (e.g., often, occasionally, or never). Checklist Question Allows the respondent to provide multiple answers to a single question by checking off items. Scale Likert, Semantic Differential, Stapel, etc.

Influences on Question Phrasing Phrasing Questions for Self-Administered, Telephone, & Personal Interview Surveys Influences on Question Phrasing Means of data collection — telephone interview, personal interview, self-administered questionnaire — will influence question format & question phrasing. Questions for mail, Internet, & telephone surveys must be less complex than those used in personal interviews. Questionnaires for telephone & personal interviews should be written in a conversational style.

Best Question Formats Vary by the Interview Medium

Guidelines For Avoiding Mistakes Simpler is better Avoid leading & loaded questions Leading question – directs respondents to an answer you want them to give Loaded question – suggests a socially desirable answer or is emotionally charged.

Guidelines For Avoiding Mistakes (con’t.) “Avoid” ambiguity: Be as specific as possible Probably /Definitely; Sometimes/Always, etc. Avoid double-barreled items Double-barreled question – may induce bias because it covers two or more issues at once. Do you think the President is responsible for the federal government shut-down and the currently rising gasoline prices? Yes No Avoid making assumptions Given Macy’s skill-level at gift wrapping, ……. All-inclusive response alternatives Avoid taxing respondent’s memory

Avoid Common Wording Mistakes in Questionnaire Design

Order Bias Question Sequence Randomized Presentations Bias caused by the influence of earlier questions in a questionnaire or by an answer’s position in a set of answers. Funnel technique Asking general questions before specific questions in order to obtain unbiased responses. Randomized Presentations Used in electronic questionnaires, but rarely used in printed questionnaires due to coding difficulties. Randomized Response Techniques Randomly assigning respondents to answer either the question of interest (embarrassing) or a mundane & unembarrassing question.

Survey Flow Survey flow Breakoff Filter question Branching The ordering of questions through a survey. Breakoff A respondent who stops answering questions before reaching the end of the survey. Filter question A question that screens out respondents who are not qualified to answer a second (or follow-up) question. Branching Directing respondents to alternative portions of the questionnaire based on their response to a filter question.

Survey Flow for Eurocar Tour de France Sponsorship

Telephone Questionnaire with Skip Questions

Survey Technology Physical Features Prompting Piping Software Heat map question A graphical question that tracks the parts of an image or advertisement that most capture a respondent’s attention. Status Bar A visual indicator that tells the respondent what portion of the survey he or she has completed. Prompting Informs the respondent that he or she has skipped an item or provided implausible information. Piping Software Allows question answers to be inserted into later questions.

Tracking Points of Interest Using a Heat Map

Illustration of Status Bar & Prompts

Pretesting & Revising Questionnaires Pretesting Process Seeks to determine whether respondents have any difficulty understanding the questionnaire whether there are any ambiguous or biased questions. Preliminary Tabulation A tabulation of results of a pretest to help determine whether questionnaire will meet the objectives of the research.

I Give Up! Large portion of respondents give up before finishing & abandon the survey (break-offs). Guidelines Visually appealing & easy to read Fewer questions per page (no more than 20) 4 pages maximum for consumers 6 pages maximum for business leaders Question order (funnel vs. reverse-funnel) Sensitive questions & open-ended questions encourage break-offs &/or item non-response Sophisticated samples increase response rate Make information requests legitimate Pretesting is important (pretest EVERYTHING)

Questionnaire Reproduction (if mail survey) Professional appearance Booklet format for long questionnaires Place directions as close to questions as possible Expect reproduction errors