2 Purposes of the Questionnaire A questionnaire ensures standardization and comparability of the data across interviews, increases speed and accuracy of recordingFacilitates data processingAlso allows the researcher to collect the relevant information necessary to address the management decision problem
3 Designing the Questionnaire Logical Steps to Develop a Good QuestionnairePlan what to measureFormulate questions to obtain the needed informationDecide on the wording of questionsDecide on the order and layout of the questionnaireUsing a sample, test the questionnaire for omissions and ambiguityCorrect the problems (pretest again, if necessary)
4 The Major Decisions in Questionnaire Design 1. Content - What should be asked?2. Wording - How should each question be phrased?3. Sequence - In what order should the questions be presented?4. Layout - What layout will best serve the research objectives?The most difficult step is specifying exactly what information is to be collected from each respondent
5 Content: Questions must meet 5 requirements Are the questions relevant. Do they pertain to the research problemAre the questions accurate. Do they accurately depict the attitudes, behaviors, etc. intended to investigateDo respondents have the necessary information?Qualify respondentsDo respondents understand and interpret the question correctlyWill respondents give the information?
6 How should the questions be asked? Format: How much freedom do we give respondents in answering questionsOpen-ended questionseg. People look for different things in a job. What would you prefer most in a job?Some key advantagesWide range of responses can be obtainedLack of influence. Don't channel respondents thinkingParticularly useful as introduction to survey or topicWhen it’s important to measure the salience of an issueWhen too many possible responses to be listed or unknown
7 Open-ended questionsKey disadvantagesAbility and/or willingness of respondent to answerInterviewer’s ability to record answers quickly or summarize accurately & probe effectivelyInterviewer’s attitude influences responseTime consuming (interview sessions, tabulation, classification, assignment, validation)Difficulty in codingRequire respondents to be articulateRespondents may miss important pointsNon-response
8 Closed-ended questions (Fixed-alternative questions) People look for different things in a job. What would you prefer most in a job?____ Work that pays well____ Work that gives a sense of accomplishment____ Work where you make most decisions by yourself____ Work that is steady with little chance of being laid off.AdvantagesEase of understandingRequires less effort on part of interviewerEase of tabulation & analysisLess error proneLess time consumingAnswers directly comparable from respondent to respondent
9 Closed-ended questions (Fixed-alternative questions) DisadvantagesMiddle/Neutral categories often selected inappropriately (ignorance, safety)Less opportunity for self-expression or subtle qualificationsLess involving for respondents
10 Closed-ended questions Order of response categoriesCan have major impact on resultsKey tradeoffWant to get respondent to address issues your research is concerned with (Forced response) and at same time give respondent opportunity to honestly opt out of question (i.e., Don’t Know, No Answer, Neither Agree nor Disagree) so as not to dilute data collected
11 Question WordingCan have major impact on how respondent interprets questionThings to avoidAvoid Complexity: use simple, direct, conversational languageAvoid leading questions -- that suggest or imply certain answersAvoid loaded questions -- that suggest social desirability, or are emotionally chargedAvoid ambiguity and vagueness: be as specific as possibleAvoid long-worded questions
12 More things to avoidAvoid Ambiguity: Words such as “often”, “occasionally”, “usually”, “regularly”, “frequently”, “many”, should be used with caution. If these words have to be used, their meaning should be explained properly.Avoid double-barreled items. questions that refer to two or more issues within the same question Where respondent may agree with only 1 part of multipart statement. E.g. do you think Nike offers better pricing and variety than other brandsAvoid making implicit assumptionsAvoid burdensome questions - that may tax the respondent’s memory
13 Sequence & Layout Decisions Surveys are more than a collection of unambiguous questionsHow questions are specified and put together will influence the sample’s willingness to participate & the responses they provideSequence & Layout DecisionsInitial stagesNeed to gain & maintain respondent’s cooperationMake questionnaire simple for interviewer to administerThe opening questions should always be interesting, simple, and easy to answer.
14 Beyond initial stages Demographic questions should come at the end. General questions should be asked before more specific onesDemographic questions should come at the end.Use multiple questions instead of oneSimilar questions togetherconsistent mindset for respondentsDevelop a logical flowUse transitions between sectionsDistinguish question and responsesCAPS or BOLD or Underline versus lower case or unformatted
15 Distinguishing question and responses: LAYOUT A:Do you agree, disagree or have no opinion that this company has:A good vacation policy - disagree/not sure/agree.Good management feedback - disagree/not sure/agree.Good medical insurance - disagree/not sure/agree.LAYOUT B:Does this company have: Disagree Not Sure Agree A good vacation policy 1 2 3 Good feedback 1 2 3 Good medical insurance 1 2 3
16 Filter and pivot questions should be used as necessary. A FILTER question is one that screens out respondents who are not qualified to answer a second question.A PIVOT question is a type of filter question that is used to determine what version of a second question to ask.
17 Developing a logical flow If the questionnaire deals with several topics, complete questions on a single topic before moving on to a new topicIf topics are related, ask questions on related topics before asking questions about unrelated topicsIf you ask questions about behaviors over some time period, follow chronological order backward in timeWhen changing topics, use some transitional phrase
18 QUESTIONNAIRE LAYOUTThe layout and physical attractiveness of a questionnaire are important aspectsQuestionnaires should be designed to appear as short as possibleQuestionnaires should not appear overcrowdedQuestionnaires in booklet form are often recommended
19 Pretesting and Correcting Problems Purpose of pretest: To ensure that the questionnaire meets the expectations in terms of the information that will be obtainedMissing important variablesPretesting Specific Questions ForVariationMeaningTask difficultyRespondent interest and attentionAmbiguous, ill-defined, loaded, double-barreled questionsPretesting the QuestionnaireFlow of the questionnaireSkip patternsLengthRespondent Interest and Attention
20 A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS