1. Questionnaire Design

2. What is a Questionnaire? A set of Questions designed to generate the statistical information from a specific demographic needed to accomplish the research objectives

3. 1. Content - What should be asked? 2. Wording - How should each question be phrased? 3. Sequence - In what order should the questions be presented? 4. Layout - What layout will best serve the research objectives? The Major Decisions in Questionnaire Design The most difficult step is specifying exactly what information is to be collected from each respondent

4. 1. Are the questions relevant. Do they pertain to the research problem 2.Are the questions accurate. Do they accurately depict the attitudes, behaviors, etc. intended to investigate 3.Do respondents have the necessary information? Qualify respondents 4.Do respondents understand and interpret the question correctly 5.Will respondents give the information? 1. Content: Questions must meet 5 requirements

5. Format: How much freedom do we give respondents in answering questions Open-ended questions – What do you look for most in a job? – Is there anything else you would like to add about the product How should the questions be asked?

6. Open Ended Questions: key advantages  Wide range of responses and information can be obtained  Answers based on respondent’s not researcher’s frame of reference – consumer’s terms  Lack of influence. Don't channel respondents thinking  Can help interpret closed-ended questions - why  Particularly useful as introduction to survey or topic  When it’s important to measure the salience of an issue  When too many possible responses to be listed or unknown

7.  Interviewer’s ability to record answers quickly or summarize accurately & probe effectively  Interviewer’s attitude influences response  Time consuming (interview sessions, tabulation, classification, assignment, validation)  Difficulty in coding  Require respondents to be articulate  Respondents may miss important points  Non-response Open-ended questions: Key disadvantages

8. What do you look for most in a job? ____ Work that pays well ____ Work that gives a sense of accomplishment ____ Work where you make most decisions by yourself ____ Work that is steady with little chance of being laid off.  Ease of understanding  Requires less effort on part of interviewer and respondent  Ease of tabulation & analysis  Less error prone  Less interviewer bias  Less time consuming  Answers directly comparable from respondent to respondent Closed-ended questions (Fixed-alternative responses) Advantages

9.  Middle/Neutral categories often selected inappropriately (ignorance, safety)  Less opportunity for self-expression or subtle qualifications  Less involving for respondents  Order of response categories can have major impact on results Closed-ended questions (Fixed-alternative questions) Disadvantages Key tradeoff Want to get respondent to address issues our research is concerned with (Forced response) and at same time give respondent opportunity to honestly opt out of question (i.e., Don’t Know, No Answer, Neither Agree nor Disagree) so as not to dilute data collected

10. Dichotomous Questions Should the Alberta Government give consumers an energy rebate? 1.Agree 2.Disagree Advantages  Easy to administer and tabulate Disadvantages  Prone to large amounts of error since polarized responses prevent gaining information on the range of variation  Fail to communicate any intensity of feeling

11. Multiple Choice Questions  Are all possible alternatives included?  Too many alternatives  Position Bias Scaled Response Questions  Closed ended questions where the response choices are designed to capture an intensity of feeling (Likert, Staple, Semantic differential)  Easy to code and more powerful statistical tools  Main problem: Respondent misunderstanding

12.  Complexity: use simple, direct, conversational language  leading questions -- that suggest or imply certain answers  loaded questions -- suggest social desirability, or are emotionally charged. Have you purchased a high quality Sony TV this year  Ambiguity and vagueness: Words such as “often”, “occasionally”, “usually”, “regularly”, “frequently”, “many”, should be used with caution. If these words have to be used, their meaning should be explained properly.  Which province is bigger Manitoba or Alberta? – would your answer be based on population or area? 2. Question Wording  Can have major impact on how respondent interprets question  All respondents should interpret in the same way Things to avoid

13.  long-worded questions  double-barreled questions. Questions that refer to two or more issues within the same question. Where respondent may agree with only 1 part of multipart statement. --Do you think Nike offers better pricing and variety than other brands ? More things to avoid

14.  burdensome questions - that may tax the respondent’s memory  How many tubes of toothpaste have you purchased in the last 3 months  Have you purchased toothpaste in the past week  Embarrassing, sensitive, or threatening questions:  Have you charged more on your credit card than you should  Use third person do you think most people…..

15. Initial stages  Screening or qualifying Questions: Have you been to the movie theatre this month?  Need to gain & maintain respondent’s cooperation  Make questionnaire simple for interviewer to administer  opening questions should be interesting, simple, and easy to answer.  which theatre did you go to last? 3. Question Sequence & Layout Decisions

16. Transition Questions  What aspects of the theatre did you like best  Questions directly related to research objectives which require more effort and get respondent thinking about topic Difficult and Complicated Questions  The following 10 questions relate to the characteristics of theatres  Respondent now committed and can see an end in sight Classifying and Demographic Questions  What is your average annual household income  May not be answered (sensitive or threatening) but most questions have been answered

17.  Demographic questions should come at the end. 1. initial questions - screening and rapport questions 2.Middle - product specific questions 3.End demographic questions.

18. INSTRUCTIONS LAYOUT A: Do you agree, disagree or have no opinion that this company has: A good vacation policy - disagree/not sure/agree. Good management feedback - disagree/not sure/agree. Good medical insurance - disagree/not sure/agree. INSTRUCTIONS LAYOUT B: Does this company have: ___ Disagree Not Sure Agree A good vacation policy 1 2 3 Good management feedback 1 2 3 Good medical insurance 1 2 3 Distinguishing question and responses: Matrix question CAPS or BOLD or Underline versus lower case or unformatted

19. The layout and physical attractiveness of a questionnaire are important aspects  Questionnaires should be designed to appear as short as possible  Questionnaires should not appear overcrowded  Leave lots of space for open ended questions  Questionnaires in booklet form are often recommended 4. Questionnaire Layout

20. OPENING  Provide first name (at least)  Provide name of company doing research  Provide reason for survey and topic  State that no selling will be involved and no personal data other than for statistical purposes  Tell respondent approximate time to complete  Reinforce that respondent’s time is appreciated  Invite to participate AND CLOSING  Thank for time  Ask if they had a positive experience and remind them that their opinions count

21. Pretesting and Correcting Problems Purpose of pretest: To ensure that the questionnaire meets the expectations in terms of the information that will be obtained.  Is question necessary  Missing important variables  Match questions to objectives  Pretest Specific Questions for ill-defined, loaded, double-barreled questions  Pretest the Questionnaire for Length  Respondent Interest and Attention