Chapter Ten Questionnaire Design
Copyright © Houghton Mifflin Company. All rights reserved.10 | 2 Chapter Objectives Discuss questionnaire design as a series of interrelated steps List and describe the different question forms and discuss when and how they can be used most effectively Recognize and reword problem questions. State the suggested guidelines for proper question sequencing Discuss the importance of questionnaire appearance and layout
Copyright © Houghton Mifflin Company. All rights reserved.10 | 3 Chapter Objectives (Cont’d) Explain why pretesting the questionnaire can be critical Describe the design benefits inherent in computerized and online questionnaires List the features of a good cover letter Create appropriate openers for personal and telephone interviews Discuss the questionnaire guidelines that would be relevant in the design of an observation form
Copyright © Houghton Mifflin Company. All rights reserved.10 | 4 Circuit City – Bizrate Customer Satisfaction Study Shopzilla.com is a division of Biz-Rate.com that collects data from previous shoppers at a particular online merchant Shopzilla reports satisfaction ratings for a variety of online retailers so that shoppers can compare merchants across 15 service dimensions
Copyright © Houghton Mifflin Company. All rights reserved.10 | 5 Circuit City – Bizrate Customer Satisfaction Study (Cont’d) Circuit City has extended this survey to its in- store customers through the use of checkout receipts that invite shoppers to the Biz-Rate survey site The shoppers are them asked to rate Circuit City on ten shopping dimensions This data can be used by both shoppers and Circuit City to improve customer satisfaction
Copyright © Houghton Mifflin Company. All rights reserved.10 | 6 Shopzilla.com Courtesy of Shopzilla
Copyright © Houghton Mifflin Company. All rights reserved.10 | 7 Questionnaire A set of questions designed to generate the data necessary for accomplishing a research project's objectives
Copyright © Houghton Mifflin Company. All rights reserved.10 | 8 Complexity of Questionnaire Design Consider the following question that appeared in a mail questionnaire from an presumably experienced researcher Do you consider the many marketing research texts adequate for most of your business majors at the undergraduate level ? _____ Yes ____ No If no; briefly,why not? What do you think? Easy or difficult to answer? What exactly do the words many, adequate, and most refer to in this question?
Copyright © Houghton Mifflin Company. All rights reserved.10 | 9 Questionnaire's Impact on Data Accuracy It must communicate to the respondent what the researcher is asking It must communicate to the researcher what the respondent has said
Copyright © Houghton Mifflin Company. All rights reserved.10 | 10 Sources of Distortion A sloppy questionnaire can lead to a great deal of distortion The entry of an intermediary, or the interviewer, into the communication channel can also distort the questionnaire
Copyright © Houghton Mifflin Company. All rights reserved.10 | 11 Exhibit 10.1 Questionnaire: Link Between Researcher and Respondents
Copyright © Houghton Mifflin Company. All rights reserved.10 | 12 Questionnaire Design Elements Does each question have the most appropriate form (e.g., structured versus nonstructured)? Is each question relevant and properly worded to obtain meaningful, valid responses? Is the sequencing of the questions likely to introduce any bias? Are the layout and appearance of the questionnaire conducive to accurate and easy data collection?
Copyright © Houghton Mifflin Company. All rights reserved.10 | 13 Exhibit 10.2 The Questionnaire Design Process
Copyright © Houghton Mifflin Company. All rights reserved.10 | 14 Question Form Nonstructured questions –Open-ended Structured questions –Fixed-response
Copyright © Houghton Mifflin Company. All rights reserved.10 | 15 Nonstructured Questions Consider the following illustrative questions How old are you?___________________ What do you like most about owning your own home? _____________ Will you please describe your thoughts about a person who shoplifts items from a grocery store to keep from going hungry? _____________________________________ _____________________________________ ____________
Copyright © Houghton Mifflin Company. All rights reserved.10 | 16 Structured Questions In which of the following categories does your age fall? _______ Less than 18 _______ 18 to 30 _______ 31 to 45 _______ 46 to 60 _______ Over 60
Copyright © Houghton Mifflin Company. All rights reserved.10 | 17 Structured Questions: Dichotomous Questions Do you smoke cigarettes? ___Yes___No Have you ever watched CNN News? ___Yes___No
Copyright © Houghton Mifflin Company. All rights reserved.10 | 18 Multiple-Category Questions In your opinion, which product category is the most suited for making purchases on the Internet? ____Automobiles ____Books ____Electronic Items (TVs, video cameras, VCR and DVD players, etc) ____Videos, CDs ____Computer-related items ____Travel (airlines, car rental, hotel reservations, etc.)
Copyright © Houghton Mifflin Company. All rights reserved.10 | 19 Multiple-Category Questions (Cont’d) What do you like about Bank of America’s online banking service? ____24 hour service ____Transaction privacy ____No need to travel – do it from home ____Paying bills online– no need to mail bills ____Other ____(please specify)
Copyright © Houghton Mifflin Company. All rights reserved.10 | 20 Response Category Sequence Approximately how many long-distance telephone calls do you make per week? ______ 0 to 1 call ______ 2 to 3 calls ______ 4 to 5 calls ______ 6 to 7 calls ______ More than 7 calls The response categories in this question follow a natural sequence
Copyright © Houghton Mifflin Company. All rights reserved.10 | 21 Response Category Content Response choices should be –Collectively exhaustive taken together –Mutually exclusive not overlap
Copyright © Houghton Mifflin Company. All rights reserved.10 | 22 Response Category The response choices are collectively exhaustive but not mutually exclusive On the average, how many cans of cola do you drink per week ____0 to 3 cans ____3 to 6 cans ____More than 6 cans Changing the 3 in the second category to a 4 will make the response choices mutually exclusive
Copyright © Houghton Mifflin Company. All rights reserved.10 | 23 Response Category (Cont’d) What do you like about Bank of America’s online banking service? ____24 hour service ____Transaction privacy ____No need to travel – do it from home ____Paying bills online – no need to mail bills ____Other ____(please specify) What is wrong with this question ?
Copyright © Houghton Mifflin Company. All rights reserved.10 | 24 Response Category (Cont’d) Modify the question to read, "Which of the following do you like most about Bank of America’s Online Banking Service?" With this revision each respondent would, or should, check only one response category What do you like about Bank of America’s Online Banking Service? (Please check as many categories as apply.)
Copyright © Houghton Mifflin Company. All rights reserved.10 | 25 Number of Response Categories Multiple-category questions fall between dichotomous questions & open-ended questions
Copyright © Houghton Mifflin Company. All rights reserved.10 | 26 Question Relevance and Wording Can the research objective(s) be fulfilled without asking this question? The question may still be a candidate for revision or deletion and must be further examined question should be deleted Yes No
Copyright © Houghton Mifflin Company. All rights reserved.10 | 27 Can the Respondent Answer the Question? In your opinion, how many students in your marketing courses will become potentially successful marketing managers? Ten percent______ Twenty percent______ Thirty percent______ ______percent______
Copyright © Houghton Mifflin Company. All rights reserved.10 | 28 Can the Respondent Answer the Question? (Cont’d) Consider the following question taken from a questionnaire used by a marketing research firm in conducting a face-to-face survey of female household heads –How much has your family spent in the last 12 months on cookware? $ _______
Copyright © Houghton Mifflin Company. All rights reserved.10 | 29 Writing a Questionnaire for International Markets Caution –Use of the same word can have different meanings in different cultures Word Equivalence –In Japan, China and non-English speaking European countries “Very Satisfied” means “Somewhat Satisfied” “Very Satisfied” should be replaced by “Totally Satisfied”
Copyright © Houghton Mifflin Company. All rights reserved.10 | 30 Will the Respondent Answer the Question? Questions about personal financial matters or sexual behavior are examples of sensitive questions Words used in the questions should not be ambiguous to the respondents
Copyright © Houghton Mifflin Company. All rights reserved.10 | 31 Double-Barreled Questions Suppose the following question is to be used in conducting a survey of the general public –Do you feel firms today are concerned about their employees and customers? _____ Yes _____ No _____ No opinion A “no” response can be interpreted three different ways –The respondent feels firms are concerned about neither employees nor customers –The respondent feels firms are concerned about employees but not customers –The respondent feels firms are not concerned about employees although they are concerned about customers
Copyright © Houghton Mifflin Company. All rights reserved.10 | 32 Avoiding Double-Barreled Questions This revision is usually accomplished by breaking one question into several questions Thus our illustrative question can be reworded as two separate questions –Do you feel firms today are concerned about their employees? ___ Yes ___ No ___ No opinion –Do you feel firms today are concerned about their customers? ___ Yes ___ No ___ No opinion
Copyright © Houghton Mifflin Company. All rights reserved.10 | 33 Leading Questions Don't you think offshore drilling for oil is environmentally unsound? ___Yes ___ No ___ No opinion Do you think the quality of products on the market today is as high as it used to be ten years ago? ___ Yes ___ No ___ No opinion
Copyright © Houghton Mifflin Company. All rights reserved.10 | 34 Avoiding Leading Questions The two leading questions we saw earlier can be modified for neutrality as follows –What is your feeling about the environmental impact of offshore drilling for oil? ___ Offshore drilling is environmentally sound ___ Offshore drilling is environmentally unsound ___ No opinion
Copyright © Houghton Mifflin Company. All rights reserved.10 | 35 Avoiding Leading Questions (Cont’d) How do you feel about the quality of products on the market today compared with the quality of products on the market ten years ago? ___Product quality is better now than ten years ago. ___Product quality is worse now than ten years ago. ___Product quality is the same now as ten years ago. ___ No opinion.
Copyright © Houghton Mifflin Company. All rights reserved.10 | 36 One-Sided Questions Do you feel firms today are concerned about their employees? ___Yes ___ No ___ No opinion Would you agree or disagree that the deregulation of the airline industry has benefited customers? ___Yes ___ No ___ No opinion
Copyright © Houghton Mifflin Company. All rights reserved.10 | 37 Avoiding One-sided Questions Do you feel firms today are concerned about their employees? ___Concerned ___ Unconcerned ___ No opinion Do you feel the deregulation of the airline industry has benefited consumers, has had no impact on consumers, or has hurt consumers? ___Has benefited customers ___Has had no impact on customers ___Has hurt customers ___No opinion
Copyright © Houghton Mifflin Company. All rights reserved.10 | 38 Unbalanced Questions How important is price to you in buying a new car? ___More important than any other factor ___Extremely important ___Important ___Somewhat important ___Unimportant
Copyright © Houghton Mifflin Company. All rights reserved.10 | 39 Balanced Questions How important is price to you in buying a new car? ___Very important ___Relatively important ___Neither important nor unimportant ___Relatively unimportant ___Very unimportant
Copyright © Houghton Mifflin Company. All rights reserved.10 | 40 Avoiding Questions with Implicit Assumptions Are you favorable, indifferent, or unfavorable toward a 10% increase in city taxes? ___Favorable___Indifferent ___ Unfavorable The above question can be improved as –Are you favorable, indifferent, or unfavorable toward a 10% increase in city taxes to repair potholes in the city's streets? ___Favorable___Indifferent ___Unfavorable
Copyright © Houghton Mifflin Company. All rights reserved.10 | 41 Avoiding Questions with Implicit Assumptions (Cont’d) Do you drink coffee while watching football, baseball, and so on? ____ Yes____ No The above question can be reworded as –During which of the following activities do you drink coffee?(Check as many as apply.) ___ Attending ballgames ___Attending sporting events other than ballgames ___Watching ballgames on TV ___Watching sporting events other than ballgames on TV
Copyright © Houghton Mifflin Company. All rights reserved.10 | 42 Guarding Against Errors How often do you eat eggs for breakfast? _____ Frequently _____ Occasionally _____ Rarely _____ Never
Copyright © Houghton Mifflin Company. All rights reserved.10 | 43 Guarding Against Errors (Cont’d) To guard errors due to misinterpretations, the previous question can be revised as –On average, how many days per week do you eat eggs for breakfast? ___ Every day ___ 5 or 6 days ___ 3 or 4 days ___ 1or 2 days ___ Less than 1 day per week ___ Never eat eggs for breakfast
Copyright © Houghton Mifflin Company. All rights reserved.10 | 44 Complex Questions In which of the following do you typically invest your liquid assets? ___Insured accounts ___Stock market ___Insured accounts and stock market ___Other accounts Of the total number of miles you drove during the past month, approximately what percentage was traveled in driving to and from work? ___Percent
Copyright © Houghton Mifflin Company. All rights reserved.10 | 45 How Does USA Today Avoid Biases in Polling? Question posed to Jim Norman in Hackensack, New Jersey “With all of the little idiosyncrasies available to pollsters to potentially skew the answers of the respondents, e.g., voice tone, making statements vs. asking questions (a.k.a. push polling), or the historical success of polling where the last choice offered automatically becomes the preferred answer,with all of these potential manipulations, how does a polling unit like yours avoid ANY appearance of partisanship or impropriety?”
Copyright © Houghton Mifflin Company. All rights reserved.10 | 46 How Does USA Today Avoid Biases in Polling? (Cont’d) Jim Norman Responds –“Voice Tone: There can be a wide range in the professionalism of the interviewers from polling outfit to polling outfit, but the best outfits use people who are TRUE professionals. With them voice tone is not an issue.” –“Making statements vs. asking questions: If you sat through just one of our often excruciating conference calls to determine question wording, you would know how much the people involved are concerned that we wring out any bias in our questions. It still can happen,but I think it’s rare.”
Copyright © Houghton Mifflin Company. All rights reserved.10 | 47 How Does USA Today Avoid Biases in Polling? (Cont’d) Jim Norman Responds (Cont’d) –“Last choice offered automatically becomes the preferred answer: we rotate the responses, so that each possible response is the last one offered to an equal number of respondents.”
Copyright © Houghton Mifflin Company. All rights reserved.10 | 48 Sequencing of Questions Position of Demographic and Sensitive Questions Arrangement of Related Questions Funnel and Inverted-Funnel Sequences Skip Patterns
Copyright © Houghton Mifflin Company. All rights reserved.10 | 49 Position of Demographic and Sensitive Questions Respondents' personal or demographic characteristics, i.e., age, education level, or income are placed at the end of the questionnaire
Copyright © Houghton Mifflin Company. All rights reserved.10 | 50 Arrangement of Related Questions Cluster questions that focus on the same topic –In a survey of business executives Views on the economy Competition Employee turnover Grouping questions into meaningful clusters can increase respondents' ease in answering the questions and reduce the chance of response errors
Copyright © Houghton Mifflin Company. All rights reserved.10 | 51 Funnel Sequences Which of the following types of TV shows do you watch? (Check all categories that apply.) ____ News shows ____ Quiz shows____ Game shows Which of the following types of TV shows do you like the most? (Check as many categories as apply.) ____ News shows ____ Quiz shows____ Game shows
Copyright © Houghton Mifflin Company. All rights reserved.10 | 52 Funnel Sequences (Cont’d) Which of the following specific shows did you watch during the past seven days? (Check "yes" or "no" for each show listed.) Deal or No Deal ___ Yes ___ No Wheel of Fortune ___ Yes ___ No CBS Evening News___ Yes___ No
Copyright © Houghton Mifflin Company. All rights reserved.10 | 53 Inverted-Funnel Sequences A survey of consumers to measure the image they have of Netgrocer.com What do you think of the specific attributes of Netgrocer.com? ___ Website ___ Products ___ Customer Service
Copyright © Houghton Mifflin Company. All rights reserved.10 | 54 Skip Patterns Proper sequencing of questionnaire items is essential to avoid complicated skip patterns that may confuse interviewers and respondents
Copyright © Houghton Mifflin Company. All rights reserved.10 | 55 Question Sequence Needing Improvement 11. Do you own or rent your current place of residence? Own Rent 12. How long have you lived in this state? Less than one year One year to less than 5 years 5 years or more 13. If you have lived outside this state, in which state did you live immediately before moving here?
Copyright © Houghton Mifflin Company. All rights reserved.10 | 56 Question Sequence Needing Improvement (Cont’d) 14. If you have lived in this state 5 years or more and if you currently rent your residence, do you intend to buy a home within the next two years? Yes No Don’t know 15. How long have you lived in your current place of residence? Less than one year One year to less than 5 years 5 years or more
Copyright © Houghton Mifflin Company. All rights reserved.10 | 57 Improved Question Sequence 11. Have you always lived in this state? Yes (Go to Question 12) No (Go to Question 11a) 11a. In which state did you live immediately before moving into this state? 12. How long have you lived in this state? Less than one year One year to less than 5 years 5 years or more
Copyright © Houghton Mifflin Company. All rights reserved.10 | 58 Improved Question Sequence (Cont’d) 13. Do you own or rent the place where you live? Own (Go to Question 15) Rent (Go to Question 14) 14. Do you intend to buy a home within the next two years? Yes No Don’t know 15. How long have you lived in your current place of residence? Less than one year One year to less than 5 years 5 years or more
Copyright © Houghton Mifflin Company. All rights reserved.10 | 59 Questionnaire Appearance And Layout Version 1 –How old are you? ___Less than 18 ___18 to 25 ___26 to 40 ___Over 40 Version 2 –How old are you? ___Less than 18 ___18 to 25 ___26 to 40 ___Over 40
Copyright © Houghton Mifflin Company. All rights reserved.10 | 60 International Marketing Research First Step –A professional who is fluent in both the languages should translate the original questionnaire Second Step –A different translator must translate the local language questionnaire back to the original version Third Step –Researchers should compare the original version and the translated original version to ensure that they are identical in meaning
Copyright © Houghton Mifflin Company. All rights reserved.10 | 61 International Marketing Research (Cont’d) Fourth Step –The local language version should also be checked by the marketing research professionals in the country of interest Fifth Step –The local language version should be pretested to make sure that the questionnaire serves the intended purpose
Copyright © Houghton Mifflin Company. All rights reserved.10 | 62 Pretesting Pretesting is administering a questionnaire to a limited number of potential respondents and other individuals capable of pointing out design flaws
Copyright © Houghton Mifflin Company. All rights reserved.10 | 63 Questionnaires for Computerized and Online Interviewing –Randomizing response choices – Checking for response consistency – Incorporating complex skip patterns – Personalization –Ability to draw questions from computer libraries – Adding “new” response categories
Copyright © Houghton Mifflin Company. All rights reserved.10 | 64 Randomizing Response Choices The computer can be programmed to randomize the order of presentation of the categories separately for each respondent for multiple-category questions in which category sequence may influence response choice
Copyright © Houghton Mifflin Company. All rights reserved.10 | 65 Checking for Response Category The computer can be programmed to check for consistency between the response given to a question at hand and responses given to certain key questions asked earlier
Copyright © Houghton Mifflin Company. All rights reserved.10 | 66 Incorporating Complex Skip Patterns Computerized online interviewing can handle very complex skip patterns The computer can check as many “if then” statements and previous responses as necessary,decide which question should be asked next and pose that question on the computer screen instantaneously
Copyright © Houghton Mifflin Company. All rights reserved.10 | 67 Personalization When a respondent’s name is entered into the computer keyboard at the start of an interview, that name can be automatically inserted into key questions and instructions throughout the questionnaire to provide a degree of personalization that would be very time consuming to achieve in non- computerized interviewing
Copyright © Houghton Mifflin Company. All rights reserved.10 | 68 Ability to Draw Questions from Computer Libraries A questionnaire writer simply clicks and drags a standard questionnaire from an online library and modifies it as necessary to create a desired questionnaire
Copyright © Houghton Mifflin Company. All rights reserved.10 | 69 Adding New Response Categories In a multiple-category question with an “other” category, when a prespecified number of respondents provide the same open-ended response to the “other” category, that response can be automatically converted into an explicit check off category by adding it to the set of prespecified categories
Copyright © Houghton Mifflin Company. All rights reserved.10 | 70 Designing Cover Letters For Mail Questionnaires Primary purpose –To win the cooperation of respondents A good cover letter should be – Concise – Objective
Copyright © Houghton Mifflin Company. All rights reserved.10 | 71 Exhibit 10.5 Illustrative Cover Letter and Key Features
Copyright © Houghton Mifflin Company. All rights reserved.10 | 72 Openers for Personal and Telephone Interviews Features –An appropriate salutation, such as "Good morning! I am Ralph Johnston with Marketing Research Associates Company.” –A brief statement about the project and its purpose, such as "We are conducting a survey of heads of household regarding their feelings about modern home appliances.”
Copyright © Houghton Mifflin Company. All rights reserved.10 | 73 Openers for Personal and Telephone Interviews (Cont’d) Features (Cont’d) –An indication of how long the interview might last, such as "This survey should take no more than ten minutes.” –A polite request for permission to conduct the interview, such as "May I please talk to the male or female head of your household for a few minutes?"
Copyright © Houghton Mifflin Company. All rights reserved.10 | 74 Designing Observation Forms Guidelines – Wording – Sequencing – Layout
Copyright © Houghton Mifflin Company. All rights reserved.10 | 75 Exhibit 10.6 Observation Form for Recording Characteristics and Behavior of Customers Stopping at a Special Display
Copyright © Houghton Mifflin Company. All rights reserved.10 | 76 Creating Online Questionnaires Creating an online questionnaire is simplified by a growing number of websites that allow researchers to design and launch their survey from one site SurveyZ.com is an example of one such site
Copyright © Houghton Mifflin Company. All rights reserved.10 | 77 SurveyZ SurveyZ allows users to register for a free trial and sample the functionality of the website There are also a number of sample surveys posted for users to view The site is a powerful tool that allows anyone to create a professional looking surveys very quickly
Copyright © Houghton Mifflin Company. All rights reserved.10 | 78 Using SurveyZ Surveyz! The Ultimate Survey Software Solution - Online Survey Software - Customer Satisfaction Survey - Employee Survey - Web Survey SoftwareSurveyz! The Ultimate Survey Software Solution - Online Survey Software - Customer Satisfaction Survey - Employee Survey - Web Survey Software Follow the screen shots in your text to explore the website