2. 14 Questionnaire Design and Issues 1234 0001 897251 00000 CHAPTER 14-2
Kraft Foods addressed two of the key issues involved with designing questionnaires; namely, “How do we increase respondent cooperation and reduce survey costs and timing?” (Alert Magazine, Vol.38. No. 12 (December 2000). One of the key recommendations which came out of an extensive study was to focus on the most actionable aspects of surveys, and information that isn’t obtained anywhere else. The flowerpot approach to survey design, which takes up the lion’s share of discussion in Chapter Fourteen, goes a long way toward approximating this goal.
14-2

3. The Value of A Questionnaire
Regardless of whether you choose a career in marketing research the ability to distinguish a “good” from a “bad” survey is an essential skill to acquire.
A questionnaire is the “central processing unit” (CPU) for capturing primary data needed to assist management in addressing business opportunities, and resolving marketing information problems.
Many business issues are not one-dimensional in nature; therefore an instrument needs to be designed to collect information from a number of different perspectives and positions to these issues and questions to closure.
A “good” questionnaire exhibits the properties of logic, organization, and the scientific method.
When class participants begin to design questionnaires there’s a tendency to emphasize “art” rather than “form” or science. More often than not, the students in your class will have been exposed to (and requested to complete) a page or two parading as a survey when it’s little more than a cat’s breakfast. Ask your students if they’ve been exposed this kind of “crazy quilt” questionnaire design before taking this course. Solid examples will go a long way to set the stage for your discussion of the flowerpot approach in later slides. Any examples which you have should be brought to class in order to stimulate discussion.
14-3

4. Four Components of a Questionnaire
Words
Questions
Formats
Hypotheses
The first, second, and third items are, by this time, fairly transparent to your class participants. However, it’s crucial that you spend some time emphasizing that working hypothesis must be present within the questionnaire. Odd as it seems, class participants often sculpt a questionnaire and leave the opportunity to test the very hypothesis which got them launched on the journey out of the landscape. Still, the degree to which you have to cover these components depends on the chemistry of the particular class. A useful way to incite a spirited class discussion is with the subject of “bad” questions which surfaces on page of our text. Double-barreled questions appear to be the most prevalent slip-up with provisional drafts of survey instruments requested for student projects, or the review and discussion questions which appear at the end of the chapter. Insure your students understand what a double-barreled question is and why it poses so many challenges when we get around to analyzing the data captured in a sampling effort later in the course.
14-4

5. Types of Hypotheses in Information Research
There is no significant difference between the preferences toward specific banking methods exhibited by white-collar customers and blue-collar customers.
No significant differences will be found to exist in the requests for specific medical treatments from emergency medical walk-in clinics between users and nonusers of annual physical preventive health care products.
Null Hypothesis
Significant differences do exist in the sexual be- havior profiles of academically strong and weak male and female students.
There is a significant difference in the satisfaction levels among 2002 Mazda Millenia owners according to how much they have driven the car.
It is worthwhile spending ten to fifteen minutes on this exhibit, especially if most of the research endeavors in your sections involve causal, rather than exploratory and/or descriptive research studies. It’s recommended that you begin your discussion of hypothesis development by focusing solely on the alternate and null hypothesis. Once you’re satisfied class participants have a good sense of how they’re framed from a conceptual standpoint you can take them up a notch and cover positive, negative, and nondirectional hypotheses.
Nondirectional Hypothesis
14-5a

6. Types of Hypotheses in Information Research
Those students who exhibit high levels of self- confidence and knowledge toward the topic and positive overall study habits will demonstrate low profiles of introverted social behavior.
The greater the amount of outside-classroom studying done by marketing majors, the less chance there is that they will turn to cheating to improve their grades.
Inverse (Negative) Directional Hypothesis
Positive study habits are related positively to GPA.
College students who tend to worry a lot about what other students think of them will tend to be more conservative in their overall classroom and social behavior than those students who don’t worry much about their self-image..
It is worthwhile spending ten to fifteen minutes on this exhibit, especially if most of the research endeavors in your sections involve causal, rather than exploratory and/or descriptive research studies. It’s recommended that you begin your discussion of hypothesis development by focusing solely on the alternate and null hypothesis. Once you’re satisfied class participants have a good sense of how they’re framed from a conceptual standpoint you can take them up a notch and cover positive, negative, and nondirectional hypotheses.
Direct (Positive) Directional Hypothesis
14-5b

7. The Flowerpot Approach: Phase I
Step 1
Transform Research Objectives into Information Objectives
Establish hierarchical order of the objectives.
Establish research hypotheses within each objective.
Step 2
Determine the Appropriate Data Collection Method
Determine who the sampled respondents are.
Step 3
Determine Information Requirements for Each Objective
Establish hierarchical order of requirements.
There are two key things to convey to your class participants regarding the flowerpot approach here – the importance of nailing down which information objectives really represent a flowerpot of data and how many flowerpots will need to be stacked up in designing the questionnaire. As noted in the opening slide regarding Kraft Foods’ efforts in this arena, the beauty of the flowerpot approach is transparent… it helps the research team “winnow out” the neat but not required data requirements, and the stuff that’s just plain irrelevant to the study proper.
Step 4
Develop Specific Question/Scale Measurement Formats
Decide needed type of raw data.
Decide needed question/scale measurement format.
Decide needed question and scale point wording.
Decide what type of data analysis is desired.
14-6

8. The Flowerpot Approach: Phase II
Step 5
Evaluate Question/Scale Measurements
Test for reliability and validity of scales.
Step 6
Establish Flowerpot Format and Layout
Determine the number of information flowerpots needed.
Determine the sizes of the flowerpots needed.
Determine the type of base needed.
While your class participants ought to be “slain in the spirit” of Step 5 given your coverage of Chapter Twelve and Thirteen, Steps 6 and 7 are “patio-stones” on the path of progress which require constant attention and care. The attributes of logic and stability are key to the success of the survey design process at this juncture, both for respondents and the research team. The five questions profiled on page of our textbook act as a portfolio of “checks and balances” to insure the cogency of each question in the questionnaires developed by your student teams. Exhibit 14.6 can be used to provide another evaluative perspective at this point in your delivery, or, (as recommended) later on as a way of wrapping up the key conceptual threads in Chapter Fourteen into a cohesive whole.
Step 7
Evaluate the Questionnaire and Layout
Check the type of scale format, scaling properties, measures of central tendencies, and variations.
Check the information flow of general to specific within each flowerpot, and the identification questions.
14-7

9. The Flowerpot Approach: Phase III
Step 8
Obtain the Client’s Approval
Revise questionnaire layout if needed.
Step 9
Pretest and Revise the Questionnaire
Check for specific ambiguity of words, phrases, instructions.
Check for completion time and difficulties of answering questions
Of the remaining four steps, previous classes have shown the need to really emphasize the importance of Step 9: Perform a pretest of the survey instrument and revise it accordingly. At this point it’s more comfortable for you and for class participants to slide over this key step, especially given the desire to “get on with the game” associated with misconceptions about what a course in marketing research involves. Your efforts to set parameters of accountability (contact names, dates, times, and locations for the pretest activity) will pay dividends later on. In order to justify the importance of pre-testing the survey set aside some class time (preferably in the lab at the end of this week, or the beginning of next week) to discuss the results. There’s never been a survey which didn’t require some revision after a pretest. At this point, probably only you and the members of the team which created the survey have subjected it to any kind of meaningful critique. The litmus test, of course, is not with you or your class participants but with the target population the survey was intended for.
Step 10
Finalize Questionnaire Format
Step 11
Implement the Survey
14-8

10. The Flowerpot Approach: The Big Picture
Raw Data
(Cover Letter)
Introduction Section
First Information Objective Contains the Broadest Information Requirements
General Data/Information Requirements (Questions/Scale Measurements)
More Specific Data Requirements (Questions/Scale Measurements)
Second Information Objective
Contains More Specific Data/Information Requirements
General Question/Scale Measurements
More Specific Question/ Scale Measurements)
If Needed, General Opinion Information Psychographic Questions/Scale Measurements
Identification Section Demographic/Socioeconomic Questions/Scale Measurements
Thank-You Statement
General Information Requirements
Specific Information Requirements
Identification Information Requirements
Rather than walk-through this exhibit where it’s placed in the text, it’s recommended you return to it after you’ve covered each of the phases and steps in the design process.
14-9

11. Rules of Thumb for Evaluating Questions
Questions should be simple and straight forward whenever possible.
Questions should be expressed clearly whenever possible.
Questions should avoid qualifying phrases or extraneous references, unless they are being used as a type of qualifying (screening) factor.
Descriptive words should be avoided, unless absolutely necessary.
The question/setups, attribute statements, and data response categories should be unidimensional, except when there is a need for a multi-response question.
One of the most interesting things to do with this exhibit is to ask your class participants: “How far, and in what ways, do these guidelines shift when we launch a survey instrument online, via a program like “Snap” or “Zoomerang”?
14-10a

12. Rules of Thumb for Evaluating Questions
Raw data response categories (scale points) should be mutually exclusive.
The question/setups and the response categories should be meaningful to the respondent.
Question/scale measurement formats should avoid arrangement of response categories that might bias the respondent’s answer.
Unless called for, question/setups should avoid undue stress of particular words.
Question/setups should avoid double negatives.
One of the most interesting things to do with this exhibit is to ask your class participants: “How far, and in what ways, do these guidelines shift when we launch a survey instrument online, via a program like “Snap” or “Zoomerang”?
14-10b

13. Rules of Thumb for Evaluating Questions
Question/scale measurements should avoid technical or sophisticated language, unless necessary.
Where possible, question/setups should be phrased in a realistic setting.
Question/scale measurements should be designed to read logically.
Question/scale measurements should always avoid the use of double-barreled items.
One of the most interesting things to do with this exhibit is to ask your class participants: “How far, and in what ways, do these guidelines shift when we launch a survey instrument online, via a program like “Snap” or “Zoomerang”?
14-10c

14. Important Considerations in Questionnaire Design
Determine the information objectives and the number of information flowerpots required to meet those objectives.
Determine the specific data requirements (i.e., the size) for each information flowerpot, and stack the pots from largest to smallest.
Introduction section should include a general description of what the study is pertaining to; this may well be in a disguised format.
All types of instructions, if necessary, should be given clear expression.
Perhaps most important, the question/scale measurements have to follow some logical order—that is, an order that appears logical to the respondent rather than to the researcher or practitioner.
As with the previous slide, Exhibit 14.7 gets added value by asking your students to reconsider these “considerations” in terms of some of the offline/online issues in questionnaire design and implementation which surfaced (and hopefully has been maintained) as a theme in Chapter Nine.
14-11a

15. Important Considerations in Questionnaire Design
Begin an interview or questionnaire with simple questions that are easy to respond to, and then gradually lead up to the more difficult questions. Create a general-to-specific data flow.
Postpone highly personal questions (state-of-being data) until late in the interview or survey (i.e., place in the base after the last information flowerpot).
Place questions that involve psychological tests (i.e., lifestyle beliefs) toward the end of the interview or survey, but before the identification base.
Do not ask too many questions of the same measurement format (i.e., nominal, ordinal, interval, ration scale formats) in sequence.
As with the previous slide, Exhibit 14.7 gets added value by asking your students to reconsider these “considerations” in terms of some of the offline/online issues in questionnaire design and implementation which surfaced (and hopefully has been maintained) as a theme in Chapter Nine.
14-11b

16. Important Considerations in Questionnaire Design
Taper off an interview or survey with a few relatively simple questions that do not require extensive thoughts or expressions thoughts or expressions of feelings (i.e., the demographic data questions are very appropriate here).
Always end the interview or survey with the appropriate thank-you statement.
As with the previous slide, Exhibit 14.7 gets added value by asking your students to reconsider these “considerations” in terms of some of the offline/online issues in questionnaire design and implementation which surfaced (and hopefully has been maintained) as a theme in Chapter Nine.
14-11c

17. The Role of a Cover Letter
A solid cover letter is set up with the intention of:
Increasing a respondent’s willingness to complete the survey and return it in a timely fashion.
Improving the response rate associated with the overall goals of the research endeavor.
Introducing the research project and the research team to the respondent.
Instilling a feeling of trust and respect with the respondent, for the study and the client/research team soliciting the data.
As noted in the Instructor’s Manual, you need to decide how much emphasis you want to place on the next two slides. Recently, class participants have provided evidence that they’ve been provided with ample information about the role of a cover letter, the guidelines which need to be brought alongside its development and examples. Still, the rewarding thing about this discussion is how it conveys the information (which students may already have via a career development class or student success programs) with an eye to the principles and practice of doing marketing research.
14-12

18. Guidelines for Developing a Cover Letter
Factors
Description
Personalization
Cover letter should be addressed to the specific prospective respondent; use research firm’s professional letterhead stationery.
Identification of the organization doing the study
Clear identification of the name of the research firm conducting the survey or interview; decide on disguised or undisguised approach of revealing the actual client (or sponsor) of the study.
Clear statement of the study’s purpose and importance
Describe the general topic of the research and emphasize its importance to the prospective respondent.
Anonymity and confidentiality
Give assurances that the prospective respondent's name will not be revealed. Explain how the respondent was chosen, and stress that his or her meaningful input is important to the study’s success.
Again, this is where it makes some sense to highlight how aspects of cover letter development shift in important and exciting ways when a cover letter is used in a research study. One of the best ways of meeting some of the “we’ve had all this in our career management classes” comments is to note that while Factor #1 is shared by cover letters directed to employers and those designed to introduce and legitimate research studies, it can be argued that Factors #2-#10 are singular to the marketing research industry. Once your class participants recognize this “shift” the justification for proceeding with a more in-depth discussion of the importance of cover letters surfaces nicely.
14-13a

19. Guidelines for Developing a Cover Letter
Factors
Description
General time frame of doing the study
Communicate the overall time frame of the survey or interview.
Reinforcement of the importance of the respondent’s participation
Where appropriate, communicate the importance of prospective respondents’ participation.
Acknowledgment of reasons for nonparticipation in survey or interview
Point out “lack of leisure time,” “surveys classified as junk mail,” and “forgetting about survey” reasons for not participating, and defuse them.
Time requirements and compensations
Clearly communicate the approximate time required to complete the survey; discuss incentive program, if any.
Again, this is where it makes some sense to highlight how aspects of cover letter development shift in important and exciting ways when a cover letter is used in a research study. One of the best ways of meeting some of the “we’ve had all this in our career management classes” comments is to note that while Factor #1 is shared by cover letters directed to employers and those designed to introduce and legitimate research studies, it can be argued that Factors #2-#10 are singular to the marketing research industry. Once your class participants recognize this “shift” the justification for proceeding with a more in-depth discussion of the importance of cover letters surfaces nicely.
14-13b

20. Guidelines for Developing a Cover Letter
Factors
Description
Completion date and where and how to return the survey
Communicate to the prospective respondent all instructions for returning the completed questionnaire.
Advance thank-you statement for willingness to participate
Thank the prospective respondent for his or her cooperation.
Again, this is where it makes some sense to highlight how aspects of cover letter development shift in important and exciting ways when a cover letter is used in a research study. One of the best ways of meeting some of the “we’ve had all this in our career management classes” comments is to note that while Factor #1 is shared by cover letters directed to employers and those designed to introduce and legitimate research studies, it can be argued that Factors #2-#10 are singular to the marketing research industry. Once your class participants recognize this “shift” the justification for proceeding with a more in-depth discussion of the importance of cover letters surfaces nicely.
14-13c

21. Cover Letter: An Example to Consider
Marketing Resources Group 3205 Windsor Oaks Drive, Suite 1105 Baton Rouge, Louisiana 70814
CONSUMER BANKING OPINION STUDY BATON ROUGE, LOUISIANA
If you have a bank account—
We need your opinion,
With more bank locations, new banking services, and the growth of credit unions and savings and loans, financial institutions are indeed changing. These changes will have an effect on you and your family, and that’s why your opinion is important.
Your name has been selected in a sample of Baton Rouge residents to determine what people in our community think about the present products and services offered by banks. Your individual opinions in this survey can never be traced back to you, and all results will be held strictly confidential. The results of the study will provide the banking industry with insight into how to better serve the needs of its customers.
The brand-new quarter enclosed with this letter is not enough to compensate you for your time, but it may brighten the day of a youngster you know.
Thank you for your assistance.
Sincerely,
Thomas L. Kirk
MRG Project Director
P.S. Please return no later than June 21, A postage-paid envelope enclosed.
In an effort to “add spice” to this discussion, it would be useful if you can access any cover letters which might have accompanied research efforts performed by a Department for Institutional Research or a Research Center on campus.
14-14

22. Summary of Learning Objectives
Identify and discuss the critical factors that can contribute to directly improving the accuracy of surveys, and explain why questionnaire development is not a simple process.
Discuss the theoretical principles of questionnaire design, and explain why a questionnaire is more than just asking a respondent some questions.
Identify and explain the communication roles of questionnaires in the data collection process.
Explain why the type of information needed to address a decision maker’s questions and problems will substantially influence the structure and content of questionnaires.
List and discuss the 11 steps in the questionnaire development process, and tell how to eliminate some common mistakes in questionnaire design.
Discuss and employ the “flowerpot” approach in developing scientific questionnaires.
Discuss the importance of cover letters, and explain the guidelines to help eliminate common mistakes in cover letter designs.
After hearing any questions, observations, or concerns about coverage of Chapter Fourteen it’s important to inform your class participants that the textbook will take a new and exciting turn for the next four chapters. Chapters Fifteen, Sixteen, Seventeen, and Eighteen focus on preparing data for analysis, and the many and varied statistical techniques which can be brought to bear on the data your student teams will “capture” from respondents in the very near future.. As mentioned in our notes to Chapter One of this Instructor’s Manual, this is perhaps the “stretch” of the journey some of your students dread the most. The key thing at this point is to let them know you are there for them, and the appropriate amount of time will be set aside in the lab over the next four weeks to insure they will finish up your course feeling more comfortable about the quantitative side of marketing research. Again, your job, as the instructor is to deliver a product which has “balance”. The quantitative side of marketing research continues to stand as one of the most fascinating areas of innovation and change, and a “given” when it comes to finding and forging a successful career path in the industry.
14-15