1. The (mis)measure of people

2. Agenda Basics of survey structure & questionnaire design Types and formatting of items

3. What kind of surveys do you typically receive during the year? What is your usual response to them? What makes them more intriguing so that you take them? Think of an example for your survey– customer satisfaction Questions

4. Targeting your survey who is your intended market? how diverse are they? what do you most need to know? what level of information is needed? how are they best accessed? what are their characteristics that may affect response? what are their expected objections?

5. Levels of measurement

6. Designing the Questionnaire Step 1:Specify what information will be sought Step 2:Determine type of questionnaire and administration method Step 3:Determine content of individual questions Step 4:Determine form of response to each question Step 5:Determine wording of each question Step 6: Determine sequence of questions Step 7:Determine physical characteristics of questionnaire Step 8:Reexamine steps 1 - 7 and revise if necessary Step 9:Pretest questionnaire and revise if necessary

7. Wording of Questions (a) Use simple words & short sentences (b) Avoid ambiguous words and questions (occasionally, sometimes, often, etc. Try to anchor each point with a specific description) (c) Avoid leading questions (d) Avoid implicit alternatives (e) Avoid implicit assumptions (f) Avoid generalizations and estimates (g) Avoid double-barreled questions (h) Avoid response order (i.e., position) bias

8. The attribute "customer service," for example, can be phrased as any one of the following: Direct question How satisfied are you with customer service? [Very satisfied, satisfied, Neither satisfied nor disatisfied, Disatisfied, Very Dissatisfied Direct request Please rate customer service. [Excellent, Very Good, Good, Fair, Poor] Statement with which to agree or disagree (Likert scale) ABC Company provides good customer service. [Strongly agree, Agree, Neither agree nor disagree, Disagree, Strongly Disagree] Question of frequency How often does customer service exceed expectations? [Very Frequently, Frequently, Neither frequently nor infrequently, Infrequently, Very infrequently] Question of extent To what extent does customer service exceed expectations? [To very great extent, To great extent, To some extent, To little extent, To very little extent]

9. Flame-broiled or Fried? Original Question: “Do you prefer your hamburgers flame-broiled or fried?” Result:flame-broiling (Burger King) beat frying (McDonald’s) by a 3-1 margin Revised Question #1: “Do you prefer a hamburger that is grilled on a hot stainless-steel grill or cooked by passing the raw meat through an open gas flame?” Result:53% preferred the grill (McDonald’s) Revised Question #2: “The chain that grills on a hot stainless-steel griddle serves its cooked hamburgers at the proper temperature without having to use a microwave oven. And the chain that uses the gas flame puts the hamburgers after they are cooked into a microwave oven before serving them. Just knowing this, from which of these two chains would you prefer to buy a hamburger?” Result:the grill (McDonald’s) beat the flame (Burger King) by a 5.5 to 1 margin

10. Handling Sensitive Questions Don’t ask unless absolutely necessary! Give broad response categories (e.g., income, age) Place near end of questionnaire “Hide” them in less sensitive questions Use counterbiasing statement Use randomized response technique Transition sections & inform of rights

11. Use of Counterbiasing Statement “Recent studies have shown that a high percentage of males use their wives’ cosmetics to hide blemishes. Have you used your wife’s cosmetics in the past week?” Providing an acceptable context for answering what might have been sensitized before

12. Rank-Order Scales Rank the following soft-drinks from 1 (best) to 5 (worst) according to your taste preference: Coca-Cola_____ 7-Up_____ Dr. Pepper_____ Pepsi-Cola_____ Tab_____

13. Paired Comparison Scale For each of the following pairs, which soft-drink do you think is better (please check one soft-drink for each pair). ____Coca-Colaor______7-Up ____7-Upor ______Pepsi-cola ____Tabor ______Coca-Cola ____Dr. Pepperor ______Pepsi-cola ____ Pepsi-colaor ______Tab ____ Coca-Cola or ______ DR. Pepper ____ Pepsi-cola or ______Coca-Cola ____Tabor ______ Dr. Pepper ____7-Upor ______Tab ____ Dr. Pepper or ______ 7-Up

14. Constant Sum Scales Allocate a total of 100 points among the following soft-drinks depending on how favorable you feel toward each; the more highly you think of each soft-drink, the more points you should allocate to it. (Please check that the allocated points add to 100.) Coca-Cola_____points 7-Up_____ points Dr. Pepper_____ points Tab _____ points Pepsi-Cola _____ points 100 points

15. Customized Rating Scales Odd vs. Even Scale Points Odd Strongly Agree_____ Agree_____ Neutral _____ Disagree_____ Strongly disagree_____ Even Strongly Agree_____ Agree _____ Disagree _____ Strongly disagree___

16. Balanced vs. Unbalanced Scales Balanced Very good______ Good ______ Fair______ Poor______ Very Poor______ Unbalanced Excellent______ Very Good ______ Good______ Fair______ Poor______

17. Forced vs. Unforced Scales Forced Extremely Reliable___ Very Reliable___ Somewhat Reliable ___ Somewhat Unreliable ___ Very Unreliable ___ Extremely Unreliable___ Unforced Extremely Reliable___ Very Reliable___ Somewhat Reliable ___ Somewhat Unreliable ___ Very Unreliable ___ Extremely Unreliable___ Don’t know___

18. Labeled vs. End Anchored Scales Labeled Excellent_____ Very Good_____ Fair _____ Poor_____ Very Poor_____ End Anchored Excellent_____ _____ _____ Poor_____

19. Number of Scale Points 5 Point Excellent_____ _____ Poor_____ 10 Point Excellent_____ _____________ _____________ Poor

20. Recommended Number of Scale Points Amount of Information Number of Scale Points 1 2 3 4 5 10.................100 More than 5 options will provide little additional discrimination Begins to flatten out in return of information

21. Agreement : Strongly Agree Agree Undecided Disagree Strongly Disagree Frequency : Very Frequently Frequently Occasionally Rarely Very Rarely Never Importance : Very Important Important Moderately Important Of Little Importance Unimportant Quality : Extremely Poor Below Average Average Above Average Excellent Likelihood : Almost Always True Usually True Often True Occasionally True Sometimes But Infrequently True Usually Not True Almost Never True Formats for Likert Items 1…2…3…4…5

22. Semantic Differential Scale X= ABC Bank 0= NBC Bank old-fashioned ____ ____ ____ ____ ____ ____ ____ modern not friendly ____ ____ ____ ____ ____ ____ ____ friendly expensive ____ ____ ____ ____ ____ ____ ____ inexpensive not trustworthy ____ ____ ____ ____ ____ ____ ____ trustworthy XO XO XO XO

23. Student’s fruit preferences by eating occasion (importance of time in consumption decisions) grape kiwi apple plum orange strawberry banana peach grape kiwi apple plum orange strawberry banana peach grape kiwi apple plum orange strawberry banana peach Breakfast Snack during day Supper dessert PreferencesPreferences

24. Physical Characteristics of Questionnaire: Recommendations questionnaire should appear simple to complete (white space is your friend!) minimize number of pages (smaller fonts are OK, provided form appears simple) & items mix-up response formats occasionally (avoids response set bias and breaks monotony) use directions as necessary for each group of items, but keep them short and simple number items within each section have respondents check boxes rather than lines Use consistent response formats (don’t change too often) use shading, boxes, lines, etc., to keep it interesting

25. Question considerations start with easy, non-threatening questions (warm up) place more personal, threatening questions at end never start mail survey with open-ended questions for historical demographics, follow chronological order ask about one topic at a time when switching topics use a transition reduce response set for filter or contingency questions, make a flow chart thank the respondent at beginning and end keep the survey short keep language simple include “don’t know” and “NA” for appropriate options try not to make questions dependent on each other

26. Foot in the door… First make a small request, then when granted, make a larger (more desirable) request works best with prosocial/altruistic requests works best if no extra incentives are offered 10% improvement over simple request, 20% if no incentive works by cognitive dissonance & self esteem

27. Door in the face… Make a large request, then when it is refused, make a smaller (more desired) request more effective if prosocial/altruistic (do it for everybody) than selfish should be no delay between requests slight delay produces 10% improvement, no delay 20% works by reciprocal concessions

28. Supplementary Slides (not assigned for study)

29. Scale Development define target population define specific domains to be measured generate items reflect each domain construct item format construct survey test items for feedback revise administer to representative group frequency distribution (representative?) Cronbach’s alpha (above.70?) validation & reliability checks complete final version Items

30. Cronbach’s alpha as a measure of scale internal reliability

31. Frequency distribution Cross tab & chi-square Correlation & regression t-test & ANOVA Cluster analysis CHAID MDS Analyzing Data

32. Frequency distribution

33. Crosstabs & chi-square If significant, then rows & columns are dependent

34. Correlation & regression r =.50 r 2 =.25

35. T-test: Comparison of differences between means

36. ANOVA & Post Hoc Comparisons

37. Each cluster represents a similarity grouping Cluster Analysis: hierarchical grouping on the basis of similarity Unity

38. Multidimensional Scaling

39. People with older computers more likely to upgrade Sole user more likely to personalize by upgrading Influence of system characteristics Online realize need for faster modem CHAID Focus on less recent PC buyers, target for upgrade packages with faster modems

40. Comparison of Three Primary Methods of Administration mail telephone personal (A) Sampling Control ability to secure sampling frame++ ability to secure correct respondent-++ response rate-++ (B) Information Control ability to probe for detailed answers-+ ability to handle complex information-+ amount of information obtained-+ flexibility of question sequencing-++ protection from interviewer bias+-- ability to obtain personal information+-- ability to show visual displays-+ ability to establish rapport with respondent-+ ability to offer anonymity+-- (C) Administrative Control time requirements-+ cost requirements+- quality control / supervisory requirements+-

41. Return Rate sample size calculator notify beforehand free postage (SASE) personalize (name & address) relevant to needs & interests incentives closing deadline in 2-2.5 weeks reminder security, confidentiality, anonymity if <50% check for bias (what have in common) mail-ins have 3-5% response rate (10% is good)

42. Considerations for Web Based Surveys Draft the questions Provide a password (if needed) E-mail reminders (know return cutoffs) Personalize them (mailing list) Design for people with minimal skills Use a larger font Response icons Textured background Open questions use a text box E-mail to researcher for comments Thank you page

43. Personal and Telephone Interviews: “foot-in-the-door” strategy (Reingen & Kernan 1977) Mail Surveys (1) pre-notification, especially by telephone (Schelegelmilch & Diamantopoulos 1991) (2) cover letter / type of appeal (Houston and Nevin 1977) (3) use SASE, not business reply envelope (4) personalized address (5) no bulk mail (6) sponsorship (if corporate, refer to “research department” and never include sales pitch (7) monetary incentives (Brennan, Hoek & Astridge 1991) (8) offer survey results (9) follow-up postcard, duplicate questionnaire, or telephone call