1. BAD BOY MATRIX QUESTION Whatcha gonna do when they come for you? Florian Tress ODC Services GmbH, Germany GENERAL ONLINE RESEARCH 2012 5 – 7 March 2012 at the DHBW Mannheim

2. 2 THE VENN DIAGRAM OF FIELDWORK PEOPLERESEARCHERS Fun & entertainment Incentives Disport Curiosity Express an opinion Solve problems Collect (a lot of) valid data Gain a lot of knowledge Implement special (and possibly boring) methods

3. 3 BAD BOY MATRIX QUESTION PEOPLE  Monotonous, boring  Overwhelming variety of options: “decision paralysis”  Nondifferentiation, satisficing behavior  Another question format would be more appropriate: Inferior data RESEARCHERS  Standardized data format, comparability of responses (with common orientation)  Maximize amount of data, minimize length of interview  Facilitate statistical procedures, e.g. factorial analysis, indices, etc.  Availability of validated test instruments, e.g. Big Five

4. 4 STANDARD ALTERNATIVES MULTIPLE CHOICE  Matrix with a two point scale  Additional option “None of these” (mandatory question) RANKING  Bring statements in an order  Perfectly differentiated data

5. 5 SPECIAL ALTERNATIVE: THE CAROUSEL  Only one statement presented at once  New statements slide in from the left  Response options remain in the same place

6. 6 SPECIAL ALTERNATIVE: DRAG ME  All statements presented at once  Center stands for respondent  Measures distance from center  Arrangement of statements unimportant

7. 7 COMPARISON OF THESE ALTERNATIVES Monadic Questionnaire : Random assignment to one of these alternatives n = 1080; 216 interviews per alternative; good spread over age and education Indicators: Comparability, Trustworthiness, Data Quality, Satisfaction, Technical Requirements

8. 8 COMPARABILITY: BRAND LIKEABILITY 3 Factors (Cum.%: 49) | Cronbach‘s α: n.a. 3 Factors (Cum.%: 56) | Cronbach‘s α: 0,77.3 Factors (Cum.%: 58) | Cronbach‘s α: 0,78. 3 Factors (Cum.%: 57) | Cronbach‘s α: 0,73. 3 Factors (Cum.%: 60) | Cronbach‘s α: 0,80.

9. 9 COMPARABILITY: ATTITUDES (A) Wenn ich gute Erfahrungen mit einer Marke mache, empfehle ich sie aktiv weiter. (B) Werbung sollte mich stärker über Marken informieren, die ich noch nicht kenne. (C) Ich habe feste Marken, die ich bevorzugt einkaufe. (D) Neuartige und innovative Produkte passen gut zu meinem Lebensstil. (E) Ich probiere häufig neue Marken aus, die ich noch nicht kenne. (F) Ich bevorzuge Marken, die auf eine lange Tradition zurückblicken. (G) Ich bin bereit, für Markenprodukte mehr Geld auszugeben. (H) Werbung sollte mich stärker über Marken informieren, die ich bereits gut kenne. (I) Je bekannter eine Marke ist, desto leichter kann man ihr vertrauen. (J) Die Bekanntheit einer Marke sagt etwas ihre Qualität aus. 3 Factors (Cum.%: 51) | Cronbach‘s α: n.a. 3 Factors (Cum.%: 50) | Cronbach‘s α: 0,56.3 Factors (Cum.%: 61) | Cronbach‘s α: 0,80. 3 Factors (Cum.%: 63) | Cronbach‘s α: 0,82.

10. 10 RESULT: COMPARABILITY WINNER Results correspond (roughly) for the most / least likeable brands (agreeable statements) But: Drag Me seems to measure something different / to be biased by third variables LOSER

11. 11 TRUSTWORTHINESS: BRAND LIKEABILITY MULTIPLE CHOICE Ø Words pos.: 5,5 Ø Words neg.: 5,4 Non-Response: 3% MATRIX Ø Words pos.: 6,8 Ø Words neg.: 7,3 Non-Response: 2% RANKING Ø Words pos.: 6,3 Ø Words neg.: 6,7 Non-Response: 3% CAROUSEL Ø Words pos.: 7,7 Ø Words neg.: 7,0 Non-Response: 3% DRAG ME Ø Words pos.: 7,2 Ø Words neg.: 7,2 Non-Response: 2% Follow-Up-Exploration: Why do you think, this is the most / least likeable brand?

12. 12 TRUSTWORTHINESS: ATTITUDES MULTIPLE CHOICE Ø Words pos.: 7,4 Ø Words neg.: 8,6 Non-Response: 11% MATRIX Ø Words pos.: 7,9 Ø Words neg.: 9,6 Non-Response: 9% RANKING Ø Words pos.: 8,3 Ø Words neg.: 8,1 Non-Response: 7% CAROUSEL Ø Words pos.: 9,3 Ø Words neg.: 10,0 Non-Response: 8% DRAG ME Ø Words pos.: 8,4 Ø Words neg.: 8,6 Non-Response: 7% Follow-Up-Exploration: Why did you agree / disagree with this statement?

13. 13 WINNER RESULT: TRUSTWORTHINESS LOSER nonspecific / general „because I like it“ specific / rich in detail „good quality, fair prices“

14. 14 DATA QUALITY: NONDIFFERENTIATION Low sample size!

15. 15 DATA QUALITY: NONDIFFERENTIATION MULTIPLE CHOICEMATRIXCAROUSEL Education Ageyounger  40  older lower  A-L.  higher Low sample size!

16. 16 WINNER RESULT: DATA QUALITY LOSER The data is perfectly differentiated with the Ranking and Drag Me Question. Among the other three alternatives, the Carousel performs best.

17. 17 SATISFACTION

18. 18 WINNER LOSER

19. 19 SUMMARY MULTIPLE CHOICE RANKINGMATRIXCAROUSELDRAG ME Comparability Trustworthiness Data Quality Satisfaction Technical Requirements noneJScriptnoneJScript, Flash

20. 20 RECOMMENDATIONS CHECK, IF YOUR STUDY PERMITS THE USAGE OF JSCRIPT AND FLASH! (in most cases, it will) SELECT THE QUESTIONTYPES CAREFULLY! (there might be better alternatives) LAYOUT AND USABILITY MATTER! (the longer the interview, the more they matter) IF YOU HAVE DOUBTS ASK YOUR FIELDWORK PROVIDER! (they should have enough experience)

21. www.odc-services.com f.tress@odc-services.com @FTress FLORIAN TRESS THANK YOU VERY MUCH!