1. IDENTIFYING TRUE DIFFERENCES ACROSS COUNTRIES WHILE MAXIMIZING DATA QUALITY www.answersresearch.com 380 Stevens Avenue, #214 Solana Beach, CA 92075 858-792-4660

2. A PPROACHES A VAILABLE T ODAY Likert Scales Respondents rate each item on a 5, 7, or 9-point scale. Constant Sum Respondents divide a given sum (usually100) among two or more items. Ranking Respondents rank the items in order. Max-Diff Respondents are shown subsets of the items and asked to specify best/worst for each set. Answers Research 2008 2

3. P ROS A ND C ONS O F E ACH A PPROACH Pros Easily administered Easily understood by respondents Cons Heavily influenced by scale use bias Difficult to discern true differences between countries Pros Forces trade-offs Avoids yea-saying Cons Difficult to administer if there are more than 5 items Programming does not work on all browsers if done by web Likert ScalesConstant Sum Answers Research 2008 3

4. P ROS A ND C ONS O F E ACH A PPROACH C ONT ’ D Pros Each point is used only once Easily understood by respondents Cons Influenced by order bias Does not allow ties Pros No scale use or order bias Provides better discrimination among items Cons Data is relative not absolute Takes slightly longer to implement Ranking*Max-Diff *Max-Diff is the preferred approach when looking for differences across countries and when relative difference is important. Answers Research 2008 4

5. H OW M AX -D IFF W ORKS Step 1: Identify the items you want to test. Item A Item B Item C Item D Item E Item F Item G Item H Item I Item J Item K Item L Answers Research 2008 5

6. H OW M AX -D IFF W ORKS Step 2: Put items into pre-defined tasks of 4. In this case we are testing12 items. To complete the exercise respondents will need to complete 9 of these tasks. ○ Item C ○ ○ Item F ○ ○ Item A ○ ○ Item H ○ Tasks 1-9 Answers Research 2008 6

7. H OW M AX -D IFF W ORKS Step 3: For each task respondents are asked to select the least important and most important item. Least Important Most Important ○ Item C ○ ● Item F ○ ○ Item A ● ○ Item H ○ Answers Research 2008 7

8. W HAT D ID W E L EARN ? Least Important Most Important ○ Item C ○ ● Item F ○ ○ Item A ● ○ Item H ○ C>FA>CA>FA>HH>F Answers Research 2008 8

9. T YPICAL O UTPUT Less ImportantMore Important Answers Research 2008 9

10. R ECENT S TUDY Importance Score (Max-Diff) Importance Rating (T2B) Importance Rating (Likert mean) Reliability 27.362.3%4.4 TCO 24.458.3%4.3 Speed 18.658.6%4.2 Trust 17.456.3%4.0 Security 7.753.5% 3.8 Quality Support 4.645.0%3.6 Results from the Max-Diff exercise are much cleaner compared to traditional Likert scale data. Little differentiation Distinct tiers Answers Research 2008 10

11. R ECENT S TUDY C ONT ’ D Max-Diff Importance ScoresMean Importance Scores USUKJPUSUKJP Reliability 27.3 26.418.1 4.43.93.1 TCO 24.4 24.517.5 4.33.93.2 Speed 18.6 17.58.1 4.23.63.9 Trust 17.4 18.129.3 4.03.94.1 Security 7.7 9.220.2 3.8 3.23.5 Quality Support 4.6 4.36.8 3.64.13.7 Clear priorities emerge from the Max-Diff scores that would be missed analyzing mean scores. Little differentiation Story differs by Country Answers Research 2008 11

12. A BILITY T O S EE P ATTERNS W ITHIN G ROUPS More ImportantLess Important Roughly all US respondents feel Reliability is important. Answers Research 2008 12

13. A BILITY T O S EE P ATTERNS W ITHIN G ROUPS More ImportantLess Important Unlike the US, UK respondents are divided on whether Reliability is important. Answers Research 2008 13

14. A BILITY T O S EE P ATTERNS W ITHIN G ROUPS More ImportantLess Important While Trust is seen as the most important item in Japan, we also see it is not the consensus. Answers Research 2008 14

15. Ron Weidemann Answers Research, LLC rweidemann@answersresearch.com 858-792-4660 x221 www.AnswersResearch.com Q UESTIONS Answers Research 2008 15