IDENTIFYING TRUE DIFFERENCES ACROSS COUNTRIES WHILE MAXIMIZING DATA QUALITY Stevens Avenue, #214 Solana Beach, CA

A PPROACHES A VAILABLE T ODAY Likert Scales Respondents rate each item on a 5, 7, or 9-point scale. Constant Sum Respondents divide a given sum (usually100) among two or more items. Ranking Respondents rank the items in order. Max-Diff Respondents are shown subsets of the items and asked to specify best/worst for each set. Answers Research

P ROS A ND C ONS O F E ACH A PPROACH Pros Easily administered Easily understood by respondents Cons Heavily influenced by scale use bias Difficult to discern true differences between countries Pros Forces trade-offs Avoids yea-saying Cons Difficult to administer if there are more than 5 items Programming does not work on all browsers if done by web Likert ScalesConstant Sum Answers Research

P ROS A ND C ONS O F E ACH A PPROACH C ONT ’ D Pros Each point is used only once Easily understood by respondents Cons Influenced by order bias Does not allow ties Pros No scale use or order bias Provides better discrimination among items Cons Data is relative not absolute Takes slightly longer to implement Ranking*Max-Diff *Max-Diff is the preferred approach when looking for differences across countries and when relative difference is important. Answers Research

H OW M AX -D IFF W ORKS Step 1: Identify the items you want to test. Item A Item B Item C Item D Item E Item F Item G Item H Item I Item J Item K Item L Answers Research

H OW M AX -D IFF W ORKS Step 2: Put items into pre-defined tasks of 4. In this case we are testing12 items. To complete the exercise respondents will need to complete 9 of these tasks. ○ Item C ○ ○ Item F ○ ○ Item A ○ ○ Item H ○ Tasks 1-9 Answers Research

H OW M AX -D IFF W ORKS Step 3: For each task respondents are asked to select the least important and most important item. Least Important Most Important ○ Item C ○ ● Item F ○ ○ Item A ● ○ Item H ○ Answers Research

W HAT D ID W E L EARN ? Least Important Most Important ○ Item C ○ ● Item F ○ ○ Item A ● ○ Item H ○ C>FA>CA>FA>HH>F Answers Research

T YPICAL O UTPUT Less ImportantMore Important Answers Research

R ECENT S TUDY Importance Score (Max-Diff) Importance Rating (T2B) Importance Rating (Likert mean) Reliability %4.4 TCO %4.3 Speed %4.2 Trust %4.0 Security % 3.8 Quality Support %3.6 Results from the Max-Diff exercise are much cleaner compared to traditional Likert scale data. Little differentiation Distinct tiers Answers Research

R ECENT S TUDY C ONT ’ D Max-Diff Importance ScoresMean Importance Scores USUKJPUSUKJP Reliability TCO Speed Trust Security Quality Support Clear priorities emerge from the Max-Diff scores that would be missed analyzing mean scores. Little differentiation Story differs by Country Answers Research

A BILITY T O S EE P ATTERNS W ITHIN G ROUPS More ImportantLess Important Roughly all US respondents feel Reliability is important. Answers Research

A BILITY T O S EE P ATTERNS W ITHIN G ROUPS More ImportantLess Important Unlike the US, UK respondents are divided on whether Reliability is important. Answers Research

A BILITY T O S EE P ATTERNS W ITHIN G ROUPS More ImportantLess Important While Trust is seen as the most important item in Japan, we also see it is not the consensus. Answers Research

Ron Weidemann Answers Research, LLC x221 Q UESTIONS Answers Research