1. Comparative Usability Study Reporting of Results Robert Murphy

2. Study Overview Comparative usability study between SHOP.COM and Amazon / Shopping.com with a concentration on search UI elements and use of search functionality Participants 18 Shoppers –16 women –2 men –2 – age 20-29 –4 – age 30-39 –7 – age 40-49 –5 – age 50-59 –6 w/SHOP.COM experienced –12 w/out SHOP.COM experience –13 shoppers mentioned Amazon as a commonly used ecommerce site

3. Study Overview Methodology Each shopper performed 2 - 4 unaided searches on each website beginning from the home page Total of 90 searches performed (search and navigation) –25 on SHOP.COM and Shopping.com –20 on SHOP.COM and Amazon –SHOP.COM was always presented first Categories of searches included: –jewelry, shoes, sporting goods, clothing, electronics, auto supplies, books & video Questions rating their experience and preference between sites were asked after performing searches and after using both sites Various additional pages were visited, involving tasks in both navigation and general site usage

4. Study Overview Comparison to previous study Generally, improved response over similar study conducted April, 2006 –87% success rate over 60% –Overall satisfaction rating 4.3 over 2.8 Several favorable responses from shoppers: –I had a good experience –You guys have a great product –I like the 1-cart concept

5. Overall Findings SHOP.COM Shoppers successfully found the product they were looking for 87% of the time Shoppers felt the results were relevant in 70% of the searches Shoppers felt an acceptable number of results were returned in 60% of the searches

6. Use of Navigational Elements SHOP.COM Breakdown of total usage of navigational elements: –Performing a search again –Use of Left Navigation links –Using Search Within function –Clicking to view the next page of results –Sorting of results

7. Use of Navigational Elements SHOP.COM Breakdown of total usage of navigation functions by experience on SHOP.COM –Inexperienced shoppers relied primarily on searching again to refine their results –Experienced shoppers utilized searching again, Search Within and (primarily) Left Nav links –Experienced shoppers did not need to go from page to page of results

8. Comparative Analysis Successful Searches Percentage of searches where shopper successfully found the product they were looking for Q: Do you feel you successfully found the product you were looking for?

9. Comparative Analysis Relevancy of Results Percentage of searches in which shopper felt their search returned relevant results Areas identified for improvement on SHOP.COM: –shoes, books, sporting equip, clothing Underperforming category on Amazon: –household Q: When you were searching or navigating through the results, was the relevancy of the products displayed what you expected?

10. Comparative Analysis Quantity of Results Shoppers perception of the amount of results returned for their search NOTE : Amazon only returns the first 48 products (3 pages of results) and does not display the number of results for their all department search Q: Were the amount of results initially returned: too few, too many or an acceptable amount?

11. Comparative Analysis Ease of Navigation Shoppers impression of which site was easier to navigate around in Q: Which site was easier to navigate around?

12. Comparative Analysis Overall Layout Shoppers preference of overall layout of the sites Q: In general, which sites overall layout do you prefer?

13. Comparative Analysis Site Preference Shoppers overall preference of sites –6 of 6 shoppers preferring Amazon, listed Amazon as a commonly used site –2 of 3 shoppers preferring SHOP.COM, listed Amazon as a commonly used site Q: Overall, which site do you prefer?

14. Qualitative Comparative Feedback Shopping.com (vs. SHOP.COM) Pros Price links on top of Left Navigation Cleaner UI Easier to read Less cluttered Cons Fixed width Not enough products above the fold Too far to scroll (defaults to 30 products per page) Some text too small Sort was difficult to locate

15. Qualitative Comparative Feedback Amazon (vs. SHOP.COM) Pros Easier to read Not as busy Better use of spacing; more open feeling Minimum use of boxes, lines Better color usage – organic, subtle colors Price closer to photos Attributes more visible Better logic in attribute (Left Nav) hierarchy Friendlier; more inviting, personal, shopping-like Cons Only 3 pages of results (misleading) Too stretched out Too open No descriptions In-line department links caused confusion Too much information in the lower left navigation bar

16. Qualitative Comparative Feedback SHOP.COM Cons Fonts too small Overwhelming amount of info; too much to read Too crowded Too grid-like; too many boxes and lines Too top-heavy; red search too overwhelming Column of red buttons too much attention No color harmony; too much contrast Grey too dull Cold, clinical, techy, utilitarian Pros Search within Colors OK; easy on the eyes More products above the fold Easy to narrow-by Clean NOTE Row view was primary format of search results

17. Use of Navigational Elements SHOP.COM 9 of 18 shoppers missed the Next Page links at the top of the page Multiple shoppers complained about the size of the text and links in the grey bar Shoppers complained The Sort By links and text seemed to be a lot of links and information Breadcrumb text seemed easy enough for shoppers to indicate quantity of results and location within department hierarchy

18. Search Results Messaging Live Site 6 of 8 shoppers missed the message: There are 0 matches… All shoppers missed the message: Or see matches in all department… (has since changed to non-italized) 5 out of 7 shoppers missed the message: Here are the closest matches… –When pointed out, this particular message was understood

19. Navigation Elements Live Site Very little use of Left Navigation links and Search Within function for first-time shoppers 1 shopper using Left Navigation links, assumed they expanded results 3 additional shoppers were unsure whether links would expand or narrow

20. Navigation Elements Mockup Changes Made Based on Shopper Feedback: Search within moved to left nav 2 sets of labels added to Left Navigation bar Vertical line added to Left Navigation bar Sort removed for All Department search Showing 1-24… enlarged and moved out of gray bar Box removed from photos Vertical line next to price column removed Descriptions shortened

21. Navigation Elements Mockup Results of Usage: Significant increase in usage of Left Nav and Search Within All but 2 shoppers preferred the new Search Within position All shoppers preferred this mockup to original –Better; Cleaner; Less Cluttered All but 1 shopper preferred photos without the outline

22. Result Listings 14 out of 18 shoppers prefer to have the price closer to the product name/photo 13 out of 18 shoppers indicated they liked having the description

23. Attribute Quantities 12 out of 18 shoppers preferred to have the quantities displayed with the attribute links 6 out of 18 shoppers indicated that large quantities of results may discourage them from clicking the links (supports findings in previous study)

24. Single Product Search Results No shopper show any confusion or surprise when 1 product search result or navigation link returned a product page Supports current functionality (recent change to live site)

25. Search-Specific Recommendations Overall Explore options to enhance visibility of Search Messaging Increase font size and visibility of Showing 1-24 of 86 Increase font size of Next Page links Explore dropping Next Page links on top of results Left Nav Bar Keep Search Within at top of Left Nav bar Use 2 Left Nav headers (Search Within and Narrow Results By) Maintain visibility of Left Nav headings/labels (as per peach colored background) Increase font size of attribute listing Continue to restrict showing quantity of results until shopper narrows their results –need to establish rule – is currently inconsistent Left Nav Bar (cont) Show price attributes high in left nav column Remove Showing # and counts from long list of attribute listings (i.e. brand listing) Display of Results Minimize usage of lines Remove boxes around photos Shorten descriptions Provide detailed/important information (i.e. books: author, publication date, …) Explore moving price closer to photo and product information Explore dropping or changing color of view details buttons

26. Home Page Initial Reaction 14 out of 18 shoppers (78%) had initial positive reactions to this page Half the shoppers thought the Featured Store logos were link-offs

27. Home Page Verbal Feedback Positive Feedback: Like store listing on right Like the colors Broad selection of products Clean, wide variety of products Nice variety of things Looks pretty good Negative Feedback: Animated graphic distracting Lack of focus Not enough access points, few top-level departments displayed Crowded and busy, a lot of things going on A lot of information to comprehend Things too close together

28. Home Page Ratings of Visual Measures As reflected in Shopper Feedback, Organization of the Home Page was lowest scoring measure 3 point scale Q sample: Do you consider this page attractive, unattractive, or neither?

29. Merchant Home Page 8 out of 10 shoppers expected to see more products 6 out of 10 shoppers thought only 3 products were available from the merchant displayed 4 Shoppers thought the strike-through pricing meant the product was unavailable for sale (missing explanation on search results also) 2 of 3 shoppers navigating in the catalog expressed confusion: –Different UI –Looking for narrow by functionality

30. Multi-Merchant Product Page 10 out of 18 shoppers initially thought the product review stars indicated 5 Star Rating (not grayed out = active) 9 out of 18 shoppers could not find the Zip Code / shipping calculator (some clicked View shipping info) No shopper saw the need for pricing at the top and at the merchant level Confusion with red price (at merchant level) - not indicated as Sale

31. On-Line Shopping Concept of on-line shopping as fun: –Unexpected finds –Variety of products Things that would most influence your decision to return to an ecommerce site: –Ease of use –Shipping charges –Product pricing –Order fulfillment –Variety of products –Visual appeal of site