1. Online Market Research © 2001 Ann Schlosser, University of Washington Business School

2. Agenda What accounts for differences in actual and forecasted numbers? –Survey panels vs. server log files –Response and sampling biases Online market research methodologies –VR/Simulated environments –E-mail surveys –ISP’s data warehouse (unobtrusive) –Quasi-experimental design Using the results

3. Who Has a Reliable Estimate of a Site’s Audience? Internal Server Third-Party Ratings Firms For pageviews, ratings-firm numbers were anywhere from 85% less than to 300% more than that reported by the site’s server.

4. Comparing Audience Measurement Firms Using Panel Data COMPANYU.S PANEL SIZE RECRUITMENT METHOD INCENTIVE Media Metrix 55,000Random-digit dialing, follow-up mail $50 annual maximum, giveaways, sweepstakes Nielsen NetRating s 65,000Random-digit dialing$50 U.S. savings bonds every six months PC Data120,000Random-digit dialing, advertising $40 annually, sweepstakes ComScore1.4 million Advertising, partnered with pollsters Provides faster Internet- page download service

5. The Trouble With Online Panels

6. Some Benefits of Panels Level of detail –Geography –Unique visitors (different people accessing the site) –Captures PC activity

7. Internal Server Records are Imperfect Tallies computer (not human) access –Over counts due to: Same user with home and work access Spiders and other bots ISPs that indiscriminately assign users to IP addresses –Under counts due to: Different users of same computer Cached pages –Especially problematic if done by ISPs and other content aggregators

8. Some Factors Influencing the Reliability of Data Response biases Sampling biases Incentives

9. Impact of the Internet on Market Research Kannan et al, 1998

10. Virtual Shopping Market Research

11. Comparing Electronic With Mail Surveys E-mailMail Response rate6%-73%27%-56% Response speed7-8 days7.5-13 days CompletionLessMore Bad address notification10 minutesWeeks Percent of bad addresses24.5%2% Weible & Wallace (1998)

12. Collecting Data Through ISPs while Protecting Privacy ISP login/street address Anonymous ID, Geocode, Append Demog. Foveon Collection (now Plurimus) login Anonymous ID Data Warehouse street address = security key 3rd Party 50 participating ISPs and 3.5 million Web users

13. Raw Material: Collector Lines 23540 12:04:30 com.egghead.www /store2/ent/eggs_ordstat.browse 23540 16:30:57 com.egghead.www /store1/ent/images/space.gif 23540 12:04:34 com.egghead.www /media/bnr/B_EggheadSpecials_08 23540 12:04:42 com.egghead.www /store2/ent/images/space.gif 23540 16:30:57 com.egghead.www /media/images/bdot.gif 23540 16:30:11 com.egghead.www /media/images/but_previous.gif 23540 16:30:11 com.egghead.www /store1/ent/eggs_shop.additem?s 23540 16:30:55 com.egghead.www /store1/ent/eggs_shop.additem?s 23540 16:30:32 com.egghead.www /media/images/portal_egg-header 23540 16:30:57 com.egghead.www /media/images/recalculate.gif 23540 16:30:08 com.egghead.www /store1/ent/eggs_prod.browse?se 23540 16:30:07 com.egghead.www /store/ent/eggs_prod.browse?sec 23540 16:31:03 com.egghead.www /media/images/but_proceed-order 23540 12:04:34 com.egghead.www /media/images/but_nav_auction[3 23540 12:04:34 com.egghead.www /media/images/but_search.gif 25150 12:21:41 com.yahoo.rd /results/a/?http://www.altavist 25150 14:41:43 com.yahoo.rd /results/a/?http://www.altavist 26004 22:07:53 com.yahoo.rd /search/navbar/top/*http://ink. FIDDate/TimeHostURL

14. Classification of Behavior 23540 OCT16:200012:04:30 com.egghead.www2 /store2/ eggs_shop.additem Duration Time of Day Calendar Day www.egghead.com “Internet Computers” NAICS Code “Shopping Cart” Geography (BG ID) Demography Geocoding/Appending Calculation Site Classification URL Classification FIDDate/TimeHostURL

15. Log File Data User Start Date/ End Date/ Packets BytesHost URL ID Time Time To/From To/From

16. GBF vs. GIRF: Reasons for Dot-Com Failures Poor revenue, cost investment and profit models61% No competitive advantage44% No consumer benefit21% Organization, implementation and execution problems17% Channel conflict10% Ineffective fulfillment process7% Source: BCG

17. Various Performance Metrics 65% abandon shopping carts Overall average buyer conversion rate < 4% Low customer conversion, upselling and loyalty, even at best sites –At 10 leading e-commerce sites 20% converted from visitors to buyers 10% upsold 5% returned within 6 months

18. Retention Measures

19. KEY RESEARCH QUESTIONS –Are you reaching the right audience? –Do they remember your website? –Does it enhance the brand’s image? –Does it increase sales? What is the objective(s) of your website?

20. Web Site Report Page Views by Hour Site Navigation Summary Report

21. Disentangling the effect of your website A quasi-experimental design is the best means of isolating the impact of a website from the numerous external factors which may impact attitudes and opinions Online Marketing Traditional Marketing website Peer influencePress influence Pre-existing attitudes

22. Methodology = Website effect Experimental Effect First time users Experimental Group - Repeat visitors Control Group

23. User accesses website www.your.website JavaScript sampling procedure Visitor continues surfing, unaware of the sampling script Not Sampled

24. Survey 1 Sampled Returned to the page originally requested Survey 2 E-mail invitation User accesses website www.your.website JavaScript sampling procedure Visitor continues surfing, unaware of the sampling script Not Sampled

25. ONLINE QUESTIONNAIRES What should you ask? –Questions relating to your website’s objectives –Audience profile questions –Satisfaction questions –Relationship questions –Intent questions –Comparative questions

26. Type of Information Available 7.65 Type of Information Available 7.65 Detail of Information Available 6.98 Detail of Information Available 6.98 The Layout of the Site 7.84 The Layout of the Site 7.84 Navigation Aides on the Site 6.75 Navigation Aides on the Site 6.75 Speed of Loading Pages 7.10 Speed of Loading Pages 7.10 OverallSatisfaction I=.000I=.533I=.39I=.272I=.000 Customer Satisfaction Model Visit Again Recommend to others Recommend to others Spend more time Download Information I=.120 I=.320 I=.420

27. Customer Satisfaction Model CRITICAL IMPROVEMENTSTRENGTH LOWER LEVERAGE AREALOWER PRIORITY Detail of information available Navigation aides on the site Layout of the site Type of information available Speed of loading the pages SATISFACTION WITH FEATURE IMPACT ON OVERALL SATISFACTION

28. Strategic Improvement Matrix