Presentation on theme: "Unparalleled Insight into Consumer Behavior The Professional Respondent Problem In Online Survey Panels Today Gian M. Fulgoni MRA Annual Conference June,"— Presentation transcript:
Unparalleled Insight into Consumer Behavior The Professional Respondent Problem In Online Survey Panels Today Gian M. Fulgoni MRA Annual Conference June, 2005
comScore Networks, Inc. Founded in 1999 Corporate headquarters: Reston, VA Offices in Chicago, NYC, San Francisco, Seattle, Toronto and London 300 full-time employees 80 employed in sales/servicing survey-based research Experienced senior leadership team with an outstanding record of innovation in the market research industry Created the only continuous survey panel that also passively captures all web-wide behavioral and transaction data for a sample of 3.5 million people Global access to survey 10 Million people, with 5 Million resident in the US. More than 500 clients
Recognized For Our Accomplishments Americas Fastest Growing Private Companies comScore ranked 97 th among the nations fastest growing privately held firms. November 2004 Top 100 Innovative Companies Named one of the 100 companies that will shape the next year in technology. December 2004 Fifth Fastest Growing Research Firm Currently ranked the 30 th largest U.S. research firm – up ten spots from previous year. June 2004 Worlds Largest Database Grand prize winner for developing the largest NT decision support warehouse. December 2003 Honomichl Top 50 Marketing News Rising Star Named one of only three Rising Stars among Virginia-based technology companies. October 2004 High Tech Award Recognized as a technology leader for successfully developing innovative products. November 2004
Online Surveys: Many Positives … But Also Some Emerging Concerns Custom survey research is increasingly moving online because of the benefits of faster turnaround and lower cost This is creating an apparently insatiable demand for online samples But, problems of over-fishing and declining response rates are raising concerns about respondent quality
A Possible Cause of the Problem: Multiple Suppliers Using the Same Recruiting Site
Other Suppliers Are Using the Same Recruiting Sites Favored By their Competitors
The comScore Panel comScore has recruited an opt-in consumer panel that has been validated to be representative of the online population and projectable to the total U.S. population. The comScore panel consists of millions of people who have given comScore explicit permission to passively measure all of their online activities: –Validated in comparison to Dept. of Commerce sales data –Validated in comparison to individual site server log data –Validated in comparison to offline surveys Panelists are recruited across thousands of sites not used by other panel suppliers and they do not have to be willing to answer surveys to be accepted to the panel
How The comScore Panel Benefits You A higher quality panel that delivers more value to your end clients: comScore is the ONLY panel that can: 1. Screen out professional respondents. 2.Truly validate the identity of the respondent. 3.Contextually pop surveys anywhere on the internet based on actual behavior. 4.Append surveys with online behavioral data. 5.Append surveys with other profile and offline information. This enables your company to: 1. Deliver better quality, accuracy, validation, and insights. 2. Have the assurance and confidence that your results and advice are based on sound and projectable information.
comScores Customer Knowledge Platform: A 360° View of Millions of People: Observed Behavior and Survey Measurement Designed to be projectable to the total U.S. population Ernst & Young certified for information privacy & security DEMOGRAPHICS Self-reported Appended (e.g. Axiom) Individual & Household Level Technology Ownership (e.g. PDA, cell) WEB VISITING & VIEWING Web Site/Page Click Streams HTML & Proprietary Content Search Engine Queries Product & Service Research ATTITUDES General Behavior-specific Awareness & Intent New Product Concepts, etc. ONLINE TRANSACTIONS Purchases/Subscriptions Applications/Configurations Value/Price Paid Shipping & Handling Promotion Incentives OFFLINE PURCHASING Third Party Sources: Cars, Houses, etc Purchase Surveys Supermarket Shopping Data Credit Card Purchases MARKETING STIMULI Banner Ads Referrals
Measurement Capabilities Of the comScore Panel comScores technology captures all panelists online activity, including the details of all surveys taken. All demographics (including name and address) are verified Using the comScore panel, its possible to measure the number of surveys taken by the members of any survey panel, encompassing surveys from all panels to which they belong.
The Professional Survey Respondent: Alive and Well Online comScore research shows that: –Only 0.25% of all Internet households account for 30% of all surveys taken –These heavy survey responders each took an average of 80 surveys per quarter in 2004 – thats about 1 a day! –Some, but not all, of the leading panels contain too many of these professional respondents –On average, a member of each of the leading panels belongs to at least 8 other panels
Some, But Not All, Leading Panels Are Biased Towards Heavy Survey Respondents (Data Shows % of Each Panel Completing Specified Number of Surveys From All Suppliers in a Three Month Period) A Leading panelOther Leading Panel
The Professional Survey Respondent: Some important skews to consider when doing CPG research comScore research shows that while heavy survey respondents are normal in many ways, there are important differences that can create problems in certain surveys. Compared to the average person, professional survey respondents… –Are significantly less likely to be employed full time and are less well educated –Are significantly more likely to be influenced by advertising –Shop much more frequently at grocery stores but are less likely to be warehouse/club store shoppers –Are not as likely to consider themselves to be impulse shoppers –Are significantly more likely to use coupons when they shop –Are significantly more likely to think that buying food and/or household products with a brand name that you can trust is important –Underestimate how many survey panels they belong to, as well as grossly underestimate how many surveys they actually complete in an average month
But, professional respondents are also similar to the general population in some ways Spend similar amounts of time grocery shopping Spend similar amounts of money on groceries As likely to use a list for groceries As likely to have a budget for groceries and stick to it Were as likely to vote in 2004 election – and who they voted for was similar Are similar in age Similar household size and composition Similar residence profiles As likely to think of themselves as: –Politically active –Spiritual –Creative –Organized –Comfortable w/change –Action-oriented –Positive/hopeful about US Economy –Concerned about environment Participate in many activities w/similar frequency: –Exercise –Go to movies –Read a book –Eat out at a restaurant –Go to religious services –Go out with friends
The Screen-out Challenge comScore research shows that professional respondents: –Underestimate how many survey panels they belong to –Grossly underestimate how many surveys they actually complete in an average month (or they dont want you to know!) This makes it very difficult to screen out professional respondents simply by asking them how many surveys theyve taken.
Some Ways to Minimize the Inclusion of Professional Survey Respondents in an Online Survey Panel Avoid recruiting panelists at sites favored by other panel suppliers Dont recruit only those panelists who want to participate in continuous survey programs: –Use large numbers of occasional survey responders also Limit the aggregate financial incentive you offer panelists by limiting the number of surveys you invite them to complete: –And be sure to take into consideration that your panelists are likely to also be compensated by their membership in other survey panels
Advantages of comScores Survey Panel Unsurpassed accuracy, quality and panel hygiene -ARF-endorsed sampling methodology -Random online recruiting integrated with RDD calibration sample avoids the bias of recruiting at only a few large sites -Patent-pending electronic monitoring of online behavior allows confirmation of respondent demographics -comScores ability to measure households participation in all survey programs permits the identification and exclusion (as required) of professional respondents Contextual pop survey based on actual behavior -Increases accuracy of screening by reducing reliance on respondent memory and minimizing recall errors -Permits real time surveying of consumer behavior - Address the latency issue
Gian M. Fulgoni Chairman comScore Networks, Inc