Internet Survey Methods

Sources of Error in Internet Surveys Coverage Error Mismatch between frame and target populations Web users not representative of entire population High likelihood that those outside of frame differ from sample on variables of interest Sampling Error Inability to construct representative frame Every member does not have a known non- zero chance of being sampled Even if generalizing to web users, no list exists from which to draw sample

Sources of Error in Internet Surveys Non-Response Error Inability to use common motivating tools similar to increase response rates for less responsive populations (e.g. signatures, incentives) Technical difficulties filtering Measurement Error Differences in survey instrument due to different web browsers & configurations Panel conditioning

Combating Error in Internet Surveys Harris Interactive Probability and Quota- based sampling within panel Demographic information is known; can apply weights Limitations Based on volunteers Still limited to internet users Possibility for variance in instrument

Combating Error in Internet Surveys Knowledge Networks Initial sample based on probability sample (RDD) Provides access to web Standardized web interface Limitations Limited coverage of WebTV (excludes appx. 17% of population) Attrition Possible panel effects

Combating Error in Internet Surveys Polimetrix Uses Sample matching Random sample of target population, who is then matched demographically on race, age, and gender and geographically to individuals within their opt-in panel of respondents Limitations Limited to web users Limited to volunteers Potential for variance in instrument Relies on demographic profiles provided by external databases

Benefits of Internet Surveys Ability to use multimedia More accurate for sensitive data Speed/ Turnaround time Better ability to implement skips/ tailoring of questions asked

Additional References Clinton (2001). Panel bias from attrition and conditioning Couper (2000). Web surveys: A review of issues and approaches. The Public Opinion Quarterly 64(4): Dennis (2001). Are internet panels creating professional respondents? Marketing Research Summer 2001: Huggins (2001). Probability based internet surveys: A synopsis of early methods and survey research results. Paper presented at Research Conference of the Federal Committee on Statistical Methodology. Krosnick & Chang (2001). A comparison of the random digit dialing telephone survey methodology with internet survey methodology as implemented by Knowledge Networks and Harris Interactive. Polimetrix. Sample matching. Accessed on 2/22/07. Available at atching.pdf atching.pdf