1. Evaluating Surveys March 28, 2011

2. Objectives Evaluate a survey by examining sample size, margin of error, question wording, dates or interviews, and sampling methodology. Evaluate a survey by detecting whether it may be prone to systematic error. Evaluate polls for their moral consequences.

3. Accuracy of Polls Many people have argued that polls are problematic due to perceived inaccuracy. This is potentially an issue in survey research and one of the easiest ways to combat that fact is to pay close attention to the description of the survey. Things to look for: Sample size Margin of error Question wording Dates of interviews Methodology of sampling and weighting

4. Anchorage Daily News 8-9-07 For which of the following, if any, would you be willing to give up your right to vote? The trade would be anonymous. Source: Opinion Research Corporation, margin of error +/- 3%.

5. Anchorage Daily News 8-9-07

6. Sources of Polls One major thing to consider is who sponsored a given poll. While, excluding insta-polls, most major news agencies use respectable standards for surveying other interest, trade groups and random polling firms may not. Controlling the poll plays a major role in how the results will come out. When in doubt try to seek similar results from reputable agencies.

7. Consider Question wording Interest group favoring national health care Do you believe that federal government should provide health insurance to those than cannot afford it? Trade group representing health insurance companies Would you be in favor of government-run health care if it meant limiting your choice of doctor and raising taxes?

8. Other factors that can affect results Timing Social desirability bias Appropriate sample Question wording and choices Question repeating Refusal rate Surveyor oversight

9. Errors Remember the reported margin of error is only one type of error. In practice there are loads of places for error to crop in surveys. Random error Systematic error

10. Morality of Polls Does polling have a negative effect on the political process? Electoral process? Polling on election night, west coast 1980 Bandwagon effects? Reduce turnout? ‘Spiral of Silence’? Noelle-Neumann 1980 pay to poll debate poll Could lead to stigmatization Ginsberg’s weakening of public opinion

11. Polls What is gained by polling? What could be lost or damaged by polling?

12. For Next Time Read WKB chapter 17. Answer question 2 on page 364.