# Part IV – When the MOE Doesn’t Apply

## Presentation on theme: "Part IV – When the MOE Doesn’t Apply"— Presentation transcript:

Part IV – When the MOE Doesn’t Apply
Statistical Sampling 1

This video is designed to accompany pages 41-76 in
Making Sense of Uncertainty Activities for Teaching Statistical Reasoning Van-Griner Publishing Company 2

Non-Sampling Errors The Margin of Error is a nice mathematical way of addressing sampling variability, also called “random sampling error.” There are lots of other “errors” that can affect data collection and the MOE simply doesn’t apply. 3

Working Definition Non-Sampling Error – An error or discrepancy caused by something other than the fact that a sample was selected instead of the entire population. These errors include, but are not limited to, data entry errors, nonresponse, biased questions in a questionnaire, question order, and false information provided by respondents. 4

Muddled Questions “Do you plan to leave your job and look for another one during the coming year?” “Have you often, sometimes, hardly ever, or never felt bad because you were unfaithful to your wife?” (Harris Poll: One percent said often, 14 percent said sometimes or hardly ever, and 85 percent said they never felt bad because of this.) “Should we increase taxes in order to get more housing and better schools, or should we keep them about the same?” 5

Percentage Nonresponse

How bad is it? No hard data to compare sampling errors to non-sampling errors, but it is generally thought that non-sampling errors might be the larger of the two. Harris Poll Disclaimer: All sample surveys and polls, whether or not they use probability sampling, are subject to multiple sources of error which are most often not possible to quantify or estimate, including sampling error, coverage error, error associated with nonresponse, error associated with question wording and response options, and post-survey weighting and adjustments. Therefore, Harris Interactive avoids the words “margin of error” as they are misleading. 7

What to do? Large body of literature on things to try and things that seem to work Use of inducements for non-responders Use of technology-assisted confidential interview techniques Awareness of psychology of question order Awareness of experiences with question wording Use of internal consistency checks 8

One-Sentence Reflection
Non-sampling errors can substantially detract from the accuracy of survey data, and are not addressed by the margin of error.