1. Honors Statistics Chapter 12 Part 2
Sample Surveys
Honors Statistics
Chapter 12 Part 2

2. Learning Goals Population versus Sample Sample Surveys
Generalizing Results
Sampling Frame & Sampling Design
Simple Random Sample (SRS)
Convenience Samples
Other Random Sampling Designs
Types of Bias
The Valid Survey

3. Other Random Sampling Designs
Learning Goal #7
Other Random Sampling Designs

4. Learning Goal #7: Other Random Sampling Designs
Stratified Random Sample
Cluster Random Sample
Multistage Sampling
Systematic Sampling

5. Learning Goal #7: Other Random Sampling Designs
It is not always possible to conduct a SRS so it is necessary to have well designed, informative sampling methods that are not a SRS , e.g., sample methods that use randomization.
Simple random sampling is not the only fair way to sample.
More complicated designs may save time or money or help avoid sampling problems.
All statistical sampling designs have in common the idea that chance (randomization), rather than human choice, is used to select the sample.

6. Learning Goal #7: Stratified Random Sample
The population is first sliced into homogeneous groups, called strata, before the sample is selected.
These strata are made up of individuals similar in some way that may affect the response variable.
Then simple random sampling is used within each stratum before the results are combined.
This common sampling design is called stratified random sampling.

7. Learning Goal #7: Stratified Random Sample
With this procedure we can acquire information about
the whole population
each stratum
the relationships among strata.
Examples of strata
Sex
Male
Female
Age
under 20
20-30
31-40
41-50
Occupation
professional
clerical
blue-collar

8. Learning Goal #7: Stratified Random Sample
Steps
Divide the population into separate homogeneous groups, called strata.
Select a simple random sample from each strata.
Combine the samples from all strata to form complete sample.

9. Learning Goal #7: Stratified Random Sampling Procedure

10. Learning Goal #7: Stratified Random Sample

11. Learning Objective 1: Stratified Random Sample - Example
Suppose a university has the following student demographics:
Undergraduate Graduate First Professional Special
55% % % %
In order to insure proper coverage of each demographic, a stratified random sample of 100 students could be chosen as follows: select a SRS of 55 undergraduates, a SRS of 20 graduates, a SRS of 5 first professional students, and a SRS of 20 special students; combine these 100 students.

12. Learning Goal #7: Stratified Random Sample
The most important benefit is Stratifying can reduce the variability of our results.
When we restrict by strata, additional samples are more like one another, so statistics calculated for the sampled values will vary less from one sample to another.
Stratified random sampling can reduce bias.
Stratified sampling can also help us notice important differences among groups.

13. Learning Goal #7: Stratified Random Sample
Advantage is that you can include in your sample enough subjects in each stratum you want to evaluate.
Disadvantage is that you must have a sampling frame and know the stratum into which each subject belongs.

14. Learning Goal #7: Cluster Random Sample
Sometimes stratifying isn’t practical and simple random sampling is difficult.
Splitting the population into similar parts or clusters can make sampling more practical.
Then we could select one or a few clusters at random and perform a census within each of them.
This sampling design is called cluster random sampling.
If each cluster fairly represents the full population, cluster sampling will give us an unbiased sample.

15. Learning Goal #7: Cluster Random Sample
Steps
Divide the population into a large number of clusters, such as city blocks
Select a simple random sample of the clusters
Use the subjects in those clusters as the sample

16. Learning Goal #7: Cluster Random Sample
Summary – Cluster Sampling
Divide the population into heterogeneous groups called clusters.
Take an SRS of some of the clusters.
Every member of the cluster is included in the sample.
Usually used to reduce the cost of obtaining a sample.
Extensively used by government agencies and certain private research organizations.

17. Learning Goal #7: Cluster Random Sample
Example: In conducting a survey of school children in a large city, we could first randomly select 5 schools and then include all the children from each selected school.
Although cluster sampling can save time and money, it does have disadvantages.
Ideally, each cluster should mirror the entire population. However, that is often not the case, as members of a cluster are frequently more homogeneous than the members of the population as a whole.

18. Learning Goal #7: Cluster Random Sample - Procedure

19. Learning Goal #7: Cluster Random Sample

20. Learning Goal #7: Cluster Random Sample
Cluster sampling is not the same as stratified sampling.
We stratify to ensure that our sample represents different groups in the population, and sample randomly within each stratum.
Strata are internally homogeneous, but differ from one another.
Clusters are more or less alike, are internally heterogeneous and each resembling the overall population.
We select clusters to make sampling more practical or affordable.

21. Learning Goal #7: Cluster Random Sample
Preferable when
A reliable sampling frame is unavailable.
The cost of selecting a SRS is excessive.
Disadvantage
Usually need a larger sample size than with a SRS in order to achieve a similar results.

22. Learning Goal #7: Comparing Random Sampling Methods

23. Learning Goal #7: Multistage Sample
Sometimes we use a variety of sampling methods together.
Sampling schemes that combine several methods are called multistage samples.
Most surveys conducted by professional polling organizations use some combination of stratified and cluster sampling as well as simple random sampling.

24. Learning Goal #7: Multistage Sample
Example: A national polling service may stratify the country by geographical regions, select a random sample of cities from each region, and then interview a cluster of residents in each city.
Multistage samples may also use multiple stages of stratification. They are often used by the government to obtain information about the U.S. population.
Example: Sampling both urban and rural areas, people in different ethnic and income groups within the urban and rural areas, and then individuals of different ethnicities within those strata. Data are obtained by taking an SRS for each substrata.

25. Learning Goal #7: Systematic Sample
Sometimes we draw a sample by selecting individuals systematically.
For example, you might survey every 10th person on an alphabetical list of students.
To make it random, you must still start the systematic selection from a randomly selected individual.
When there is no reason to believe that the order of the list could be associated in any way with the responses sought, a systematic sample can give a representative sample.

26. Learning Goal #7: Systematic Sample
Method of sampling in which the sample is selected in some predetermined way.
For example, we may obtain a list of our population of interest and from that list choose every fifth individual to be part of the sample.
Although each individual has an equal chance of being chosen, this method is not a SRS because each possible sample of size n individuals does not have an equal chance of being chosen.

27. Learning Goal #7: Systematic Sample
Example: If we are choosing a sample of 30 students from the 300 students in the senior class by selecting every 10th student from the alphabetical directory, the first 30 students on the list will never all be chosen as the sample group.
Easier to execute than SRS.
Usually provides results comparable to SRS.

28. Learning Goal #7: Systematic Sample - Procedure

29. Learning Goal #7: Systematic Sample

30. Learning Goal #7: Systematic Sample
Systematic sampling can be much less expensive than true random sampling.
When you use a systematic sample, you need to justify the assumption that the systematic method is not associated with any of the measured variables.

31. Learning Goal #8
Types of Bias

32. Learning Goal #8: Types of Bias
Undercoverage Bias
Voluntary Response Bias
Nonresponse Bias
Response Bias

33. Learning Goal #8: Bias
Bias: Tendency to systematically favor certain parts of the population over others. Any systematic failure of a sample to represent its population.
Sampling methods that, by their nature, tend to over- or under- emphasize some characteristics of the population are said to be biased.
Bias is the bane of sampling—the one thing above all to avoid.
There is usually no way to fix a biased sample and no way to salvage useful information from it.
The best way to avoid bias is to select individuals for the sample at random.
The value of deliberately introducing randomness is one of the great insights of Statistics.
A Large Sample Does Not Guarantee An Unbiased Sample!

34. Learning Goal #8: Bias

35. Learning Goal #8: Undercoverage
A sampling scheme that fails to sample part of the population or that gives a part of the population less representation than it has in the population suffers from undercoverage.
A classic example of undercoverage is the Literary Digest voter survey, which predicted that Alfred Landon would beat Franklin Roosevelt in the 1936 presidential election. The survey sample suffered from undercoverage of low-income voters, who tended to be Democrats. Undercoverage is often a problem with convenience samples.

36. Learning Goal #8: Undercoverage Example: Literary Digest Poll
1936 presidential election Literary Digest magazine poll.
The survey team asked a sample of the voting population whether they would vote for Franklin D. Roosevelt, the democratic candidate or Alfred Landon, the republican candidate.
Based on the results, the magazine predicted an easy win for Landon.

37. Learning Goal #8: Undercoverage Example – Result
When the actual results were in, Roosevelt won by a landslide.
What happened?
The sample was obtained from among people who owned a car or had a telephone. In 1936, that group included mostly rich people and they historically voted republican.
The response rate was low, less than 25% of those polled responded. A disproportionate number of those responding were Landon supporters.
Whatever the reason for the poll’s failure, the sample was not representative of the population.

38. Learning Goal #8: Undercoverage
Undercoverage occurs when parts of the population are left out in the process of choosing the sample.
Because the U.S. Census goes “house to house,” homeless people are not represented. Illegal immigrants also avoid being counted. Geographical districts with a lot of undercoverage tend to be poor ones. Representatives from richer areas typically strongly oppose statistical adjustment of the census.
Historically, clinical trials have avoided including women in their studies because of their periods and the chance of pregnancy. This means that medical treatments were not appropriately tested for women. This problem is slowly being recognized and addressed.

39. Learning Goal #8: Voluntary Response Bias
When choice rather than randomization is used to obtain a sample, the sample suffers from voluntary response bias.
Voluntary response bias occurs when sample members are self-selected volunteers (voluntary response sample).
An example would be call-in radio shows that solicit audience participation in surveys on controversial topics (abortion, affirmative action, gun control, etc.).
The resulting sample tends to over represent individuals who have strong opinions, either for or against the topic.

40. Learning Goal #8: Nonresponse Bias
Occurs in a sample design when individuals selected for the sample fail to respond, cannot be contacted, or decline to participate.
No survey succeeds in getting responses from everyone. The problem is that those who don’t respond may differ from those who do.
A common problem with mail surveys. Response rate is often low (5% - 30%), making mail surveys vulnerable to nonresponse bias.

41. Learning Goal #8: Response Bias
Anything in a survey that influences responses falls under the heading of response bias.
Examples are biased wording of survey questions, lack of privacy while being surveyed, and appearance of the interviewer.
Both Question Bias and Interviewer Bias are examples of response bias.

42. Learning Goal #8: Response Bias - Question Bias
Wording of the questions or the questions themselves lead to bias.
People often don’t want to be perceived as having unpopular or unsavory views and so may not respond truthfully.
Example: Given that the threat of nuclear war is higher now than it has ever been in human history, and the fact that a nuclear war poses a threat to the very existence of the human race, would you favor an all-out nuclear test ban?
Question is biased in favor of a nuclear test ban.

43. Learning Goal #8: Response Bias - Question Bias

44. Learning Goal #8: Response Bias - Interviewer Bias
The sex, age, race, dress, attitude, or actions of the interviewer and how the interviewer asks the questions have an influence on the way a subject responds.
Example: A male interviewer asking sex related questions to women.
To prevent this, interviewers must be trained to remain neutral throughout the interview. They must also pay close attention to the way they ask each question. If an interviewer changes the way a question is worded, it may impact the respondent's answer.

45. Learning Goal #9
The Valid Survey

46. Learning Goal #9: Key Parts of a Sample Survey
Identify the population of all subjects of interest
Construct a sampling frame which attempts to list all subjects in the population
Use a random sampling design to select n subjects from the sampling frame
Be cautious of sampling bias due to nonrandom samples
We can make inferences about the population of interest when sample surveys that use random sampling are employed.

47. Learning Goal #9: The Valid Survey
It isn’t sufficient to just draw a sample and start asking questions. A valid survey yields the information we are seeking about the population we are interested in. Before you set out to survey, ask yourself:
What do I want to know?
Am I asking the right respondents?
Am I asking the right questions?
What would I do with the answers if I had them; would they address the things I want to know?

48. Learning Goal #9: The Valid Survey
These questions may sound obvious, but there are a number of pitfalls to avoid.
Know what you want to know.
Understand what you hope to learn and from whom you hope to learn it.
Use the right frame.
Be sure you have a suitable sampling frame.
Tune your instrument.
The survey instrument itself can be the source of errors - too long yields less responses.

49. Learning Goal #9: The Valid Survey
Ask specific rather than general questions.
Ask for quantitative results when possible.
Be careful in phrasing questions.
A respondent may not understand the question or may understand the question differently than the way the researcher intended it.
Even subtle differences in phrasing can make a difference.

50. Learning Goal #9: The Valid Survey
Be careful in phrasing answers.
It’s often a better idea to offer choices rather than inviting a free response.
The best way to protect a survey from unanticipated measurement errors is to perform a pilot survey.
A pilot is a trial run of a survey you eventually plan to give to a larger group.

51. Learning Goal #9: Valid Survey – How to Think About Bias
Look for bias in any survey you encounter before you collect the data—there’s no way to recover from a biased sample of a survey that asks biased questions.
Spend your time and resources reducing bias.
If you possibly can, pilot-test your survey.
Always report your sampling methods in detail.