1. Chapter 12 Sample Surveys
math2200
How to survey in a good way.

2. How to study a population
A population is the entire group of individuals we want information about.
Impractical to examine the entire population
A sample is a smaller group of the population we actually examine in order to gather information.
A population is the entire group of individuals we want information about.
Impractical to examine the entire population
A sample is a smaller group of the population we actually examine in order to gather information.

3. What is a survey?
A survey is the process of collecting data from a sample in an attempt to draw conclusions about the entire population.
The conclusions can only be accurate if the selected sample properly represents the population.
Analogy: Soup tasting.
What is a survey?
A survey is the process of collecting data from a sample in an attempt to draw conclusions about the entire population.
The conclusions can only be accurate if the selected sample properly represents the population.
Analogy: Soup tasting.
Analogy: you cooked a pot of soup, Instead of drinking it up, you might just have spoonful or two to see what it tastes like. This is like what a survey is.

4. Opinion polls Example of Sample surveys.
Designed to ask questions of a small group of people in the hope of learning something about the entire population.
Opinion polls is an Example of Sample surveys.
they are Designed to ask questions of a small group of people in the hope of learning something about the entire population.
Professional pollsters work quite hard to ensure that the sample they take is representative of the population.
If not, the sample can give misleading information about the population. Therefore give us biased result.

5. Bias
Sampling methods that tend to over- or under- emphasize some characteristics of the population are said to be biased.
AVOID BIAS!!!
Sampling methods that tend to over- or under- emphasize some characteristics of the population are said to be biased
Bias is the one thing above all to avoid.
There is usually no way to fix a biased sample and no way to salvage useful information from it.

6. Landon vs. Roosevelt in 1936
The poll results pointed to a Landon victory over Roosevelt, indicating that he would get some 57% of the vote.
Actually, Roosevelt polled 62.5% of the major-party vote and won 523 out of possible 531 electoral votes.
Reason: Biased Sampling.
There is an example of an opinion poll that ended up being terribly biased. therefore led to the wrong conclusion.
The 1936 election was Landon vs. Roosevelt. Landon was governor of Kansas then. You probably never heard of him and that’s because he lost.’
Literary Digest which have been successfully running surveys to predict the outcomes of presidential elections did a poll in And predicted Landon would win by a wide margin.
But the opposite occurred. Roosevelt won by a lot. How could Literary Digest skewed up so badly? Biased sampling, that’s how. They got the names from phone book. But that was during the Great Depression, only rich people have phones. So most people in their sample were rich. Does it sound like a good representative of American population at that time? The answer is no. And it turned out that Roosevelt’s supporters tended to be less well off.
Literary Digest went bankrupt shortly after. It might be a coincidence.

7. Randomization
To avoid biased samples, we should select individuals for the sample at random!
In 1936, using a different sample of 50,000, Gallup predicted that Roosevelt would get 56% of the vote to Landon’s 44%.
This guy George Gallup on the other hand used a totally different sample and successfully predicted the outcome.
Gallup Organization went on to become one of the leading polling companies.
How did he do it?
Think about the soup analogy, suppose you think there is not enough salt. So you add some salt, if you don’t stir when you take another bite it might seem too salty or it might taste like there is no difference at all. You have to stir i.e. randomize it.

8. The Benefit of Randomization
Protects against both known and unknown factors.
Makes it possible to draw inference about the population based on the sample.
For example, in the poll example we saw just now, suppose you know the distribution of the income levels of the entire population, you could intentionally choose the sample such that the income distribution of the sample matches that of the entire population, that’s great. But what about other unknown factors, like race, or political affiliation. When you try to match one, you might miss another. There is a million factors could effect people’s opinion and bias your survey, but randomization matches all of them automatically.
And such inferences are among the most powerful things we can do with Statistics.

9. Sample Size
The sample size but not the fraction of the population matters.
Exception: If the population is small enough and the sample is more than 10% of the whole population, the population size can matter.
A larger sample size leads a more precise result in general.
What else goes into a successful survey? Sample size is important as well
The sample size but not the fraction of the population matters.
For example , the sample size of 50,000 in the presidential election survey in 1936 was sufficient. That sample size still works today after many years of population growth in the US.
Exception: If the population is small enough and the sample is more than 10% of the whole population, the population size can matter. But this is not the interesting case. If the population is small ,you can just go and ask everyone.
A larger sample size leads a more precise result in general.
But once a sample reaches a certain size, making it larger would really improve the accuracy that much

10. Does a Census Make Sense?
A sample include everyone in the entire population is called a census.
There are problems with taking a census:
It is hard to track down every individual in the population.
Population changes.
Taking a census is way more complicated than sampling.
A sample include everyone in the entire population is called a census.
Why bother determining the right sample size?
Well lots of reasons.
Census requires team work. And sometimes people are counted twice due to all kinds of errors.

11. Populations and Parameters
A parameter that is part of a model for a population is called a population parameter.
The statistics that estimate population parameters are called sample statistics.
If we can’t look at all the individuals, what should we do. We have to use math to analyze the data from our survey. The things we want to estimate are called population parameters. The estimations themselves, the things we calculate, are call sample statistics. If the sample statistics, like sample mean , sample sd, are close to the population parameters, then we win.

12. Notation
Greek letters to denote parameters and Latin letters to denote statistics. Hat on the estimation.
We typically use Greek letters to denote parameters and Latin letters to denote statistics.
Parameter
Estimate

13. Simple random samples
If the statistics we compute from a sample reflect the population parameter accurately , the sample is called representative.
If every possible sample of the size we plan to draw has an equal chance to be selected, a sample drawn in this way is called a Simple Random Sample.
The sampling frame is a list of individuals from which the sample is drawn.
Different samples lead to different sample statistics . We call these sample-to-sample differences sampling variability.
Now we introduce a few sample methods to get random samples .
If the statistics we compute from a sample reflect the population parameter accurately , the sample is called representative.
If every possible sample of the size we plan to draw has an equal chance to be selected, a sample drawn in this way is called a Simple Random Sample.
The sampling frame is a list of individuals from which the sample is drawn.
Samples drawn at random generally differ from one another.
Different samples lead to different sample statistics . We call these sample-to-sample differences sampling variability.

14. Simple Random Sample: Example
Select 5 students from 19 enrolled in our class
Sampling frame: 19 enrolled students
Sample: 5 students
Sample size: 5
Number the students from 1 to
Use your TI-83 to obtain five random integers between 1 and 80 (no repetition)
Here is an example of random sampling.
Select 5 students from 19 enrolled in our class
Sampling frame: 19 enrolled students
Sample: 5 students
Sample size: 5
One way to get a simple random sample is
Number the students from 1 to 19
Use your TI-83 to obtain five random integers between 1 and 19 (no repetition)
Any question about this example?

15. Stratified Sampling Stratified Sampling
Simple random sampling is applied within each stratum within the population before results are combined.
Stratified random sampling can reduce bias.
Stratifying can also reduce the variability of our results.
Stratified Sampling
Dive the population into homogeneous groups . And each group is called strata. Then Simple random sampling is applied within each stratum and finally combine all the result from different strata together.
Stratified random sampling can reduce bias.
Stratifying can also reduce the variability of our results.

16. Example
Survey how students feel about funding for the football team. population: 60% men and 40% women. Sample size = 100
Simple random sampling : 20 men 80 women ? 80 men 20 women?
Stratified random sampling: 60 men and 40 women. And simple random sampling within men or women.
Survey how students feel about funding for the football team. population: 60% men and 40% women. Sample size = 100
Woman and men might have different opinions of funding.
Simple random sampling : 20 men 80 women ? 80 men 20 women? The result will be skewed towards men or women.
Stratified random sampling: 60 men and 40 women
Men is a stratum and women is the other.
It maintains the gender balance and hence makes such samples more accurate in representing population opinion

17. Cluster Sampling
Split the population into similar parts or clusters, then select one or a few clusters at random and perform a census within each of them.
When stratifying isn’t practical and simple random sampling is difficult, Split the population into similar parts or clusters, then select one or a few clusters at random and perform a census within each of them.
If each cluster fairly represents the full population, cluster sampling will give us an unbiased sample.
For example, go get an idea of about the information of residence in Missouri, we can divide Missouri into counties and pick a few of them and carry census in those counties.
Cluster sampling is not the same as stratified sampling.
We stratify to ensure that our sample represents different groups in the population, and sample randomly within each stratum.
Strata are homogeneous, but differ from one another.
Clusters are more or less alike, each resembling the overall population.
We select clusters to make sampling more practical or affordable.

18. Multistage Sampling
Sampling schemes that combine several methods are called multistage samples.
Most surveys conducted by professional polling organizations use some combination of stratified and cluster sampling as well as simple random sampling.
Sometimes we use a variety of sampling methods together.
Sampling schemes that combine several methods are called multistage samples.
Most surveys conducted by professional polling organizations use some combination of stratified and cluster sampling as well as simple random sampling.

19. Example To assess the reading level of a book based on the words used
Randomly select one chapter from one part (stratified sampling)
Randomly select several pages from each of those chosen chapters (cluster sampling)
Randomly select a few sentences from each of those chosen pages (simple random sampling)
Each part is a stratum
Each page is a cluster
Sampling frame is the list of all the sentences in those chosen pages.

20. Systematic sampling Example
survey every 10th person on an alphabetical list of students
Start from a randomly selected individual

21. Who’s Who? The population of interest? The sampling frame?
The target sample, for example, a sample determined by simple random sampling?
The actual respondents?
Often, this may not be well defined.
For example, who is exactly a student?
first
Even if the population is clear, it may not be a practical group to study.
For example, those who will vote in the next election.
Second
Usually, the sampling frame is not the group you really want to know about.
The sampling frame limits what your survey can find out.
Third
These are the individuals for whom you intend to measure responses.
You’re not likely to get responses from all of them—no response is a problem in many surveys.
Four
These are the individuals about whom you do get data and can draw conclusions.
Unfortunately, they might not be representative of the sample, the sampling frame, or the population.

22. The Valid Survey BEFORE setting out the survey:
What do I want to know?
Am I asking the right respondents?
Am I asking the right questions?
What would I do with the answers if I had them; would they address the things I want to know?
It isn’t sufficient to just draw a sample and start asking questions. Before you set out to survey, ask yourself:

23. The Valid Survey (cont.)
Know what you want to know.
Use the right frame.
Time your instrument.
Ask specific rather than general questions.
Ask for quantitative results when possible.
Be careful in phrasing questions.
Even subtle differences in phrasing can make a difference.
Give a pilot survey to smaller
Know what you want to know.
Understand what you hope to learn and from whom you hope to learn it.
Use the right frame.
Be sure you have a suitable sampling frame.
Time your instrument.
The survey instrument itself can be the source of errors.
Longer questionnaires yield fewer responses
Ask specific rather than general questions.
How much did you sleep last night?
How much do you usually sleep?
Ask for quantitative results when possible.
How many hours did you sleep last night?
Be careful in phrasing questions.
Does anyone in your family belong to a union? (What about my grandfather?)
Do you approve of the recent actions of the Secretary of Labor? (I have no idea.)
Did you drink too much last night? (I do not want to answer this question.)
After 9/11, President Bush authorized government wiretaps on some phone calls in the U.S. without getting court warrants, saying this was necessary to reduce the threat of terrorism. Do you approve or disapprove of this?
After 9/11, President Bush authorized government wiretaps on some phone calls in the U.S. without getting court warrants. Do you approve or disapprove of this?
Give a pilot survey to smaller
group.is a trial run of a survey you eventually plan to give to a larger group.

24. What Can Go Wrong? Work hard to avoid influencing responses
Sample Badly with Volunteers
Sample Badly, but Conveniently
Sample from a Bad Sampling Frame
Watch out for nonrespondents
Work hard to avoid influencing responses
Sample Badly with Volunteers:
In a voluntary response sample, a large group of individuals is invited to respond, and all who do respond are counted.
Voluntary response samples are almost always biased, and so conclusions drawn from them are almost always wrong.
Voluntary response bias: Voluntary response samples are often biased toward those with strong opinions or those who are strongly motivated.
People with negative opinions tend to respond more often than those with equally strong positive opinions

25. How to Think About Biases
Look for biases in any survey you encounter—there’s no way to recover from a biased sample of a survey that asks biased questions.
Spend your time and resources reducing biases.
If you possibly can, pretest your survey.
Always report your sampling methods in detail.

26. What have we learned?
A representative sample can offer us important insights about populations.
There are several ways to draw samples.
Bias can destroy our ability to gain insights from our sample
Bias can also arise from poor sampling methods
Look for biases in the survey and be sure to report our methods how the survey was performed.