1. What Is a Test of Significance?
Basic Practice of Statistics - 3rd Edition
CHAPTER 22
What Is a Test of Significance?
STATISTICS
CONCEPTS AND CONTROVERSIES
Eighth Edition
David S. Moore, William I. Notz
Lecture Presentation
Chapter 5

2. Chapter 22 Concepts The Reasoning of Tests of Significance
Hypotheses and P-values
Statistical Significance
Calculating P-values *
Tests for a Population Mean *

3. The Reasoning of Tests of Significance
Statistical inference draws conclusions about a population on the basis of data about a sample.
Statistical tests ask if sample data give good evidence against a claim. A statistical test says, “If we took many samples and the claim were true, we would rarely get a result like this.”
To get a numerical measure of how strong the sample evidence is, we replace the vague term “rarely” by a probability.
Population
Collect data from a representative Sample...
Sample
Make an Inference about the Population.

4. The Reasoning of Tests of Significance

5. The Reasoning of Tests of Significance
72% is stronger evidence against the skeptic’s claim than 56%. But how much stronger? Is even 72% in favor in a sample convincing evidence that a majority of the population prefer fresh coffee?
The claim. The skeptic claims that coffee drinkers can’t tell fresh from instant, so that only half will choose fresh-brewed coffee. That is, he claims that the population proportion p is only 0.5. Suppose for the sake of argument that this claim is true.
The sampling distribution. If the claim p = 0.5 were true and we tested many random samples of 50 coffee drinkers, the sample proportion would vary from sample to sample according to (approximately) the Normal distribution with

6. The Reasoning of Tests of Significance
The data. Place the sample proportion on the sampling distribution. You see in Figure 22.1 that 0.56 isn’t an unusual value, but that 0.72 is unusual. We would rarely get 72% of a sample of 50 coffee drinkers preferring fresh-brewed coffee if only 50% of all coffee drinkers felt that way. So the sample data do give evidence against the claim.

7. The Reasoning of Tests of Significance
The probability. We can measure the strength of the evidence against the claim by a probability. If p-hat = 0.56, this probability is the shaded area under the Normal curve in Figure This area is Our sample actually gave p-hat = The probability of getting a sample outcome this large is only
An outcome that would occur just by chance in 20% of all samples is not strong evidence against the claim. But an outcome that would happen only 1 in 1000 times is good evidence.

8. The Reasoning of Tests of Significance
There are two possible explanations of the fact that 72% of our subjects prefer fresh to instant coffee:
The skeptic is correct (p = 0.5), and by bad luck a very unlikely outcome occurred.
In fact, the population proportion favoring fresh coffee is greater than 0.5, so the sample outcome is about what would be expected.
We cannot be certain that Explanation 1 is untrue. Our taste test results could be due to chance alone. But the probability that such a result would occur by chance is so small (0.001) that we are quite confident that Explanation 2 is right.

9. Hypotheses and P-values
In most studies, we hope to show that some definite effect is present in the population. A statistical test begins by supposing for the sake of argument that the effect we seek is not present. We then look for evidence against this supposition and in favor of the effect we hope to find.
The first step in a test of significance is to state a claim against which we will try to find evidence.
Null Hypothesis H0
The claim being tested in a statistical test is called the null hypothesis. The test is designed to assess the strength of the evidence against the null hypothesis. Usually the null hypothesis is a statement of “no effect” or “no difference.”

10. Hypotheses and P-values
The term “null hypothesis” is abbreviated H0 and is read as “H-nought,” “H-oh,” and sometimes even “H-null.” It is a statement about the population and so must be stated in terms of a population parameter.
The statement we hope or suspect is true instead of H0 is called the alternative hypothesis and is abbreviated Ha.
A significance test looks for evidence against the null hypothesis and in favor of the alternative hypothesis. The evidence is strong if the outcome we observe would rarely occur if the null hypothesis is true, but is more probable if the alternative hypothesis is true.

11. Hypotheses and P-values
A significance test answers the question by giving a probability: the probability of getting an outcome at least as far as the actually observed outcome from what we would expect when H0 is true.
What counts as “far from what we would expect” depends on Ha as well as H0.
P-value
The probability, computed assuming that H0 is true, that the sample outcome would be as extreme or more extreme than the actually observed outcome is called the P-value of the test. The smaller the P-value, the stronger is the evidence against H0 provided by the data.

12. Hypotheses and P-values
The French naturalist Count Buffon (1707–1788) considered questions ranging from evolution to estimating the number “pi,” and made it his goal to answer them. One question he explored was whether a “balanced” coin would come up heads half of the time when tossed. To investigate, he tossed a coin 4040 times. He got 2048 heads. The sample proportion of heads is

13. Hypotheses and P-values

14. Hypotheses and P-values
The conclusion. A truly balanced coin would give a result this far or farther from 0.5 in 37% of all repetitions of Buffon’s trial.
His result gives no reason to think that his coin was not balanced.

15. Hypotheses and P-values
The alternative Ha: p > 0.5 in Example 1 is a one-sided alternative because the effect for which we seek evidence says that the population proportion is greater than one-half.
The alternative Ha: p ≠ 0.5 in Example 2 is a two-sided alternative because we ask only whether or not the coin is balanced.
Whether the alternative is one-sided or two-sided determines whether sample results that are extreme in one direction or in both directions count as evidence against H0 in favor of Ha.

16. Statistical Significance
We can decide in advance how much evidence against H0 we will insist on. The way to do this is to say, before any data are collected, how small a P-value we require.
The decisive value of P is called the significance level. It is usual to write it as α, the Greek letter alpha.
Statistical Significance
If the P-value is as small or smaller than α, we say that the data are statistically significant at level α.

17. Calculating P-values
Finding the P-values we gave in Examples 1 and 2 requires doing Normal distribution calculations using Table B of Normal percentiles.
In practice, software does the calculation for us, but here is an example that shows how to use Table B.

18. Calculating P-values

19. Tests for a Population Mean
The reasoning that leads to significance tests for hypotheses about a population mean μ follows the reasoning that leads to tests about a population proportion p.
The big idea is to use the sampling distribution that the sample mean would have if the null hypothesis were true.
Locate the value of x-bar from your data on this distribution and see if it is unlikely.
A value of x-bar that would rarely appear if H0 were true is evidence that H0 is not true.
The steps are similar to those in tests for a proportion.

20. Tests for a Population Mean

21. Tests for a Population Mean

22. Tests for a Population Mean

23. Chapter 22 Concepts The Reasoning of Tests of Significance
Hypotheses and P-values
Statistical Significance
Calculating P-values *
Tests for a Population Mean *