1. BPS - 3rd Ed. Chapter 141 Tests of Significance: The Basics

2. BPS - 3rd Ed. Chapter 142 u What would happen if we repeated the sample or experiment many times? u How likely would it be to see the results we saw if the claim of the test were true? u Do the data give evidence against the claim? Reasoning of Tests of Significance

3. BPS - 3rd Ed. Chapter 143 u The statement being tested in a statistical test is called the null hypothesis. u The test is designed to assess the strength of evidence against the null hypothesis. u Usually the null hypothesis is a statement of “no effect” or “no difference”, or it is a statement of equality. u When performing a hypothesis test, we assume that the null hypothesis is true until we have sufficient evidence against it. Stating Hypotheses Null Hypothesis, H 0

4. BPS - 3rd Ed. Chapter 144 u The statement we are trying to find evidence for is called the alternative hypothesis. u Usually the alternative hypothesis is a statement of “there is an effect” or “there is a difference”, or it is a statement of inequality. u The alternative hypothesis should express the hopes or suspicions we bring to the data. It is cheating to first look at the data and then frame H a to fit what the data show. Stating Hypotheses Alternative Hypothesis, H a

5. BPS - 3rd Ed. Chapter 145 Case Study I Sweetening Colas Diet colas use artificial sweeteners to avoid sugar. These sweeteners gradually lose their sweetness over time. Trained testers sip the cola and assign a “sweetness score” of 1 to 10. The cola is then retested after some time and the two scores are compared to determine the difference in sweetness after storage. Bigger differences indicate bigger loss of sweetness.

6. BPS - 3rd Ed. Chapter 146 Case Study I Sweetening Colas Suppose we know that for any cola, the sweetness loss scores vary from taster to taster according to a Normal distribution with standard deviation  = 1. The mean  for all tasters measures loss of sweetness. The sweetness losses for a new cola, as measured by 10 trained testers, yields an average sweetness loss of = 1.02. Do the data provide sufficient evidence that the new cola lost sweetness in storage?

7. BPS - 3rd Ed. Chapter 147 Case Study I Sweetening Colas  If the claim that  = 0 is true (no loss of sweetness, on average), the sampling distribution of from 10 tasters is Normal with  = 0 and standard deviation  The data yielded = 1.02, which is more than three standard deviations from  = 0. This is strong evidence that the new cola lost sweetness in storage.  If the data yielded = 0.3, which is less than one standard deviations from  = 0, there would be no evidence that the new cola lost sweetness in storage.

8. BPS - 3rd Ed. Chapter 148 Case Study I Sweetening Colas

9. BPS - 3rd Ed. Chapter 149 The Hypotheses for Means  Null: H 0 :  =  0 u One sided alternatives H a :   H a :   u Two sided alternative H a :  0

10. BPS - 3rd Ed. Chapter 1410 Sweetening Colas Case Study I The null hypothesis is no average sweetness loss occurs, while the alternative hypothesis (that which we want to show is likely to be true) is that an average sweetness loss does occur. H 0 :  = 0H a :  > 0 This is considered a one-sided test because we are interested only in determining if the cola lost sweetness (gaining sweetness is of no consequence in this study).

11. BPS - 3rd Ed. Chapter 1411 Studying Job Satisfaction Case Study II Does the job satisfaction of assembly workers differ when their work is machine-paced rather than self-paced? A matched pairs study was performed on a sample of workers, and each worker’s satisfaction was assessed after working in each setting. The response variable is the difference in satisfaction scores, self- paced minus machine-paced.

12. BPS - 3rd Ed. Chapter 1412 Studying Job Satisfaction Case Study II The null hypothesis is no average difference in scores in the population of assembly workers, while the alternative hypothesis (that which we want to show is likely to be true) is there is an average difference in scores in the population of assembly workers. H 0 :  = 0H a :  ≠ 0 This is considered a two-sided test because we are interested determining if a difference exists (the direction of the difference is not of interest in this study).

13. BPS - 3rd Ed. Chapter 1413 Take an SRS of size n from a Normal population with unknown mean  and known standard deviation . The test statistic for hypotheses about the mean (H 0 :  =  0 ) of a Normal distribution is the standardized version of : Test Statistic Testing the Mean of a Normal Population

14. BPS - 3rd Ed. Chapter 1414 Sweetening Colas Case Study I If the null hypothesis of no average sweetness loss is true, the test statistic would be: Because the sample result is more than 3 standard deviations above the hypothesized mean 0, it gives strong evidence that the mean sweetness loss is not 0, but positive.

15. BPS - 3rd Ed. Chapter 1415 Assuming that the null hypothesis is true, the probability that the test statistic would take a value as extreme or more extreme than the value actually observed is called the P-value of the test. The smaller the P-value, the stronger the evidence the data provide against the null hypothesis. That is, a small P-value indicates a small likelihood of observing the sampled results if the null hypothesis were true. P-value

16. BPS - 3rd Ed. Chapter 1416 P-value for Testing Means  H a :  >  0 v P-value is the probability of getting a value as large or larger than the observed test statistic (z) value.  H a :  <  0 v P-value is the probability of getting a value as small or smaller than the observed test statistic (z) value.  H a :  0 v P-value is two times the probability of getting a value as large or larger than the absolute value of the observed test statistic (z) value.

17. BPS - 3rd Ed. Chapter 1417 Sweetening Colas Case Study I For test statistic z = 3.23 and alternative hypothesis H a :  > 0, the P-value would be: P-value = P(Z > 3.23) = 1 – 0.9994 = 0.0006 If H 0 is true, there is only a 0.0006 (0.06%) chance that we would see results at least as extreme as those in the sample; thus, since we saw results that are unlikely if H 0 is true, we therefore have evidence against H 0 and in favor of H a.

18. BPS - 3rd Ed. Chapter 1418 Sweetening Colas Case Study I

19. BPS - 3rd Ed. Chapter 1419 Studying Job Satisfaction Case Study II Suppose job satisfaction scores follow a Normal distribution with standard deviation  = 60. Data from 18 workers gave a sample mean score of 17. If the null hypothesis of no average difference in job satisfaction is true, the test statistic would be:

20. BPS - 3rd Ed. Chapter 1420 Studying Job Satisfaction Case Study II For test statistic z = 1.20 and alternative hypothesis H a :  ≠ 0, the P-value would be: P-value= P(Z 1.20) = 2 P(Z 1.20) = (2)(0.1151) = 0.2302 If H 0 is true, there is a 0.2302 (23.02%) chance that we would see results at least as extreme as those in the sample; thus, since we saw results that are likely if H 0 is true, we therefore do not have good evidence against H 0 and in favor of H a.

21. BPS - 3rd Ed. Chapter 1421 Studying Job Satisfaction Case Study II

22. BPS - 3rd Ed. Chapter 1422  If the P-value is as small as or smaller than the significance level  (i.e., P-value ≤  ), then we say that the data give results that are statistically significant at level .  If we choose  = 0.05, we are requiring that the data give evidence against H 0 so strong that it would occur no more than 5% of the time when H 0 is true.  If we choose  = 0.01, we are insisting on stronger evidence against H 0, evidence so strong that it would occur only 1% of the time when H 0 is true. Statistical Significance

23. BPS - 3rd Ed. Chapter 1423 The four steps in carrying out a significance test: 1. State the null and alternative hypotheses. 2. Calculate the test statistic. 3. Find the P-value. 4. State your conclusion in the context of the specific setting of the test. The procedure for Steps 2 and 3 is on the next page. Tests for a Population Mean

24. BPS - 3rd Ed. Chapter 1424

25. BPS - 3rd Ed. Chapter 1425 Sweetening Colas Case Study I 1. Hypotheses:H 0 :  = 0 H a :  > 0 2. Test Statistic: 3. P-value:P-value = P(Z > 3.23) = 1 – 0.9994 = 0.0006 4. Conclusion: Since the P-value is smaller than  = 0.01, there is very strong evidence that the new cola loses sweetness on average during storage at room temperature.

26. BPS - 3rd Ed. Chapter 1426 Studying Job Satisfaction Case Study II 1. Hypotheses:H 0 :  = 0 H a :  ≠ 0 2. Test Statistic: 3. P-value:P-value = 2P(Z > 1.20) = (2)(1 – 0.8849) = 0.2302 4. Conclusion: Since the P-value is larger than  = 0.10, there is not sufficient evidence that mean job satisfaction of assembly workers differs when their work is machine-paced rather than self-paced.

27. BPS - 3rd Ed. Chapter 1427 Confidence Intervals & Two-Sided Tests A level  two-sided significance test rejects the null hypothesis H 0 :  =  0 exactly when the value  0 falls outside a level 1 –  confidence interval for .

28. BPS - 3rd Ed. Chapter 1428 Case Study II A 90% confidence interval for  is: Since  0 = 0 is in this confidence interval, it is plausible that the true value of  is 0; thus, there is not sufficient evidence (at  = 0.10) that the mean job satisfaction of assembly workers differs when their work is machine-paced rather than self-paced. Studying Job Satisfaction