2. Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: Significance STAT 250 Dr. Kari Lock Morgan SECTION 4.3 Significance level (4.3) Statistical conclusions (4.3)

3. Statistics: Unlocking the Power of Data Lock 5 p-value and H 0 If the p-value is small, then a statistic as extreme as that observed would be unlikely if the null hypothesis were true, providing significant evidence against H 0 The smaller the p-value, the stronger the evidence against the null hypothesis and in favor of the alternative

4. Statistics: Unlocking the Power of Data Lock 5 The smaller the p-value, the stronger the evidence against H o. p-value and H 0

5. Statistics: Unlocking the Power of Data Lock 5 Which of the following p-values gives the strongest evidence against H 0 ? a) 0.005 b) 0.1 c) 0.32 d) 0.56 e) 0.94 p-value and H 0

6. Statistics: Unlocking the Power of Data Lock 5 Which of the following p-values gives the strongest evidence against H 0 ? a) 0.22 b) 0.45 c) 0.03 d) 0.8 e) 0.71 p-value and H 0

7. Statistics: Unlocking the Power of Data Lock 5 Two different studies obtain two different p- values. Study A obtained a p-value of 0.002 and Study B obtained a p-value of 0.2. Which study obtained stronger evidence against the null hypothesis? a) Study A b) Study B p-value and H 0

8. Statistics: Unlocking the Power of Data Lock 5 Formal Decisions If the p-value is small:  the sample would be extreme if H 0 were true  the results are statistically significant  REJECT H 0  we have evidence for H a If the p-value is not small:  the sample would not be too extreme if H 0 were true  the results are not statistically significant  DO NOT REJECT H 0  the test is inconclusive; either H 0 or H a may be true

9. Statistics: Unlocking the Power of Data Lock 5 A formal hypothesis test has only two possible conclusions: 1.The p-value is small: reject the null hypothesis in favor of the alternative 2.The p-value is not small: do not reject the null hypothesis Formal Decisions How small?

10. Statistics: Unlocking the Power of Data Lock 5 Significance Level The significance level, , is the threshold below which the p-value is deemed small enough to reject the null hypothesis p-value <   Reject H 0 p-value >   Do not Reject H 0

11. Statistics: Unlocking the Power of Data Lock 5 Significance Level If the p-value is less than , the results are statistically significant, and we reject the null hypothesis in favor of the alternative If the p-value is not less than , the results are not statistically significant, and our test is inconclusive Often  = 0.05 by default, unless otherwise specified https://www.openintro.org/stat/why05.php

12. Statistics: Unlocking the Power of Data Lock 5 Statistical Significance p-value < α Results would be rare, if the null were true Reject H 0 We have evidence that the alternative is true! p-value ≥ α Results would not be rare, if the null were true Do not reject H 0 We can make no conclusions either way

13. Statistics: Unlocking the Power of Data Lock 5 Statistical Conclusions In a hypothesis test of H 0 :  = 10 vs H a :  < 10 the p-value is 0.002. With α = 0.05, we conclude: a) Reject H 0 b) Do not reject H 0 c) Reject H a d) Do not reject H a

14. Statistics: Unlocking the Power of Data Lock 5 Statistical Conclusions In a hypothesis test of H 0 :  = 10 vs H a :  < 10 the p-value is 0.002. With α = 0.01, we conclude: a) There is evidence that  = 10 b) There is evidence that  < 10 c) We have insufficient evidence to conclude anything

15. Statistics: Unlocking the Power of Data Lock 5 Statistical Conclusions In a hypothesis test of H 0 :  = 10 vs H a :  < 10 the p-value is 0.21. With α = 0.01, we conclude: a) Reject H 0 b) Do not reject H 0 c) Reject H a d) Do not reject H a

16. Statistics: Unlocking the Power of Data Lock 5 Statistical Conclusions In a hypothesis test of H 0 :  = 10 vs H a :  < 10 the p-value is 0.21. With α = 0.01, we conclude: a) There is evidence that  = 10 b) There is evidence that  < 10 c) We have insufficient evidence to conclude anything

17. Statistics: Unlocking the Power of Data Lock 5 H 0 : X is an elephant H a : X is not an elephant Would you conclude, if you get the following data? X walks on two legs X has four legs Elephant Example

18. Statistics: Unlocking the Power of Data Lock 5 “For the logical fallacy of believing that a hypothesis has been proved to be true, merely because it is not contradicted by the available facts, has no more right to insinuate itself in statistical than in other kinds of scientific reasoning…” -Sir R. A. Fisher Never Accept H 0 “Do not reject H 0 ” is not the same as “accept H 0 ”! Lack of evidence against H 0 is NOT the same as evidence for H 0 !

19. Statistics: Unlocking the Power of Data Lock 5 Resveratrol, an ingredient in red wine and grapes, has been shown to promote weight loss in rodents, and has recently been investigated in primates (specifically, the Grey Mouse Lemur). A sample of lemurs had various measurements taken before and after receiving resveratrol supplementation for 4 weeks Red Wine and Weight Loss BioMed Central (2010, June 22). “Lemurs lose weight with ‘life-extending’ supplement resveratrol. Science Daily.

20. Statistics: Unlocking the Power of Data Lock 5 Red Wine and Weight Loss In the test to see if the mean resting metabolic rate is higher after treatment, the p-value is 0.013. Using  = 0.05, is this difference statistically significant? (should we reject H 0 : no difference?) a) Yes b) No

21. Statistics: Unlocking the Power of Data Lock 5 Red Wine and Weight Loss In the test to see if the mean resting metabolic rate is higher after treatment, the p-value is 0.013. Using  = 0.05, what can we conclude? a) Mean resting metabolic rate is higher after resveratrol supplementation in lemurs b) Mean resting metabolic rate is not higher after resveratrol supplementation in lemurs c) Nothing

22. Statistics: Unlocking the Power of Data Lock 5 Red Wine and Weight Loss In the test to see if the mean body mass is lower after treatment, the p-value is 0.007. Using  = 0.05, is this difference statistically significant? (should we reject H 0 : no difference?) a) Yes b) No

23. Statistics: Unlocking the Power of Data Lock 5 Red Wine and Weight Loss In the test to see if the mean body mass is lower after treatment, the p-value is 0.007. Using  = 0.05, what can we conclude? a) Mean body mass is higher after resveratrol supplementation in lemurs b) Mean body mass is not higher after resveratrol supplementation in lemurs c) Nothing

24. Statistics: Unlocking the Power of Data Lock 5 Red Wine and Weight Loss In the test to see if locomotor activity changes after treatment, the p-value is 0.980. Using  = 0.05, is this difference statistically significant? (should we reject H 0 : no difference?) a) Yes b) No

25. Statistics: Unlocking the Power of Data Lock 5 Red Wine and Weight Loss In the test to see if locomotor activity changes after treatment, the p-value is 0.980. Using  = 0.05, what can we conclude? a) Locomotor activity changes after resveratrol supplementation in lemurs b) Locomotor activity does not change after resveratrol supplementation in lemurs c) Nothing

26. Statistics: Unlocking the Power of Data Lock 5 Red Wine and Weight Loss In the test to see if mean food intake changes after treatment, the p-value is 0.035. Using  = 0.10, is this difference statistically significant? (should we reject H 0 : no difference?) a) Yes b) No

27. Statistics: Unlocking the Power of Data Lock 5 Red Wine and Weight Loss In the test to see if mean food intake changes after treatment, the p-value is 0.035. Using  = 0.01, is this difference statistically significant? (should we reject H 0 : no difference?) a) Yes b) No

28. Statistics: Unlocking the Power of Data Lock 5 Informal strength of evidence against H 0 : Formal decision of hypothesis test, based on  = 0.05 : Statistical Conclusions

29. Statistics: Unlocking the Power of Data Lock 5 Multiple Sclerosis and Sunlight It is believed that sunlight offers some protection against multiple sclerosis, but the reason is unknown Researchers randomly assigned mice to one of: Control (nothing) Vitamin D Supplements UV Light All mice were injected with proteins known to induce a mouse form of MS, and they observed which mice got MS Seppa, Nathan. “Sunlight may cut MS risk by itself”, Science News, April 24, 2010 pg 9, reporting on a study appearing March 22, 2010 in the Proceedings of the National Academy of Science.

30. Statistics: Unlocking the Power of Data Lock 5 Multiple Sclerosis and Sunlight For each situation below, write down  Null and alternative hypotheses  Informal description of the strength of evidence against H 0  Formal decision about H 0, using α = 0.05  Conclusion in the context of the question In testing whether UV light provides protection against MS (UV light vs control group), the p-value is 0.002. In testing whether Vitamin D provides protection against MS (Vitamin D vs control group), the p- value is 0.47.

31. Statistics: Unlocking the Power of Data Lock 5 Multiple Sclerosis and Sunlight In testing whether UV light provides protection against MS (UV light vs control group), the p-value is 0.002.

32. Statistics: Unlocking the Power of Data Lock 5 Multiple Sclerosis and Sunlight In testing whether Vitamin D provides protection against MS (Vitamin D vs control group), the p-value is 0.47.

33. Statistics: Unlocking the Power of Data Lock 5 Conclusions p-value < α Generic conclusion: Reject H 0 Conclusion in context: We have (strong?) evidence that [fill in alternative hypothesis] p-value ≥ α Generic conclusion: Do not reject H 0 Conclusion in context: We do not have enough evidence to conclude that [fill in alternative hypothesis] H0H0 HaHa

34. Statistics: Unlocking the Power of Data Lock 5 To Do Read Section 4.3 (will cover errors next class) Do HW 4.3 (due Friday, 3/20) Enjoy spring break!!!