Presentation is loading. Please wait.

Presentation is loading. Please wait.

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing, Synthesis STAT 101 Dr. Kari Lock Morgan 10/4/12 SECTION 4.5, Essential Synthesis B Connecting.

Similar presentations


Presentation on theme: "Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing, Synthesis STAT 101 Dr. Kari Lock Morgan 10/4/12 SECTION 4.5, Essential Synthesis B Connecting."— Presentation transcript:

1 Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing, Synthesis STAT 101 Dr. Kari Lock Morgan 10/4/12 SECTION 4.5, Essential Synthesis B Connecting intervals and tests (4.5) Statistical versus practical significance (4.5) Multiple testing (4.5) Synthesis activities

2 Statistics: Unlocking the Power of Data Lock 5 Exam 1: Thursday 10/11 Open only to a calculator and one double sided page of notes prepared by you Emphasis on conceptual understanding Exam 1

3 Statistics: Unlocking the Power of Data Lock 5 Last year’s midterm, with solutions, are available on the course website (under documents) Review problems are posted for you to work through Doing problems is the key to success!!! Practice

4 Statistics: Unlocking the Power of Data Lock 5 Work lots of practice problems! Take last year’s exams under realistic conditions (time yourself, do it all before looking at the solutions, etc.) Prepare a good cheat sheet and use it when working problems Read the corresponding sections in the book if there are concepts you are still confused about Keys to In-Class Exam Success

5 Statistics: Unlocking the Power of Data Lock 5 Monday Heather 4 – 6pm, Old Chem 211A Sam, 6 – 9pm, Old Chem 211A Tuesday Kari 1:30 – 2:30 pm, Old Chem 216 Tracy 5 – 7 pm, Old Chem 211A Wednesday Kari 1 – 3pm, Old Chem 216 Tracy 4:30 – 5:30 pm, Old Chem 211A Heather 8 – 9pm, Old Chem 211A Thursday Kari 1 – 2:30 pm, Old Chem 216 Office Hours Next Week

6 Statistics: Unlocking the Power of Data Lock 5 Clickers Reminder: sharing clickers is a case of academic dishonesty and will be treated as such. If caught clicking in with two clickers, everyone involved will receive a 0 for their entire clicker grade (10% of the final grade) be reported to the dean to follow up regarding academic dishonesty

7 Statistics: Unlocking the Power of Data Lock 5 Body Temperature

8 Statistics: Unlocking the Power of Data Lock 5 Body Temperature We also created a randomization distribution to see if average body temperature differs from 98.6  F by adding 0.34 to every value to make the null true, and then resampling with replacement from this modified sample:

9 Statistics: Unlocking the Power of Data Lock 5 Body Temperature These two distributions are identical (up to random variation from simulation to simulation) except for the center The bootstrap distribution is centered around the sample statistic, 98.26, while the randomization distribution is centered around the null hypothesized value, 98.6 The randomization distribution is equivalent to the bootstrap distribution, but shifted over

10 Statistics: Unlocking the Power of Data Lock 5 Body Temperature Bootstrap Distribution Randomization Distribution H 0 :  = 98.6 H a :  ≠

11 Statistics: Unlocking the Power of Data Lock 5 Body Temperature Bootstrap Distribution Randomization Distribution H 0 :  = 98.4 H a :  ≠ 98.4

12 Statistics: Unlocking the Power of Data Lock 5 Intervals and Tests If a 95% CI misses the parameter in H 0, then a two-tailed test should reject H 0 at a 5% significance level. If a 95% CI contains the parameter in H 0, then a two-tailed test should not reject H 0 at a 5% significance level.

13 Statistics: Unlocking the Power of Data Lock 5 Intervals and Tests A confidence interval represents the range of plausible values for the population parameter If the null hypothesized value IS NOT within the CI, it is not a plausible value and should be rejected If the null hypothesized value IS within the CI, it is a plausible value and should not be rejected

14 Statistics: Unlocking the Power of Data Lock 5 Using bootstrapping, we found a 95% confidence interval for the mean body temperature to be (98.05 ,  ) This does not contain 98.6 , so at α = 0.05 we would reject H 0 for the hypotheses H 0 :  = 98.6  H a :  ≠ 98.6  Body Temperatures

15 Statistics: Unlocking the Power of Data Lock 5 Both Father and Mother “Does a child need both a father and a mother to grow up happily?” Let p be the proportion of adults aged in 2010 who say yes. A 95% CI for p is (0.487, 0.573). Testing H 0 : p = 0.5 vs H a : p ≠ 0.5 with α = 0.05, we a) Reject H 0 b) Do not reject H 0 c) Reject H a d) Do not reject H a 0.5 is within the CI, so is a plausible value for p. millennials-parenthood-trumps-marriage/#fn

16 Statistics: Unlocking the Power of Data Lock 5 Both Father and Mother “Does a child need both a father and a mother to grow up happily?” Let p be the proportion of adults aged in 1997 who say yes. A 95% CI for p is (0.533, 0.607). Testing H 0 : p = 0.5 vs H a : p ≠ 0.5 with α = 0.05, we a) Reject H 0 b) Do not reject H 0 c) Reject H a d) Do not reject H a 0.5 is not within the CI, so is not a plausible value for p. millennials-parenthood-trumps-marriage/#fn

17 Statistics: Unlocking the Power of Data Lock 5 Intervals and Tests Confidence intervals are most useful when you want to estimate population parameters Hypothesis tests and p-values are most useful when you want to test hypotheses about population parameters Confidence intervals give you a range of plausible values; p-values quantify the strength of evidence against the null hypothesis

18 Statistics: Unlocking the Power of Data Lock 5 Interval, Test, or Neither? Is the following question best assessed using a confidence interval, a hypothesis test, or is statistical inference not relevant? On average, how much more do adults who played sports in high school exercise than adults who did not play sports in high school? a) Confidence interval b) Hypothesis test c) Statistical inference not relevant

19 Statistics: Unlocking the Power of Data Lock 5 Interval, Test, or Neither? Is the following question best assessed using a confidence interval, a hypothesis test, or is statistical inference not relevant? Do a majority of adults riding a bicycle wear a helmet? a) Confidence interval b) Hypothesis test c) Statistical inference not relevant

20 Statistics: Unlocking the Power of Data Lock 5 Interval, Test, or Neither? Is the following question best assessed using a confidence interval, a hypothesis test, or is statistical inference not relevant? On average, were the 23 players on the 2010 Canadian Olympic hockey team older than the 23 players on the 2010 US Olympic hockey team? a) Confidence interval b) Hypothesis test c) Statistical inference not relevant

21 Statistics: Unlocking the Power of Data Lock 5 With small sample sizes, even large differences or effects may not be significant With large sample sizes, even a very small difference or effect can be significant A statistically significant result is not always practically significant, especially with large sample sizes Statistical vs Practical Significance

22 Statistics: Unlocking the Power of Data Lock 5 Example: Suppose a weight loss program recruits 10,000 people for a randomized experiment. A difference in average weight loss of only 0.5 lbs could be found to be statistically significant Suppose the experiment lasted for a year. Is a loss of ½ a pound practically significant? Statistical vs Practical Significance

23 Statistics: Unlocking the Power of Data Lock 5 Diet and Sex of Baby Are certain foods in your diet associated with whether or not you conceive a boy or a girl? To study this, researchers asked women about their eating habits, including asking whether or not they ate 133 different foods regularly A significant difference was found for breakfast cereal (mothers of boys eat more), prompting the headline “Breakfast Cereal Boosts Chances of Conceiving Boys”.

24 Statistics: Unlocking the Power of Data Lock 5 “Breakfast Cereal Boosts Chances of Conceiving Boys” I’m pregnant (with identical twins!), and am very curious about whether I’m going to have boys or girls! I eat breakfast cereal every morning. Do you think this boosts my chances of having boys? a) yes b) no c) impossible to tell

25 Statistics: Unlocking the Power of Data Lock 5 Hypothesis Tests For each of the 133 foods studied, a hypothesis test was conducted for a difference between mothers who conceived boys and girls in the proportion who consume each food State the null and alternative hypotheses If there are NO differences (all null hypotheses are true), about how many significant differences would be found using α = 0.05? A significant difference was found for breakfast cereal (mothers of boys eat more), prompting the headline “Breakfast Cereal Boosts Chances of Conceiving Boys”. How might you explain this?

26 Statistics: Unlocking the Power of Data Lock 5 Hypothesis Tests State the null and alternative hypotheses If there are NO differences (all null hypotheses are true), about how many significant differences would be found using α = 0.05? A significant difference was found for breakfast cereal (mothers of boys eat more), prompting the headline “Breakfast Cereal Boosts Chances of Conceiving Boys”. How might you explain this? p b : proportion of mothers who have boys that consume the food regularly p g : proportion of mothers who have girls that consume the food regularly H 0 : p b = p g H a : p b ≠ p g 133  0.05 = 6.65 Random chance; several tests (about 6 or 7) are going to be significant, even if no differences exist

27 Statistics: Unlocking the Power of Data Lock 5 Multiple Testing When multiple hypothesis tests are conducted, the chance that at least one test incorrectly rejects a true null hypothesis increases with the number of tests. If the null hypotheses are all true, α of the tests will yield statistically significant results just by random chance.

28 Statistics: Unlocking the Power of Data Lock 5 Author: JB Landers

29 Statistics: Unlocking the Power of Data Lock 5 Multiple Comparisons Consider a topic that is being investigated by research teams all over the world  Using α = 0.05, 5% of teams are going to find something significant, even if the null hypothesis is true

30 Statistics: Unlocking the Power of Data Lock 5 Multiple Comparisons Consider a research team/company doing many hypothesis tests  Using α = 0.05, 5% of tests are going to be significant, even if the null hypotheses are all true

31 Statistics: Unlocking the Power of Data Lock 5 This is a serious problem The most important thing is to be aware of this issue, and not to trust claims that are obviously one of many tests (unless they specifically mention an adjustment for multiple testing) There are ways to account for this (e.g. Bonferroni’s Correction), but these are beyond the scope of this class Multiple Comparisons

32 Statistics: Unlocking the Power of Data Lock 5 Publication Bias publication bias refers to the fact that usually only the significant results get published The one study that turns out significant gets published, and no one knows about all the insignificant results This combined with the problem of multiple comparisons, can yield very misleading results

33 Statistics: Unlocking the Power of Data Lock 5 Jelly Beans Cause Acne!

34 Statistics: Unlocking the Power of Data Lock 5

35

36

37 Statistics: Unlocking the Power of Data Lock 5 Summary If a null hypothesized value lies inside a 95% CI, a two-tailed test using α = 0.05 would not reject H 0 If a null hypothesized value lies outside a 95% CI, a two-tailed test using α = 0.05 would reject H 0 Statistical significance is not always the same as practical significance Using α = 0.05, 5% of all hypothesis tests will lead to rejecting the null, even if all the null hypotheses are true

38 Statistics: Unlocking the Power of Data Lock 5 Synthesis You’ve now learned how to successfully collect and analyze data to answer a question! Let’s put that to use…

39 Statistics: Unlocking the Power of Data Lock 5 Exercise and Pulse Does just 5 seconds of exercise increase pulse rate?  What are the cases and variables? Are they categorical or quantitative? Identify explanatory and response.  Does the question imply causality? How would you collect data to answer it?  Merge with 3 other groups to collect data. (check pulse rate)check pulse rate  Visualize and summarize your data. Before doing any formal inference, take a guess at answering the question.  Conduct a hypothesis test to answer the question. State your hypotheses, calculate the p-value, make a conclusion in context.  How much does 5 seconds of exercise increase pulse rate by? State the parameter of interest and give and interpret a confidence interval.

40 Statistics: Unlocking the Power of Data Lock 5 Tongue Curling What proportion of people can roll their tongue?  Can you roll your tongue? (a) Yes(b) No  Visualize and summarize the data. What is your point estimate?  Give and interpret a confidence interval.  Tongue rolling has been said to be a dominant trait, in which case theoretically 75% of all people should be able to roll their tongues. Do our data provide evidence otherwise?

41 Statistics: Unlocking the Power of Data Lock 5 Tuesday Tuesday’s class with be a review session There will be no clicker questions and no new material, so attendance is optional I’ll spend the first half reviewing the key topics we’ve covered so far, and then will have open Q and A

42 Statistics: Unlocking the Power of Data Lock 5 To Do Read Essential Synthesis A, B Prepare for Exam 1 (Thursday, 10/11)  Study  Make page of notes for Exam 1  Do review problemsreview problems  Take practice exampractice exam  Solutions under documents on course webpage


Download ppt "Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing, Synthesis STAT 101 Dr. Kari Lock Morgan 10/4/12 SECTION 4.5, Essential Synthesis B Connecting."

Similar presentations


Ads by Google