Presentation is loading. Please wait.

Presentation is loading. Please wait.

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: Intervals and Tests STAT 101 Dr. Kari Lock Morgan 10/2/12 SECTION 4.3, 4.4, 4.5 Type.

Similar presentations


Presentation on theme: "Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: Intervals and Tests STAT 101 Dr. Kari Lock Morgan 10/2/12 SECTION 4.3, 4.4, 4.5 Type."— Presentation transcript:

1 Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: Intervals and Tests STAT 101 Dr. Kari Lock Morgan 10/2/12 SECTION 4.3, 4.4, 4.5 Type I and II errors (4.3) More randomization distributions (4.4) Connecting intervals and tests (4.5)

2 Statistics: Unlocking the Power of Data Lock 5 Proposals Project 1 proposal comments  Give spreadsheet with data in correct format  Cases and variables

3 Statistics: Unlocking the Power of Data Lock 5 Reminders Highest scorer on correlation guessing game gets an extra point on Exam 1! Deadline: noon on Thursday, 10/11.correlation guessing game First student to get a red card gets an extra point on Exam 1!

4 Statistics: Unlocking the Power of Data Lock 5 There are four possibilities: Errors Reject H 0 Do not reject H 0 H 0 true H 0 false TYPE I ERROR TYPE II ERROR Truth Decision A Type I Error is rejecting a true null A Type II Error is not rejecting a false null

5 Statistics: Unlocking the Power of Data Lock 5 In the test to see if resveratrol is associated with food intake, the p-value is o If resveratrol is not associated with food intake, a Type I Error would have been made In the test to see if resveratrol is associated with locomotor activity, the p-value is o If resveratrol is associated with locomotor activity, a Type II Error would have been made Red Wine and Weight Loss

6 Statistics: Unlocking the Power of Data Lock 5 A person is innocent until proven guilty. Evidence must be beyond the shadow of a doubt. Types of mistakes in a verdict? Convict an innocent Release a guilty HoHo HaHa  Type I error Type II error Analogy to Law p-value from data

7 Statistics: Unlocking the Power of Data Lock 5 The probability of making a Type I error (rejecting a true null) is the significance level, α α should be chosen depending how bad it is to make a Type I error Probability of Type I Error

8 Statistics: Unlocking the Power of Data Lock 5 If the null hypothesis is true: 5% of statistics will be in the most extreme 5% 5% of statistics will give p-values less than % of statistics will lead to rejecting H 0 at α = 0.05 If α = 0.05, there is a 5% chance of a Type I error Distribution of statistics, assuming H 0 true: Probability of Type I Error

9 Statistics: Unlocking the Power of Data Lock 5 If the null hypothesis is true: 1% of statistics will be in the most extreme 1% 1% of statistics will give p-values less than % of statistics will lead to rejecting H 0 at α = 0.01 If α = 0.01, there is a 1% chance of a Type I error Distribution of statistics, assuming H 0 true: Probability of Type I Error

10 Statistics: Unlocking the Power of Data Lock 5 Probability of Type II Error The probability of making a Type II Error (not rejecting a false null) depends on  Effect size (how far the truth is from the null)  Sample size  Variability  Significance level

11 Statistics: Unlocking the Power of Data Lock 5 Choosing α By default, usually α = 0.05 If a Type I error (rejecting a true null) is much worse than a Type II error, we may choose a smaller α, like α = 0.01 If a Type II error (not rejecting a false null) is much worse than a Type I error, we may choose a larger α, like α = 0.10

12 Statistics: Unlocking the Power of Data Lock 5 Come up with a hypothesis testing situation in which you may want to… Use a smaller significance level, like  = 0.01 Use a larger significance level, like  = 0.10 Significance Level

13 Statistics: Unlocking the Power of Data Lock 5 Randomization Distributions p-values can be calculated by randomization distributions:  simulate samples, assuming H 0 is true  calculate the statistic of interest for each sample  find the p-value as the proportion of simulated statistics as extreme as the observed statistic Today we’ll see ways to simulate randomization samples for more situations

14 Statistics: Unlocking the Power of Data Lock 5 Randomization Distribution We need to generate randomization samples assuming the null hypothesis is true.

15 Statistics: Unlocking the Power of Data Lock 5 Randomization Distribution a) 10.2 b) 12 c) 45 d) 1.8 Randomization distributions are always centered around the null hypothesized value.

16 Statistics: Unlocking the Power of Data Lock 5 Randomization Distribution Center A randomization distribution is centered at the value of the parameter given in the null hypothesis. A randomization distribution simulates samples assuming the null hypothesis is true, so

17 Statistics: Unlocking the Power of Data Lock 5 Randomization Distribution a) How extreme 10.2 is b) How extreme 12 is c) How extreme 45 is d) What the standard error is e) How many randomization samples we collected We want to see how extreme the observed statistic is.

18 Statistics: Unlocking the Power of Data Lock 5 Randomization Distribution We need to generate randomization samples assuming the null hypothesis is true.

19 Statistics: Unlocking the Power of Data Lock 5 Randomization Distribution a) 0 b) 1 c) 21 d) 26 e) 5 The randomization distribution is centered around the null hypothesized value,  1 -  2 = 0

20 Statistics: Unlocking the Power of Data Lock 5 Randomization Distribution a) The standard error b) The center point c) How extreme 26 is d) How extreme 21 is e) How extreme 5 is We want to see how extreme the observed difference in means is.

21 Statistics: Unlocking the Power of Data Lock 5 Randomization Distribution For a randomization distribution, each simulated sample should… be consistent with the null hypothesis use the data in the observed sample reflect the way the data were collected

22 Statistics: Unlocking the Power of Data Lock 5 In randomized experiments the “randomness” is the random allocation to treatment groups If the null hypothesis is true, the response values would be the same, regardless of treatment group assignment To simulate what would happen just by random chance, if H 0 were true: o reallocate cases to treatment groups, keeping the response values the same Randomized Experiments

23 Statistics: Unlocking the Power of Data Lock 5 Observational Studies In observational studies, the “randomness” is random sampling from the population To simulate what would happen, just by random chance, if H 0 were true:  Simulate resampling from a population in which H 0 is true How do we simulate resampling from a population when we only have sample data?  Bootstrap! How can we generate randomization samples for observational studies?  Make H 0 true, then bootstrap!

24 Statistics: Unlocking the Power of Data Lock 5 Body Temperatures

25 Statistics: Unlocking the Power of Data Lock 5 In StatKey, when we enter the null hypothesis, this shifting is automatically done for us StatKey Body Temperatures p-value = 0.002

26 Statistics: Unlocking the Power of Data Lock 5 Creating Randomization Samples State null and alternative hypotheses Devise a way to generate a randomization sample that  Uses the observed sample data  Makes the null hypothesis true  Reflects the way the data were collected 1. Do males exercise more hours per week than females? 2. Is blood pressure negatively correlated with heart rate?

27 Statistics: Unlocking the Power of Data Lock 5 Exercise and Gender H 0 :  m =  f, H a :  m >  f To make H 0 true, we must make the means equal. One way to do this is to add 3 to every female value (there are other ways) Bootstrap from this modified sample In StatKey, the default randomization method is “reallocate groups”, but “Shift Groups” is also an option, and will do this

28 Statistics: Unlocking the Power of Data Lock 5 Exercise and Gender p-value = 0.095

29 Statistics: Unlocking the Power of Data Lock 5 Exercise and Gender The p-value is Using α = 0.05, we conclude…. a) Males exercise more than females, on average b) Males do not exercise more than females, on average c) Nothing Do not reject the null… we can’t conclude anything.

30 Statistics: Unlocking the Power of Data Lock 5 Blood Pressure and Heart Rate H 0 :  = 0, H a :  < 0 Two variables have correlation 0 if they are not associated. We can “break the association” by randomly permuting/scrambling/shuffling one of the variables Each time we do this, we get a sample we might observe just by random chance, if there really is no correlation

31 Statistics: Unlocking the Power of Data Lock 5 Blood Pressure and Heart Rate p-value = Even if blood pressure and heart rate are not correlated, we would see correlations this extreme about 22% of the time, just by random chance.

32 Statistics: Unlocking the Power of Data Lock 5 Randomization Distribution Paul the Octopus (single proportion):  Flip a coin 8 times Cocaine Addiction (randomized experiment):  Rerandomize cases to treatment groups, keeping response values fixed Body Temperature (single mean):  Shift to make H 0 true, then bootstrap Exercise and Gender (observational study):  Shift to make H 0 true, then bootstrap Blood Pressure and Heart Rate (correlation):  Randomly permute/scramble/shuffle one variable

33 Statistics: Unlocking the Power of Data Lock 5 Randomization Distributions Randomization samples should be generated  Consistent with the null hypothesis  Using the observed data  Reflecting the way the data were collected The specific method varies with the situation, but the general idea is always the same

34 Statistics: Unlocking the Power of Data Lock 5 As long as the original data is used and the null hypothesis is true for the randomization samples, most methods usually give similar answers in terms of a p-value StatKey generates the randomizations for you, so most important is not understanding how to generate randomization samples, but understanding why Generating Randomization Samples

35 Statistics: Unlocking the Power of Data Lock 5 Bootstrap and Randomization Distributions Bootstrap DistributionRandomization Distribution Our best guess at the distribution of sample statistics Our best guess at the distribution of sample statistics, if H 0 were true Centered around the observed sample statistic Centered around the null hypothesized value Simulate sampling from the population by resampling from the original sample Simulate samples assuming H 0 were true Big difference: a randomization distribution assumes H 0 is true, while a bootstrap distribution does not

36 Statistics: Unlocking the Power of Data Lock 5 Which Distribution? (a) is centered around the sample statistic, 6.7

37 Statistics: Unlocking the Power of Data Lock 5 Which Distribution? Intro stat students are surveyed, and we find that 152 out of 218 are female. Let p be the proportion of intro stat students at that university who are female. A bootstrap distribution is generated for a confidence interval for p, and a randomization distribution is generated to see if the data provide evidence that p > 1/2. Which distribution is the randomization distribution? (a) is centered around the null value, 1/2

38 Statistics: Unlocking the Power of Data Lock 5 There are two types of errors: rejecting a true null (Type I) and not rejecting a false null (Type II) Randomization samples should be generated  Consistent with the null hypothesis  Using the observed data  Reflecting the way the data were collected Summary

39 Statistics: Unlocking the Power of Data Lock 5 To Do Read Sections 4.4, 4.5 Do Homework 4 (due Thursday, 10/4)Homework 4


Download ppt "Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: Intervals and Tests STAT 101 Dr. Kari Lock Morgan 10/2/12 SECTION 4.3, 4.4, 4.5 Type."

Similar presentations


Ads by Google