1. Hypothesis Testing I 2/8/12 More on bootstrapping Random chance
Null and alternative hypotheses
Randomization distribution
p-value
Section 4.1, 4.2
Professor Kari Lock Morgan
Duke University

2. Announcements Homework 3 (due Monday)
Research question and data for Project 1
(proposal due next Wednesday)

3. Bootstrap CI
Option 1: Estimate the standard error of the statistic by computing the standard deviation of the bootstrap distribution, and then generate a 95% confidence interval by
Option 2: Generate a P% confidence interval as the range for the middle P% of bootstrap statistics

4. Suppose we have a random sample of 6 people:
Patti

5. Original Sample
Patti
Create a “sampling distribution” using this as our simulated population

6. Bootstrap Sample: Sample with replacement from the original sample, using the same sample size.
Patti
Original Sample
Bootstrap Sample

7. Continuous versus Discrete
A continuous distribution can take any value within some range. The distribution will look like a smooth curve, without any gaps
A discrete distribution only takes certain values. The distribution will be spiky, with gaps.
Continuous
Discrete

8. Criteria for Bootstrap CI
Using the percentile method for a confidence interval bootstrapping for a confidence interval works for any statistic, as long as the bootstrap distribution is
Approximately symmetric
Approximately continuous
Using the standard error method also requires
Approximately bell-shaped
Always look at the bootstrap distribution to make sure these are true!

9. Criteria for Bootstrap CI

10. Number of Bootstrap Samples
When using bootstrapping, you may get a slightly different confidence interval each time. This is fine!
The more bootstrap samples you use, the more precise your answer will be.
For the purposes of this class, 1000 bootstrap samples is fine. In real life, you probably want to take 10,000 or even 100,000 bootstrap samples

11. Paul the Octopus

12. Paul the Octopus
Paul the Octopus predicted 8 World Cup games, and predicted them all correctly
Is this evidence that Paul actually has psychic powers?
How unusual would this be if he was just randomly guessing (with a 50% chance of guessing correctly)?
How could we figure this out?

13. Coins and Paul
Did you get all 8 heads?
(a) Yes
(b) No

14. 10,000 Simulations www.lock5stat.com/statkey
If Paul is just guessing, the chance of him getting all 8 correct is 4/1000.

15. Hypotheses
Null Hypothesis, H0 : Claim that there is no effect or no difference
Alternative Hypothesis, Ha : Claim that we seek evidence for, usually that there is some effect
Hypotheses are always given in terms of population parameters
Paul the Octopus:
H0 : 50% chance of correctly predicting each game
Ha : >50% chance of correctly predicting each game

16. Paul the Octopus
In the case of Paul the Octopus, what type of parameter are we interested in?
(Hint: Are there one or two variables?
Are the variable(s) categorical or quantitative?)
(a) Proportion
(b) Mean
(c) Difference in proportions
(d) Difference in means
(e) Correlation
Paul the Octopus:
H0 : 50% chance of correctly predicting each game
Ha : >50% chance of correctly predicting each game

17. Paul the Octopus Ha : p > 1/2
Let p denote the proportion of games that Paul guesses correctly (of all games he may have predicted)
H0 : p = 1/2
Ha : p > 1/2

18. Hypotheses
The alternative hypothesis is supported by finding evidence (data) that contradicts the null hypothesis (and supports the alternative hypothesis)
Data can only contradict or not contradict the null hypothesis, but can never confirm it

19. Alternative Hypothesis
Hypotheses
Null Hypothesis
Alternative Hypothesis
Usually the null is a very specific statement, and so straightforward to assess evidence against
ALL POSSIBILITIES

20. Paul and Hypotheses
H0 : p = 1/2
Ha : p > 1/2
What if Paul had gotten 4 out of 8 correct? What would you conclude?
H0 is true
Ha is true
H0 is false
Ha is false
Nothing

21. Your Own Hypotheses
Come up with a situation where you want to establish a claim based on data
What parameter(s) are you interested in?
What would the null and alternative hypotheses be?
What type of data would lead you to believe the null hypothesis is probably not true?

22. Measuring Evidence against H0
To see if a statistic provides evidence against H0, we need to see what kind of sample statistics we would observe, just by random chance,
if H0 were true

23. Paul the Octopus

24. Randomization Distribution
A randomization distribution is the distribution of sample statistics we would observe, just by random chance, if the null hypothesis were true
Simulate many randomizations, assuming H0 is true, calculate the sample statistic each time, and collect these together to form a distribution

25. Randomization Distribution

26. p-value We calculate this from a randomization distribution
The p-value is the probability of getting a statistic as extreme (or more extreme) as that observed, just by random chance, if the null hypothesis is true
We calculate this from a randomization distribution

27. Paul the Octopus
p-value

28. p-value
What kinds of statistics would we get, just by random chance, if the null hypothesis were true? (randomization distribution)
What proportion of these statistics are as extreme as our original sample statistic?
(p-value)

29. Exercise and Pulse
Does just 5 seconds of exercise raise your pulse rate?
Let’s find out!
How can we answer this question?