Presentation is loading. Please wait.

Presentation is loading. Please wait.

Bootstrap Distributions Or: How do we get a sense of a sampling distribution when we only have ONE sample?

Published byDakota Knell Modified over 9 years ago

Similar presentations

Presentation on theme: "Bootstrap Distributions Or: How do we get a sense of a sampling distribution when we only have ONE sample?"— Presentation transcript:

1 Bootstrap Distributions Or: How do we get a sense of a sampling distribution when we only have ONE sample?

2 Suppose we have a random sample of 6 people:

3 Bootstrap Sample: Sample with replacement from the original sample, using the same sample size. Original SampleBootstrap Sample

4 Original Sample Create a “sampling distribution” using this as our simulated population

5 Create a bootstrap sample by sampling with replacement from the original sample. Compute the relevant statistic for the bootstrap sample. Do this many times!! Gather the bootstrap statistics all together to form a bootstrap distribution.

6 Original Sample Bootstrap Sample...... Bootstrap Statistic Sample Statistic Bootstrap Statistic...... Bootstrap Distribution

7 Example: Atlanta Commutes Data: The American Housing Survey (AHS) collected data from Atlanta in 2004. What’s the mean commute time for workers in metropolitan Atlanta?

8 Sample of n=500 Atlanta Commutes Where might the “true” μ be?

9 How can we get a confidence interval from a bootstrap distribution? Method #1: Use the standard deviation of the bootstrap statistics as a “yardstick”

10 Using the Bootstrap Distribution to Get a Confidence Interval – Version #1 The standard deviation of the bootstrap statistics estimates the standard error of the sample statistic. Quick interval estimate : For the mean Atlanta commute time:

11 Using the Bootstrap Distribution to Get a Confidence Interval – Version #2 Keep 95% in middle Chop 2.5% in each tail For a 95% CI, find the 2.5%-tile and 97.5%-tile in the bootstrap distribution 95% CI=(27.35,30.96)

12 90% CI for Mean Atlanta Commute Keep 90% in middle Chop 5% in each tail For a 90% CI, find the 5%-tile and 95%-tile in the bootstrap distribution 90% CI=(27.64,30.65)

13 Bootstrap Confidence Intervals Version 1 (Statistic  2 SE): Great preparation for moving to traditional methods Version 2 (Percentiles): Great at building understanding of confidence intervals

14 That’s all folks

15 StatKey can be found at www.lock5stat.com

16 Playing with StatKey! See the purple pages in the folder.

17 We want to collect some data from you. What should we ask you for our one quantitative question and our one categorical question?

18 What quantitative data should we collect from you? A.What was the class size of the Intro Stat course you taught most recently? B.How many years have you been teaching Intro Stat? C.What was the travel time, in hours, for your trip to Boston for JMM? D.Including this one, how many times have you attended the January JMM? E.???

19 What categorical data should we collect from you? A.Did you fly or drive to these meetings? B.Have you attended any previous JMM meetings? C.Have you ever attended a JSM meeting? D.??? E.???

20 How do we assess student understanding of these methods (even on in-class exams without computers)? See the green pages in the folder.

21 http://www.youtube.com/watch?v=3ESGpRUMj9E Paul the Octopus http://www.cnn.com/2010/SPORT/football/07/08/germany.octopus.explainer/index.html

22 Paul the Octopus predicted 8 World Cup games, and predicted them all correctly Is this evidence that Paul actually has psychic powers? How unusual would this be if he were just randomly guessing (with a 50% chance of guessing correctly)? How could we figure this out? Paul the Octopus

23 Each coin flip = a guess between two teams Heads = correct, Tails = incorrect Flip a coin 8 times and count the number of heads. Remember this number! Did you get all 8 heads? (a) Yes (b) No Simulate!

24 Let p denote the proportion of games that Paul guesses correctly (of all games he may have predicted) H 0 : p = 1/2 H a : p > 1/2 Hypotheses

25 A randomization distribution is the distribution of sample statistics we would observe, just by random chance, if the null hypothesis were true A randomization distribution is created by simulating many samples, assuming H 0 is true, and calculating the sample statistic each time Randomization Distribution

26 Let’s create a randomization distribution for Paul the Octopus! On a piece of paper, set up an axis for a dotplot, going from 0 to 8 Create a randomization distribution using each other’s simulated statistics For more simulations, we use StatKey Randomization Distribution

27 The p-value is the probability of getting a statistic as extreme (or more extreme) as that observed, just by random chance, if the null hypothesis is true This can be calculated directly from the randomization distribution! p-value

28 StatKey

29 Create a randomization distribution by simulating assuming the null hypothesis is true The p-value is the proportion of simulated statistics as extreme as the original sample statistic Randomization Test

30 How do we create randomization distributions for other parameters? How do we assess student understanding? Connecting intervals and tests Technology for using simulation methods Experiences in the classroom Coming Attractions - Friday

31 Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2 Lock, Lock, Lock, Lock, and Lock Minicourse- Joint Mathematics Meetings Boston, MA January 2012

32 In a randomized experiment on treating cocaine addiction, 48 people were randomly assigned to take either Desipramine (a new drug), or Lithium (an existing drug) The outcome variable is whether or not a patient relapsed Is Desipramine significantly better than Lithium at treating cocaine addiction? Cocaine Addiction

33 RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRR RRRRRR RRRRRR RRRRRR RRRR RRRRRR RRRRRR RRRRRR Desipramine Lithium 1. Randomly assign units to treatment groups

34 RRRR RRRRRR RRRRRR NNNNNN RRRRRR RRRRNN NNNNNN RR NNNNNN R = Relapse N = No Relapse RRRR RRRRRR RRRRRR NNNNNN RRRRRR RRRRRR RRNNNN RR NNNNNN 2. Conduct experiment 3. Observe relapse counts in each group Lithium Desipramine 10 relapse, 14 no relapse18 relapse, 6 no relapse 1. Randomly assign units to treatment groups

35 Assume the null hypothesis is true Simulate new randomizations For each, calculate the statistic of interest Find the proportion of these simulated statistics that are as extreme as your observed statistic Randomization Test

36 RRRR RRRRRR RRRRRR NNNNNN RR RRRR RRRRNN NNNNNN RR NNNNNN 10 relapse, 14 no relapse18 relapse, 6 no relapse

37 RRRRRR RRRRNN NNNNNN NNNNNN RRRRRR RRRRRR RRRRRR NNNNNN RNRN RRRRRR RNRRRN RNNNRR NNNR NRRNNN NRNRRN RNRRRR Simulate another randomization Desipramine Lithium 16 relapse, 8 no relapse12 relapse, 12 no relapse

38 RRRR RRRRRR RRRRRR NNNNNN RR RRRR RNRRNN RRNRNR RR RNRNRR Simulate another randomization Desipramine Lithium 17 relapse, 7 no relapse11 relapse, 13 no relapse

39 Combine everyone into one group, and rerandomize them into the two groups Compute your difference in proportions Create the randomization distribution How extreme is the observed statistic of -0.33? Use StatKey for more simulations Simulate!

40 The observed difference in proportions was -0.33. How unlikely is this if there is no difference in the drugs? To begin to get a sense of this: How many of you had a randomly simulated difference in proportions this extreme? Was your simulated difference in proportions more extreme than the sample statistic (that is, was it less than or equal to -0.33)? A.Yes B.No

41 StatKey The probability of getting results as extreme or more extreme than those observed if the null hypothesis is true, is about.02. p-value Proportion as extreme as observed statistic observed statistic

42 Why did you re-deal your cards? Why did you leave the outcomes (relapse or no relapse) unchanged on each card? Cocaine Addiction You want to know what would happen by random chance (the random allocation to treatment groups) if the null hypothesis is true (there is no difference between the drugs)

43 How can we do a randomization test for a mean?

44 Example: Mean Body Temperature Data: A random sample of n=50 body temperatures. Is the average body temperature really 98.6 o F? H 0 :μ=98.6 H a :μ≠98.6 Data from Allen Shoemaker, 1996 JSE data set article

45 Key idea: Generate samples that are (a) consistent with the null hypothesis (b) based on the sample data. How to simulate samples of body temperatures to be consistent with H 0 : μ=98.6?

46 Randomization Samples How to simulate samples of body temperatures to be consistent with H 0 : μ=98.6?

47 Let’s try it on StatKey.

48 How can we do a randomization test for a correlation?

49 Is the number of penalties given to an NFL team positively correlated with the “malevolence” of the team’s uniforms?

50 Ex: NFL uniform “malevolence” vs. Penalty yards r = 0.430 n = 28 Is there evidence that the population correlation is positive?

51 Key idea: Generate samples that are (a) consistent with the null hypothesis (b) based on the sample data. H 0 :  = 0 r = 0.43, n = 28 How can we use the sample data, but ensure that the correlation is zero?

52 Randomize one of the variables! Let’s look at StatKey.

53 Playing with StatKey! See the orange pages in the folder.

54 Choosing a Randomization Method A=Sleep14181113181721916171415mean=15.25 B=Caffeine12 141361814161071510mean=12.25 Example: Word recall Option 1: Randomly scramble the A and B labels and assign to the 24 word recalls. H 0 : μ A =μ B vs. H a : μ A ≠μ B Option 2: Combine the 24 values, then sample (with replacement) 12 values for Group A and 12 values for Group B. Reallocate Resample

55 Question In Intro Stat, how critical is it for the method of randomization to reflect the way data were collected? A. Essential B. Relatively important C. Desirable, but not imperative D. Minimal importance E. Ignore the issue completely

56 How do we assess student understanding of these methods (even on in-class exams without computers)? See the blue pages in the folder.

57 Collecting More Data from You!

58 Rock-Paper-Scissors (Roshambo) Play a game! Can we use statistics to help us win?

59 Rock-Paper-Scissors Which did you throw? A). Rock B). Paper C). Scissors

60 Rock-Paper-Scissors Are the three options thrown equally often on the first throw? In particular, is the proportion throwing Rock equal to 1/3?

61 What about Traditional Methods?

62 Intro Stat – Revised the Topics Descriptive Statistics – one and two samples Confidence intervals (means/proportions) Hypothesis tests (means/proportions) ANOVA for several means, Inference for regression, Chi-square tests Data production (samples/experiments) Normal/sampling distributions Bootstrap confidence intervals Randomization-based hypothesis tests

63 Transitioning to Traditional Inference AFTER students have seen lots of bootstrap distributions and randomization distributions… Students should be able to Find, interpret, and understand a confidence interval Find, interpret, and understand a p-value

64 Slope :Restaurant tips Correlation: Malevolent uniforms Mean :Body Temperatures Diff means: Finger taps Mean : Atlanta commutes Proportion : Owners/dogs What do you notice? All bell-shaped distributions! Bootstrap and Randomization Distributions

65 The students are primed and ready to learn about the normal distribution!

66 Transitioning to Traditional Inference Introduce the normal distribution (and later t) Introduce “shortcuts” for estimating SE for proportions, means, differences, slope…

67 z*z* -z * 95% Confidence Intervals

68 Test statistic 95% Hypothesis Tests Area is p-value

69 Yes! Students see the general pattern and not just individual formulas!

70 Connecting CI’s and Tests Randomization body temp means when μ=98.6 Bootstrap body temp means from the original sample What’s the difference?

71 Fathom Demo: Test & CI Sample mean is in the “rejection region” Null mean is outside the confidence interval

72 Technology Sessions Choose Two! (The folder includes information on using Minitab, R, Excel, Fathom, Matlab, and SAS.)

73 Student Preferences Which way did you prefer to learn inference (confidence intervals and hypothesis tests)? Bootstrapping and Randomization Formulas and Theoretical Distributions 3919 67%33%

74 Student Preferences Which way do you prefer to do inference? Bootstrapping and Randomization Formulas and Theoretical Distributions 4216 72%28%

75 Student Preferences Which way of doing inference gave you a better conceptual understanding of confidence intervals and hypothesis tests? Bootstrapping and Randomization Formulas and Theoretical Distributions 4216 72%27%

76 Student Preferences DO inferenceSimulationTraditional AP Stat1810 No AP Stat246 LEARN inferenceSimulationTraditional AP Stat1315 No AP Stat264 UNDERSTANDSimulationTraditional AP Stat1711 No AP Stat255

77 Thank you for joining us! More information is available on www.lock5stat.com www.lock5stat.com Feel free to contact any of us with any comments or questions.

Download ppt "Bootstrap Distributions Or: How do we get a sense of a sampling distribution when we only have ONE sample?"

Similar presentations

Panel at 2013 Joint Mathematics Meetings

Panel at 2013 Joint Mathematics Meetings
What Can We Do When Conditions Arent Met? Robin H. Lock, Burry Professor of Statistics St. Lawrence University BAPS at 2012 JSM San Diego, August 2012.

What Can We Do When Conditions Arent Met? Robin H. Lock, Burry Professor of Statistics St. Lawrence University BAPS at 2012 JSM San Diego, August 2012.
Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse –

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse –
Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock, Lock, and Lock MAA Minicourse – Joint Mathematics.

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock, Lock, and Lock MAA Minicourse – Joint Mathematics.
Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 10/23/12 Sections 6.1-6.3, 6.7-6.9 Single Proportion, p Distribution (6.1)

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 10/23/12 Sections , Single Proportion, p Distribution (6.1)
Hypothesis Testing: Intervals and Tests

Hypothesis Testing: Intervals and Tests
Early Inference: Using Bootstraps to Introduce Confidence Intervals Robin H. Lock, Burry Professor of Statistics Patti Frazer Lock, Cummings Professor.

Early Inference: Using Bootstraps to Introduce Confidence Intervals Robin H. Lock, Burry Professor of Statistics Patti Frazer Lock, Cummings Professor.
Hypothesis Testing I 2/8/12 More on bootstrapping Random chance

Hypothesis Testing I 2/8/12 More on bootstrapping Random chance
Intuitive Introduction to the Important Ideas of Inference Robin Lock – St. Lawrence University Patti Frazer Lock – St. Lawrence University Kari Lock Morgan.

Intuitive Introduction to the Important Ideas of Inference Robin Lock – St. Lawrence University Patti Frazer Lock – St. Lawrence University Kari Lock Morgan.
Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: p-value STAT 250 Dr. Kari Lock Morgan SECTION 4.2 Randomization distribution p-value.

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: p-value STAT 250 Dr. Kari Lock Morgan SECTION 4.2 Randomization distribution p-value.
Introduction to Hypothesis Testing AP Statistics Chap 11-1.

Introduction to Hypothesis Testing AP Statistics Chap 11-1.
A Fiddler on the Roof: Tradition vs. Modern Methods in Teaching Inference Patti Frazer Lock Robin H. Lock St. Lawrence University Joint Mathematics Meetings.

A Fiddler on the Roof: Tradition vs. Modern Methods in Teaching Inference Patti Frazer Lock Robin H. Lock St. Lawrence University Joint Mathematics Meetings.
Hypothesis Tests for Means The context “Statistical significance” Hypothesis tests and confidence intervals The steps Hypothesis Test statistic Distribution.

Hypothesis Tests for Means The context “Statistical significance” Hypothesis tests and confidence intervals The steps Hypothesis Test statistic Distribution.
Stat 217 – Day 25 Regression. Last Time - ANOVA When?  Comparing 2 or means (one categorical and one quantitative variable) Research question  Null.

Stat 217 – Day 25 Regression. Last Time - ANOVA When?  Comparing 2 or means (one categorical and one quantitative variable) Research question  Null.
STAT 101 Dr. Kari Lock Morgan Exam 2 Review.

STAT 101 Dr. Kari Lock Morgan Exam 2 Review.
Connecting Simulation- Based Inference with Traditional Methods Kari Lock Morgan, Penn State Robin Lock, St. Lawrence University Patti Frazer Lock, St.

Connecting Simulation- Based Inference with Traditional Methods Kari Lock Morgan, Penn State Robin Lock, St. Lawrence University Patti Frazer Lock, St.
Section 4.4 Creating Randomization Distributions.

Section 4.4 Creating Randomization Distributions.
Starting Inference with Bootstraps and Randomizations Robin H. Lock, Burry Professor of Statistics St. Lawrence University Stat Chat Macalester College,

Starting Inference with Bootstraps and Randomizations Robin H. Lock, Burry Professor of Statistics St. Lawrence University Stat Chat Macalester College,
Using Simulation Methods to Introduce Statistical Inference Patti Frazer Lock Kari Lock Morgan Cummings Professor of Mathematics Assistant Professor of.

Using Simulation Methods to Introduce Statistical Inference Patti Frazer Lock Kari Lock Morgan Cummings Professor of Mathematics Assistant Professor of.
Building Conceptual Understanding of Statistical Inference with Lock 5 Dr. Kari Lock Morgan Department of Statistical Science Duke University Wake Forest.

Building Conceptual Understanding of Statistical Inference with Lock 5 Dr. Kari Lock Morgan Department of Statistical Science Duke University Wake Forest.

Similar presentations

Ads by Google