Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock, Lock, and Lock MAA Minicourse – Joint Mathematics.

Slides:

Advertisements

Similar presentations

Panel at 2013 Joint Mathematics Meetings

Advertisements

StatKey Online Tools for Teaching a Modern Introductory Statistics Course Robin Lock St. Lawrence University USCOTS Breakout – May 2013 Patti Frazer Lock.

What Can We Do When Conditions Arent Met? Robin H. Lock, Burry Professor of Statistics St. Lawrence University BAPS at 2012 JSM San Diego, August 2012.

Statistical Inference Using Scrambles and Bootstraps Robin Lock Burry Professor of Statistics St. Lawrence University MAA Allegheny Mountain 2014 Section.

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse –

Simulating with StatKey Kari Lock Morgan Department of Statistical Science Duke University Joint Mathematical Meetings, San Diego 1/11/13.

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 10/23/12 Sections , Single Proportion, p Distribution (6.1)

Hypothesis Testing: Intervals and Tests

Bootstrap Distributions Or: How do we get a sense of a sampling distribution when we only have ONE sample?

Early Inference: Using Bootstraps to Introduce Confidence Intervals Robin H. Lock, Burry Professor of Statistics Patti Frazer Lock, Cummings Professor.

Hypothesis Testing I 2/8/12 More on bootstrapping Random chance

Intuitive Introduction to the Important Ideas of Inference Robin Lock – St. Lawrence University Patti Frazer Lock – St. Lawrence University Kari Lock Morgan.

Models and Modeling in Introductory Statistics Robin H. Lock Burry Professor of Statistics St. Lawrence University 2012 Joint Statistics Meetings San Diego,

A Fiddler on the Roof: Tradition vs. Modern Methods in Teaching Inference Patti Frazer Lock Robin H. Lock St. Lawrence University Joint Mathematics Meetings.

Hypothesis Tests for Means The context “Statistical significance” Hypothesis tests and confidence intervals The steps Hypothesis Test statistic Distribution.

Stat 217 – Day 25 Regression. Last Time - ANOVA When?  Comparing 2 or means (one categorical and one quantitative variable) Research question  Null.

STAT 101 Dr. Kari Lock Morgan Exam 2 Review.

Connecting Simulation- Based Inference with Traditional Methods Kari Lock Morgan, Penn State Robin Lock, St. Lawrence University Patti Frazer Lock, St.

Starting Inference with Bootstraps and Randomizations Robin H. Lock, Burry Professor of Statistics St. Lawrence University Stat Chat Macalester College,

Using Simulation Methods to Introduce Statistical Inference Patti Frazer Lock Kari Lock Morgan Cummings Professor of Mathematics Assistant Professor of.

Building Conceptual Understanding of Statistical Inference with Lock 5 Dr. Kari Lock Morgan Department of Statistical Science Duke University Wake Forest.

Inference for Categorical Variables 2/29/12 Single Proportion, p Distribution Intervals and tests Difference in proportions, p 1 – p 2 One proportion or.

Bootstrapping: Let Your Data Be Your Guide Robin H. Lock Burry Professor of Statistics St. Lawrence University MAA Seaway Section Meeting Hamilton College,

Introducing Inference with Simulation Methods; Implementation at Duke University Kari Lock Morgan Department of Statistical Science, Duke University

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: p-value STAT 101 Dr. Kari Lock Morgan 9/25/12 SECTION 4.2 Randomization distribution.

Using Bootstrap Intervals and Randomization Tests to Enhance Conceptual Understanding in Introductory Statistics Kari Lock Morgan Department of Statistical.

Statistics: Unlocking the Power of Data Lock 5 Inference for Proportions STAT 250 Dr. Kari Lock Morgan Chapter 6.1, 6.2, 6.3, 6.7, 6.8, 6.9 Formulas for.

Introducing Inference with Bootstrap and Randomization Procedures Dennis Lock Statistics Education Meeting October 30,

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: Hypotheses STAT 101 Dr. Kari Lock Morgan SECTION 4.1 Statistical test Null and alternative.

Synthesis and Review 3/26/12 Multiple Comparisons Review of Concepts Review of Methods - Prezi Essential Synthesis 3 Professor Kari Lock Morgan Duke University.

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University

More Randomization Distributions, Connections

Understanding the P-value… Really! Kari Lock Morgan Department of Statistical Science, Duke University with Robin Lock, Patti Frazer.

Using Simulation Methods to Introduce Inference Kari Lock Morgan Duke University In collaboration with Robin Lock, Patti Frazer Lock, Eric Lock, Dennis.

Statistics: Unlocking the Power of Data Lock 5 Synthesis STAT 250 Dr. Kari Lock Morgan SECTIONS 4.4, 4.5 Connecting bootstrapping and randomization (4.4)

Using Lock5 Statistics: Unlocking the Power of Data

What Can We Do When Conditions Aren’t Met? Robin H. Lock, Burry Professor of Statistics St. Lawrence University BAPS at 2011 JSM Miami Beach, August 2011.

How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.

Statistics: Unlocking the Power of Data Lock 5 Afternoon Session Using Lock5 Statistics: Unlocking the Power of Data Patti Frazer Lock University of Kentucky.

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University

Statistics: Unlocking the Power of Data Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University University of Kentucky.

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 9/18/12 Confidence Intervals: Bootstrap Distribution SECTIONS 3.3, 3.4 Bootstrap.

Introducing Inference with Simulation Methods; Implementation at Duke University Kari Lock Morgan Department of Statistical Science, Duke University

Section 8.1 Estimating  When  is Known In this section, we develop techniques for estimating the population mean μ using sample data. We assume that.

Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2 Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse- Joint.

1 Chapter 10: Introduction to Inference. 2 Inference Inference is the statistical process by which we use information collected from a sample to infer.

Confidence Intervals: Bootstrap Distribution

Introducing Inference with Bootstrapping and Randomization Kari Lock Morgan Department of Statistical Science, Duke University with.

Implementing a Randomization-Based Curriculum for Introductory Statistics Robin H. Lock, Burry Professor of Statistics St. Lawrence University Breakout.

Statistics: Unlocking the Power of Data Lock 5 Bootstrap Intervals Dr. Kari Lock Morgan PSU /12/14.

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University Canton, New York.

Statistics: Unlocking the Power of Data Lock 5 Exam 2 Review STAT 101 Dr. Kari Lock Morgan 11/13/12 Review of Chapters 5-9.

Using Bootstrapping and Randomization to Introduce Statistical Inference Robin H. Lock, Burry Professor of Statistics Patti Frazer Lock, Cummings Professor.

Give your data the boot: What is bootstrapping? and Why does it matter? Patti Frazer Lock and Robin H. Lock St. Lawrence University MAA Seaway Section.

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 12/6/12 Synthesis Big Picture Essential Synthesis Bayesian Inference (continued)

Early Inference: Using Randomization to Introduce Hypothesis Tests Kari Lock, Harvard University Eric Lock, UNC Chapel Hill Dennis Lock, Iowa State Joint.

Statistics: Unlocking the Power of Data Lock 5 Inference for Means STAT 250 Dr. Kari Lock Morgan Sections 6.4, 6.5, 6.6, 6.10, 6.11, 6.12, 6.13 t-distribution.

Synthesis and Review 2/20/12 Hypothesis Tests: the big picture Randomization distributions Connecting intervals and tests Review of major topics Open Q+A.

StatKey Online Tools for Teaching a Modern Introductory Statistics Course Robin Lock Burry Professor of Statistics St. Lawrence University

Bootstraps and Scrambles: Letting a Dataset Speak for Itself Robin H. Lock Patti Frazer Lock ‘75 Burry Professor of Statistics Cummings Professor of MathematicsSt.

Today: Hypothesis testing p-value Example: Paul the Octopus In 2008, Paul the Octopus predicted 8 World Cup games, and predicted them all correctly Is.

SECTION 7.2 Estimating a Population Proportion. Where Have We Been?  In Chapters 2 and 3 we used “descriptive statistics”.  We summarized data using.

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock, Lock, and Lock Minicourse – Joint Mathematics.

Making computing skills part of learning introductory stats

Patti Frazer Lock Cummings Professor of Mathematics

Introducing Statistical Inference with Resampling Methods (Part 1)

When we free ourselves of desire,

Using Simulation Methods to Introduce Inference

Using Simulation Methods to Introduce Inference

Teaching with Simulation-Based Inference, for Beginners

Presentation transcript:

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock, Lock, and Lock MAA Minicourse – Joint Mathematics Meetings San Diego, CA January 2013

The Lock 5 Team Robin St. Lawrence Dennis Iowa State Eric Duke Kari Duke Patti St. Lawrence

Introductions: Name Institution

Schedule: Day 1 Wednesday, 1/9, 9:00 – 11:00 am 1. Introductions and Overview 2. Bootstrap Confidence Intervals What is a bootstrap distribution? How do we use bootstrap distributions to build understanding of confidence intervals? How do we assess student understanding when using this approach? 3. Getting Started on Randomization Tests What is a randomization distribution? How do we use randomization distributions to build understanding of p-values? 4. Minute Papers

Schedule: Day 2 Friday, 1/11, 9:00 – 11:00 am 5. More on Randomization Tests How do we generate randomization distributions for various statistical tests? How do we assess student understanding when using this approach? 6. Connecting Intervals and Tests 7. Connecting Simulation Methods to Traditional 8. Technology Options Brief software demonstration (Minitab, Fathom, R, Excel,...) – pick one! 9. Wrap-up How has this worked in the classroom? Participant comments and questions 10. Evaluations

Why use Randomization Methods?

These methods are great for teaching statistics… (the methods tie directly to the key ideas of statistical inference so help build conceptual understanding)

And these methods are becoming increasingly important for doing statistics.

It is the way of the past… "Actually, the statistician does not carry out this very simple and very tedious process [the randomization test], but his conclusions have no justification beyond the fact that they agree with those which could have been arrived at by this elementary method." -- Sir R. A. Fisher, 1936

… and the way of the future “... the consensus curriculum is still an unwitting prisoner of history. What we teach is largely the technical machinery of numerical approximations based on the normal distribution and its many subsidiary cogs. This machinery was once necessary, because the conceptually simpler alternative based on permutations was computationally beyond our reach. Before computers statisticians had no choice. These days we have no excuse. Randomization-based inference makes a direct connection between data production and the logic of inference that deserves to be at the core of every introductory course.” -- Professor George Cobb, 2007 (see full TISE article by Cobb in your binder)

Question Do you teach Intro Stat? A. Very regularly (most semesters) B. Regularly (most years) C. Occasionally D. Rarely (every few years) E. Never (or not yet)

Question How familiar are you with simulation methods such as bootstrap confidence intervals and randomization tests? A. Very B. Somewhat C. A little D. Not at all E. Never heard of them before!

Question Have you used randomization methods in Intro Stat? A. Yes, as a significant part of the course B. Yes, as a minor part of the course C. No D. What are randomization methods?

Question Have you used randomization methods in any statistics class that you teach? A. Yes, as a significant part of the course B. Yes, as a minor part of the course C. No D. What are randomization methods?

Intro Stat – Revise the Topics Descriptive Statistics – one and two samples Normal distributions Data production (samples/experiments) Sampling distributions (mean/proportion) Confidence intervals (means/proportions) Hypothesis tests (means/proportions) ANOVA for several means, Inference for regression, Chi-square tests Data production (samples/experiments) Bootstrap confidence intervals Randomization-based hypothesis tests Normal distributions Bootstrap confidence intervals Randomization-based hypothesis tests Descriptive Statistics – one and two samples

We need a snack!

What proportion of Reese’s Pieces are Orange? Find the proportion that are orange for your “sample”.

Proportion orange in 100 samples of size n=100 BUT – In practice, can we really take lots of samples from the same population?

Bootstrap Distributions Or: How do we get a sense of a sampling distribution when we only have ONE sample?

Suppose we have a random sample of 6 people:

Original Sample Create a “sampling distribution” using this as our simulated population

Bootstrap Sample: Sample with replacement from the original sample, using the same sample size. Original SampleBootstrap Sample

Simulated Reese’s Population Sample from this “population” Original Sample

Create a bootstrap sample by sampling with replacement from the original sample. Compute the relevant statistic for the bootstrap sample. Do this many times!! Gather the bootstrap statistics all together to form a bootstrap distribution.

Original Sample Bootstrap Sample ●●●●●● Bootstrap Statistic Sample Statistic Bootstrap Statistic ●●●●●● Bootstrap Distribution

Example: What is the average price of a used Mustang car? Select a random sample of n=25 Mustangs from a website (autotrader.com) and record the price (in $1,000’s) for each car.

Sample of Mustangs: Our best estimate for the average price of used Mustangs is $15,980, but how accurate is that estimate?

Original Sample Bootstrap Sample

We need technology! Introducing StatKey.

StatKey

Using the Bootstrap Distribution to Get a Confidence Interval – Method #1 The standard deviation of the bootstrap statistics estimates the standard error of the sample statistic. Quick interval estimate : For the mean Mustang prices:

Using the Bootstrap Distribution to Get a Confidence Interval – Method #2 Keep 95% in middle Chop 2.5% in each tail We are 95% sure that the mean price for Mustangs is between $11,930 and $20,238

Bootstrap Confidence Intervals Version 1 (Statistic  2 SE): Great preparation for moving to traditional methods Version 2 (Percentiles): Great at building understanding of confidence intervals

Playing with StatKey! See the purple pages in the folder.

Traditional Inference 1. Which formula? 2. Calculate summary stats 5. Plug and chug 3. Find t * 4. df? OR t * = Interpret in context CI for a mean 7. Check conditions

We want to collect some data from you. What should we ask you for our one quantitative question and our one categorical question?

What quantitative data should we collect from you? A.What was the class size of the Intro Stat course you taught most recently? B.How many years have you been teaching Intro Stat? C.What was the travel time, in hours, for your trip to Boston for JMM? D.Including this one, how many times have you attended the January JMM? E.???

What categorical data should we collect from you? A.Did you fly or drive to these meetings? B.Have you attended any previous JMM meetings? C.Have you ever attended a JSM meeting? D.??? E.???

Why does the bootstrap work?

Sampling Distribution Population µ BUT, in practice we don’t see the “tree” or all of the “seeds” – we only have ONE seed

Bootstrap Distribution Bootstrap “Population” What can we do with just one seed? Grow a NEW tree! µ

Golden Rule of Bootstraps The bootstrap statistics are to the original statistic as the original statistic is to the population parameter.

How do we assess student understanding of these methods (even on in-class exams without computers)? See the green pages in the folder.

Paul the Octopus

Paul the Octopus predicted 8 World Cup games, and predicted them all correctly Is this evidence that Paul actually has psychic powers? How unusual would this be if he were just randomly guessing (with a 50% chance of guessing correctly)? How could we figure this out? Paul the Octopus

Each coin flip = a guess between two teams Heads = correct, Tails = incorrect Flip a coin 8 times and count the number of heads. Remember this number! Did you get all 8 heads? (a) Yes (b) No Simulate!

Let p denote the proportion of games that Paul guesses correctly (of all games he may have predicted) H 0 : p = 1/2 H a : p > 1/2 Hypotheses

A randomization distribution is the distribution of sample statistics we would observe, just by random chance, if the null hypothesis were true A randomization distribution is created by simulating many samples, assuming H 0 is true, and calculating the sample statistic each time Randomization Distribution

Let’s create a randomization distribution for Paul the Octopus! On a piece of paper, set up an axis for a dotplot, going from 0 to 8 Create a randomization distribution using each other’s simulated statistics For more simulations, we use StatKey Randomization Distribution

The p-value is the probability of getting a statistic as extreme (or more extreme) as that observed, just by random chance, if the null hypothesis is true This can be calculated directly from the randomization distribution! p-value

StatKey

Create a randomization distribution by simulating assuming the null hypothesis is true The p-value is the proportion of simulated statistics as extreme as the original sample statistic Randomization Test

How do we create randomization distributions for other parameters? How do we assess student understanding? Connecting intervals and tests Connecting simulations to traditional methods Technology for using simulation methods Experiences in the classroom Coming Attractions - Friday