Bayesian Statistics Without Tears: Prelude

Slides:



Advertisements
Similar presentations
Fundamentals of Bayesian Inference
Advertisements

J. Daunizeau Institute of Empirical Research in Economics, Zurich, Switzerland Brain and Spine Institute, Paris, France Bayesian inference.
Introduction to Monte Carlo Markov chain (MCMC) methods
Chapter 7 Hypothesis Testing
1 Uncertainty in rainfall-runoff simulations An introduction and review of different techniques M. Shafii, Dept. Of Hydrology, Feb
CHAPTER 15: Tests of Significance: The Basics Lecture PowerPoint Slides The Basic Practice of Statistics 6 th Edition Moore / Notz / Fligner.
Bayesian Hypothesis Testing In Nested Models Harold Jeffreys Jeff Rouder.
CHAPTER 13 Inference Techniques. Reasoning in Artificial Intelligence n Knowledge must be processed (reasoned with) n Computer program accesses knowledge.
Welcome to Amsterdam!. Bayesian Modeling for Cognitive Science: A WinBUGS Workshop.
Bayesian inference “Very much lies in the posterior distribution” Bayesian definition of sufficiency: A statistic T (x 1, …, x n ) is sufficient for 
CHAPTER 21 Inferential Statistical Analysis. Understanding probability The idea of probability is central to inferential statistics. It means the chance.
Chapter 8: Estimating with Confidence
1 1 Slide © 2008 Thomson South-Western. All Rights Reserved Chapter 9 Hypothesis Testing Developing Null and Alternative Hypotheses Developing Null and.
1-1 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 18, Slide 1 Chapter 18 Confidence Intervals for Proportions.
Bayesian statistics – MCMC techniques
Bayesian inference Gil McVean, Department of Statistics Monday 17 th November 2008.
Maximum Likelihood. Likelihood The likelihood is the probability of the data given the model.
Introduction  Bayesian methods are becoming very important in the cognitive sciences  Bayesian statistics is a framework for doing inference, in a principled.
Bayesian estimation Bayes’s theorem: prior, likelihood, posterior
1 The Basics of Regression Regression is a statistical technique that can ultimately be used for forecasting.
Using ranking and DCE data to value health states on the QALY scale using conventional and Bayesian methods Theresa Cain.
Copyright © 2010 Pearson Education, Inc. Chapter 19 Confidence Intervals for Proportions.
Applied Bayesian Analysis for the Social Sciences Philip Pendergast Computing and Research Services Department of Sociology
Monté Carlo Simulation MGS 3100 – Chapter 9. Simulation Defined A computer-based model used to run experiments on a real system.  Typically done on a.
A Practical Course in Graphical Bayesian Modeling; Class 1 Eric-Jan Wagenmakers.
Bayesian Inference Using JASP
Chapter 8 Hypothesis testing 1. ▪Along with estimation, hypothesis testing is one of the major fields of statistical inference ▪In estimation, we: –don’t.
Introduction to MCMC and BUGS. Computational problems More parameters -> even more parameter combinations Exact computation and grid approximation become.
Essential Statistics Chapter 131 Introduction to Inference.
INTRODUCTION TO INFERENCE BPS - 5th Ed. Chapter 14 1.
CHAPTER 14 Introduction to Inference BPS - 5TH ED.CHAPTER 14 1.
10.2 Tests of Significance Use confidence intervals when the goal is to estimate the population parameter If the goal is to.
Agresti/Franklin Statistics, 1 of 122 Chapter 8 Statistical inference: Significance Tests About Hypotheses Learn …. To use an inferential method called.
1 Chapter 10: Introduction to Inference. 2 Inference Inference is the statistical process by which we use information collected from a sample to infer.
© 2013 Pearson Education, Inc. Active Learning Lecture Slides For use with Classroom Response Systems Introductory Statistics: Exploring the World through.
Not in FPP Bayesian Statistics. The Frequentist paradigm Defines probability as a long-run frequency independent, identical trials Looks at parameters.
Chapter 11 Statistical Techniques. Data Warehouse and Data Mining Chapter 11 2 Chapter Objectives  Understand when linear regression is an appropriate.
Probability Course web page: vision.cis.udel.edu/cv March 19, 2003  Lecture 15.
CHAPTER 9 Testing a Claim
1 CHAPTER 4 CHAPTER 4 WHAT IS A CONFIDENCE INTERVAL? WHAT IS A CONFIDENCE INTERVAL? confidence interval A confidence interval estimates a population parameter.
MATH 643 Bayesian Statistics. 2 Discrete Case n There are 3 suspects in a murder case –Based on available information, the police think the following.
26134 Business Statistics Tutorial 12: REVISION THRESHOLD CONCEPT 5 (TH5): Theoretical foundation of statistical inference:
Statistical Inference Statistical inference is concerned with the use of sample data to make inferences about unknown population parameters. For example,
Math 3680 Lecture #15 Confidence Intervals. Review: Suppose that E(X) =  and SD(X) = . Recall the following two facts about the average of n observations.
G. Cowan Lectures on Statistical Data Analysis Lecture 4 page 1 Lecture 4 1 Probability (90 min.) Definition, Bayes’ theorem, probability densities and.
1 Chapter 8: Model Inference and Averaging Presented by Hui Fang.
Statistical Methods. 2 Concepts and Notations Sample unit – the basic landscape unit at which we wish to establish the presence/absence of the species.
Inferential statistics by example Maarten Buis Monday 2 January 2005.
1-1 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 23, Slide 1 Chapter 24 Comparing Means.
G. Cowan Lectures on Statistical Data Analysis Lecture 12 page 1 Statistical Data Analysis: Lecture 12 1Probability, Bayes’ theorem 2Random variables and.
The Practice of Statistics, 5th Edition Starnes, Tabor, Yates, Moore Bedford Freeman Worth Publishers CHAPTER 9 Testing a Claim 9.1 Significance Tests:
Outline Historical note about Bayes’ rule Bayesian updating for probability density functions –Salary offer estimate Coin trials example Reading material:
The Practice of Statistics, 5th Edition Starnes, Tabor, Yates, Moore Bedford Freeman Worth Publishers CHAPTER 9 Testing a Claim 9.1 Significance Tests:
Lecture Slides Elementary Statistics Twelfth Edition
MCMC Output & Metropolis-Hastings Algorithm Part I
Chapter 8: Inferences Based on a Single Sample: Tests of Hypotheses
Unit 5: Hypothesis Testing
Bayesian data analysis
Warm Up Check your understanding p. 541
CHAPTER 9 Testing a Claim
CHAPTER 9 Testing a Claim
Essential Statistics Introduction to Inference
CHAPTER 9 Testing a Claim
PSY 626: Bayesian Statistics for Psychological Science
CHAPTER 9 Testing a Claim
Basic Practice of Statistics - 3rd Edition Introduction to Inference
CHAPTER 9 Testing a Claim
CHAPTER 9 Testing a Claim
Bayesian Statistics on a Shoestring Assaf Oron, May 2008
CHAPTER 9 Testing a Claim
Presentation transcript:

Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Three Schools of Statistical Inference Neyman-Pearson: α-level, power calculations, two hypotheses, guide for action (i.e., what to do). Fisher: p-values, one hypothesis (i.e., H0), quantifies evidence against H0. Bayes: prior and posterior distributions, attaches probabilities to parameters and hypotheses.

A Freudian Analogy Neyman-Pearson: The Superego. Fisher: The Ego. Bayes: The Id. Claim: What Id really wants is to attach probabilities to hypotheses and parameters. This wish is suppressed by the Superego and the Ego. The result is unconscious internal conflict.

Internal Conflict Causes Misinterpretations p < .05 means that H0 is unlikely to be true, and can be rejected. p > .10 means that H0 is likely to be true. For a given parameter μ, a 95% confidence interval from, say, a to b means that there is a 95% chance that μ lies in between a and b.

Two Ways to Resolve the Internal Conflict Strengthen Superego and Ego by teaching the standard statistical methodology more rigorously. Suppress Id even more! Give Id what it wants.

What is Bayesian Inference? Why be Bayesian? Eric-Jan Wagenmakers

What is Bayesian Inference?

What is Bayesian Inference? “Common sense expressed in numbers”

What is Bayesian Inference? “The means by which rational agents draw optimal conclusions in an uncertain environment”

What is Bayesian Inference? “The only statistical procedure that is coherent, meaning that it avoids statements that are internally inconsistent.”

What is Bayesian Inference? “A method for rational updating of beliefs about the world”

What is Bayesian Inference? “The only good statistics”

Outline Bayes in a Nutshell The Inevitability of Probability Bayesian Revolutions This Course

Bayesian Inference in a Nutshell In Bayesian inference, uncertainty or degree of belief is quantified by probability. Prior beliefs are updated by means of the data to yield posterior beliefs.

Bayesian Parameter Estimation: Example We prepare for you a series of 10 factual true/false questions of equal difficulty. You answer 9 out of 10 questions correctly. What is your latent probability θ of answering any one question correctly?

Bayesian Parameter Estimation: Example We start with a prior distribution for θ. This reflect all we know about θ prior to the experiment. Here we make a standard choice and assume that all values of θ are equally likely a priori.

Bayesian Parameter Estimation: Example We then update the prior distribution by means of the data (technically, the likelihood) to arrive at a posterior distribution. The posterior distribution is a compromise between what we knew before the experiment and what we have learned from the experiment. The posterior distribution reflects all that we know about θ.

Mode = 0.9 95% confidence interval: (0.59, 0.98) NB. We do not have to use the uniform prior!

Outline Bayes in a Nutshell The Inevitability of Probability Bayesian Revolutions This Course

The Inevitability of Probability Why would one measure “degree of belief” by means of probability? Couldn’t we choose something else that makes sense? Yes, perhaps we can, but the choice of probability is anything but ad-hoc.

The Inevitability of Probability Assume “degree of belief” can be measured by a single number. Assume you are rational, that is, not self-contradictory or “obviously silly”. Then degree of belief can be shown to follow the same rules as the probability calculus.

The Inevitability of Probability For instance, a rational agent would not hold intransitive beliefs, such as:

The Inevitability of Probability When you use a single number to measure uncertainty or quantify evidence, and these numbers do not follow the rules of probability calculus, you can (almost certainly?) be shown to be silly or incoherent. One of the theoretical attractions of the Bayesian paradigm is that it ensures coherence right from the start.

Coherence I Coherence is also key in de Finetti’s conceptualization of probability.

Coherence II One aspect of coherence is that “today’s posterior is tomorrow’s prior”. Suppose we have exchangeable (iid) data x = {x1, x2}. Now we can update our prior using x, using first x1 and then x2, or using first x2 and then x1. All the procedures will result in exactly the same posterior distribution.

Coherence III Assume we have three models: M1, M2, M3. After seeing the data, suppose that M1 is 3 times more plausible than M2, and M2 is 4 times more plausible than M3. By transitivity, M1 is 3x4=12 times more plausible than M3.

Outline Bayes in a Nutshell The Inevitability of Probability Bayesian Revolutions This Course

The Bayesian Revolution Until about 1990, Bayesian statistics could only be applied to a select subset of very simple models. Only recently, Bayesian statistics has undergone a transformation; With current numerical techniques, Bayesian models are “limited only by the user’s imagination.”

The Bayesian Revolution in Statistics

The Bayesian Revolution in Statistics

The Bayesian Revolution in Psychology?

Are Psychologists Inconsistent? The content of Psych Review shows that Psychologists are happy to develop Bayesian models for human cognition and human behavior based on the assumption that agents or people process noisy information in a rational or optimal way; But psychologist do not use Bayesian models to analyze their own data statistically!

Why Bayes is Now Popular Markov chain Monte Carlo!

Markov Chain Monte Carlo Instead of calculating the posterior analytically, numerical techniques such as MCMC approximate the posterior by drawing samples from it. Consider again our earlier example…

Mode = 0.89 95% confidence interval: (0.59, 0.98) With 9000 samples, almost identical to analytical result.

Want to Know More About MCMC?

MCMC With MCMC, the models you can build and estimate are said to be “limited only by the user’s imagination”. But how do you get MCMC to work? Option 1: write the code it yourself. Option 2: use WinBUGS!

Outline Bayes in a Nutshell The Inevitability of Probability Bayesian Revolutions This Course

A Workshop in Bayesian Modeling for Cognitive Science Eric-Jan Wagenmakers

The Bayesian Book …is a course book used at UvA and UCI. …is still regularly updated. ….is freely available at my homepage, at http://www.ejwagenmakers.com/BayesCourse/BayesBook.html …greatly benefits from your suggestions for improvement! [e.g., typos, awkward sentences, etc.]

Contributors Michael Lee http://www.socsci.uci.edu/~mdlee/

Contributors Dora Matzke

Contributors Ruud Wetzels http://www.ruudwetzels.com/

Why We Like Graphical Bayesian Modeling It is fun. It is cool. It is easy. It is principled. It is superior. It is useful. It is flexible.

Our Goals These Weeks Are… For you to experience some of the possibilities that WinBUGS has to offer. For you to get some hands-on training by trying out some programs. For you to work at your own pace. For you to get answers to questions when you get stuck.

Our Goals These Weeks Are NOT… For you become a Bayesian graphical modeling expert in one week. For you to gain deep insight in the statistical foundations of Bayesian inference. For you to get frustrated when the programs do not work or you do not understand the materials (please ask questions).

Want to Know More About Bayes?

Want to Know More About Bayes?

Bayesian inference Using Gibbs Sampling WinBUGS Bayesian inference Using Gibbs Sampling You want to have this installed (plus the registration key)

WinBUGS Knows many probability distributions (likelihoods); Allows you to specify a model; Allows you to specify priors; Will then automatically run the MCMC sampling routines and produce output.

Want to Know More About WinBUGS?

WinBUGS & R WinBUGS produces MCMC samples. We want to analyze the output in a nice program, such as R. This can be accomplished using the R package “R2WinBUGS”

R: “Here’s the data and a bunch of commands” WinBUGS: “OK, I did what you wanted, here’s the samples you asked for”

Getting Started Work through some of the exercises of the book. Most of you will want to get started with the chapter “getting started”.

Running the R programs The R scripts have extension .R. You can use “File” -> “Open Script” to read these. You can run these scripts by copying-and-pasting the scripts in the R console.

Course Webpage Check out http://www.ejwagenmakers.com/BayesCourse/BayesCourse.html for lectures and a pdf file with answers to the exercises!

Questions? Feel free to ask questions when you are really stuck.