Outline Historical note about Bayes’ rule Bayesian updating for probability density functions –Salary offer estimate Coin trials example Reading material: –Gelman, Andrew, et al. Bayesian data analysis. CRC press, 2003, Chapter 1. Slides based in part on lecture by Prof. Joo-Ho Choi of Korea Aerospace University

Historical Note Birth of Bayesian –Rev. Thomas Bayes proposed Bayes’ theory (1763):  of Binomial dist. is estimated using observed data. Laplace discovered, put his name (1812), generalized to many prob’s. –For more 100 years, Bayesian “degree of belief” was rejected as vague and subjective. Objective “frequency” was accepted in statistics. –Jeffreys (1939) rediscovered, made modern theory (1961). Until 80s, still limited due to requirement for computation. Flourishing of Bayesian –From 1990, rapid advance of HW & SW, made it practical. –Bayesian technique applied to areas of science (economics, medical) & engineering., 1999

Bayesian Probability What is Bayesian probability ? –Classical: relative frequency of an event, given many repeated trials (e.g., probability of throwing 10 with pair of dice) –Bayesian: degree of belief that it is true based on evidence at hand Saturn mass estimation –Classical: mass is fixed but unknown. –Bayesian: mass described probabilistically based on observations (e.g, uniformly in interval (a,b).

Bayes rule for pdf’s θ is a probability density to estimate based on data y. Conditional probability density functions Leading to Bayes’ rule: Often written as L used because p(y| θ ) is called the likelihood function. Instead of dividing by p(y) can divide by area under curve.

Bayesian updating –The process schematically Updated prior PDF Observed data added Prior distribution Likelihood function Observed data Posterior distribution

Salary estimate example You are considering an engineering position for which salary offers θ (in thousand dollars)have recently followed the triangular distribution Your friend received a $93K offer for a similar position, and you know that their range of offers for such positions is no more than $5K. Before your friend’s data, what was your chance of an offer <$93K ? Estimate the distribution of the expected offer and the likeliest value.

Self evaluation question What value of salary offer to your friend would leave you with the least uncertainty about your own expected offer?

Coin Trials Example Problem –For a weighted (uneven) coin, probability of heads is to be determined based on the experiments. –Assume the true θ is 0.78, obtained after ∞ trials. But we don’t know this. Only infer based on experiments. Bayesian parameter estimation This is the parameter  to be estimated. Experiment data: x times out of n trials. 4 out of 5 trials 78 out of 100 trials Experiment data: x times out of n trials. 4 out of 5 trials 78 out of 100 trials Prior knowledge on p 0 (  ) 1. No prior information 2. Normal dist centered at 0.5 with  = Uniform distribution [0.5, 0.7] Prior knowledge on p 0 (  ) 1. No prior information 2. Normal dist centered at 0.5 with  = Uniform distribution [0.5, 0.7]

Probability of heads posterior distributions Prior 1.No prior (uniform) 2.N(0.5,0.05), poor prior slows convergence. 3.U(0.5,0.7) cannot exceed barrier due to incorrect prior figure Red: prior Wide: 4 out of 5 Narrow; 78 out of 100

Probability of 5 consecutive heads Prediction using posterior (no prior case) Exact value is binom(5,5,0.78) = = Posterior PDF of  Draw random samples of  from PDF Posterior prediction process Compute p based on each  binom(5,5,  ) Compute p based on each  binom(5,5,  ) median5% CI95% CI Estimation process 10,000 samples of  10,000 samples of predicted p

practice problems 1.For the salary estimate problem, what is the probability of getting a better offer than your friend? 2.For the salary problem, calculate the 95% confidence bounds on your salary around the mean and median of your expected salary distribution. 3. Slide 9 shows the risks associated with using a prior. When is it important to use a prior? Source: Smithsonian Institution Number: