Presentation is loading. Please wait.

Presentation is loading. Please wait.

- 1 - Bayesian inference of normal distribution Problem statement –Objective is to estimate unknown two parameters    of normal distribution based.

Similar presentations


Presentation on theme: "- 1 - Bayesian inference of normal distribution Problem statement –Objective is to estimate unknown two parameters    of normal distribution based."— Presentation transcript:

1 - 1 - Bayesian inference of normal distribution Problem statement –Objective is to estimate unknown two parameters    of normal distribution based on observations y = {y 1, y 2, …}. Prior –Joint pdf of non-informative prior Joint posterior distribution This is function of two variables, which can be plotted as a surface or contour.

2 - 2 - Joint posterior distribution Distribution –There is no inherent pdf function by matlab in this case. This is function of two variables, which can be plotted as a surface or contour. –Let’s consider a case with n=20; y ̄ =2.9; s=0.2; Remark –Analysis of posterior pdf: mean, median & confidence bounds. Marginal distribution Once we have the marginal pdf, we can evaluate its mean and confidence bounds. –Posterior prediction: predictive distribution of new y based on observed y. This is posterior pdf of  2 This is likelihood of new y

3 - 3 - Analytical solutions –Marginal mean –Marginal variance How to evaluate marginal distributions –Before doing this, note that in normal dist., variable y can be transformed to standard normal variable z. –Marginal mean & variance can be evaluated in the same manner. –One can evaluate various characteristics using the matlab functions. Marginal distributions

4 - 4 - Marginal distributions Supplementary information for normal pdf –The original pdf f Y (y) and normalized pdf f Z (z) have following relation. –Therefore, –If we want use f(z) instead of f(y), use this relation. Supplementary information for chi2 pdf –The original pdf f Y (y) and normalized pdf f Z (z) have following relation. –Therefore,

5 - 5 - Posterior prediction Predictive distribution of new y based on observed y Analytical solution –One can evaluate characteristics of this using the matlab function. –Compare with the marginal mean. Marginal mean: t-distribution with location y ̄ and variance s^2/n with n-1 dof. Predictive distribution: t-distribution with location y ̄ and variance s^2*(1+1/n) with n-1 dof. posterior pdf of  2 Likelihood of new y

6 - 6 - Simulation of joint posterior distribution Why ? –Even when analytic solutions available, some are not easy to evaluate. –Once exercised, may find it more convenient and more general. –Practice simulation & validate with analytic solution. Factorization approach –Review: joint probability of A & B –Likewise, joint posterior pdf of  2 The two at the right are marginal pdfs. Simulation by drawing random samples –We can evaluate characteristics of joint posterior pdf using simulation techniques.

7 - 7 - Simulation of joint posterior distribution Approach 1: use marginal variance. –Conditional pdf of  on  2 is already derived, which is the pdf of mean with known variance. (page 10 of Lec. #4) –Marginal pdf of  2 is given in this lecture. –In order to sample the posterior pdf of p(  2 |y) 1.Draw  2 from the marginal pdf 2.Draw  from the conditional pdf Marginal pdf of  2 Conditional pdf of  on  2

8 - 8 - Simulation of joint posterior distribution Approach 1: use marginal variance. –Once you have obtained samples of joint pdf, compare (validate) the results with the analytic solution. 1.Compare the samples of (M,S2) with the analytic joint pdf. In terms of scattered pot & contour. In terms of 3-D histogram & surface (mesh). 2.Compare the samples of M with the marginal pdf, which is t distribution. 3.Compare the samples of S2 with the marginal pdf which is inv-chi2 distribution. 4.Extract features of the samples M and compare with analytic solution. 5.Extract features of the samples S2 and compare with analytic solution.

9 - 9 - Simulation of joint posterior distribution Approach 2: use marginal mean. –Conditional pdf of  2 on  is already derived, which is the pdf of variance with known mean. (page 13 of Lec. #4) –Marginal pdf of  is given in this lecture. –In order to sample the posterior pdf of p(  2 |y) 1.Draw  from the marginal pdf 2.Draw  2 from the conditional pdf –Compare (validate) the resulting samples with the analytic solution. Marginal pdf of  Conditional pdf of  2 on 

10 Simulation of joint posterior distribution Approach 2: use marginal mean. –Once you have obtained samples of joint pdf, compare (validate) the results with the analytic solution. –Details are omitted for limited time.

11 Posterior prediction by simulation Analytic approach –Result of analytic solution by double integral over infinity to get Simulation –In practice, posterior predictive distribution is obtained by random draws. –Once we have posterior distribution for   in the form of samples, the predictive new y are easily obtained by drawing each one from conditional on each individual  . –Mean & conf. intervals of posterior prediction can be obtained easily.

12 Homework

13 Homework

14 Homework Problems 1 1)Use the approach 1 to obtain samples of joint pdf (M,S2). 2)Compare scattered dots with contour of analytic solution. Compare 3-D histogram with mesh shape of analytic solution. 3)Compare the samples of M with the marginal pdf, which is t distribution. 4)Compare the samples of S2 with the marginal pdf which is inv-chi2 dist. 5)Obtain mean, 95% conf. interval of M, & compare with analytic solution. 6)Obtain mean, 95% conf. interval of S2, & compare with analytic solution. 7)Obtain samples of posterior prediction, and obtain mean, 95% conf. interval of ynew & compare with analytic solution.


Download ppt "- 1 - Bayesian inference of normal distribution Problem statement –Objective is to estimate unknown two parameters    of normal distribution based."

Similar presentations


Ads by Google