1. Reducing MCMC Computational Cost With a Two Layered Bayesian Approach
Ramin Madarshahian, Doctoral Candidate,
Juan M. Caicedo, Associate Professor

2. Outline
Introduction
Methodology
Example
Future work

3. Introduction

4. Markov Chain Monte Carlo (MCMC)
A very powerful algorithm for getting sample from high dimensional and complicated probability distribution function.
It gets more sample from high probability regions, and with enough number of samples, histogram of samples take similar shape as probability distribution of interest.

5. Markov Chain Monte Carlo (MCMC)
Von Neumann
he contributed to the development of the Monte Carlo method, which allowed solutions to complicated problems to be approximated using random numbers.
Metropolis
Ulam
Paper 1949: Using Markov chain for Monte Carlo approximation
Rossenburg
Teller
Paper 1953:
They applied MCMC for
a chemical problem

6. Bayesian modeling
Bayesian modeling is good method to deal with uncertainty.
Bayesian modeling update our belief about the model and its parameters by considering evidences.
Evidence comes from inputs and outputs.

7. Problem?
With a Bayesian model we would like to make an inference about a model and its parameters. MCMC can be used to sample the posterior, but for each sample we need to run the model.
What if our model is very
computational expensive?

8. Metamodeling
Metamodel: Simply an approximation of the computationally expensive model.
Also known as : Response surface, emulators, auxiliary models, etc.
Computationally expensive models: Models of multi-scale problems like shear band, models of complicated structures like airplane, modelling of physical and biological phenomena like protein folding, etc.

9. General approach Few numbers of input samples.
Using the expensive model to obtain corresponding output samples.
Using these I/O relationships to fit the metamodel.
For some methods, using obtained metamodel to select next input sample to better fit the metamodel.
Replacing the expensive model with obtained metamodel, and using this new model in the Bayesian process and etc.

10. Proposed method
In our proposed method, the surface of interest is obtained from the posterior of the Bayesian model, instead of direct approximation of the expensive model by the surrogate.

11. Motivation
A posterior is a probability distribution function with all common characteristics of that. It is usually more well-defined in comparison to the expensive model itself.
Depending upon the type of study, a researcher can focus on high probability regions (like model updating problems) or focus on tails (Reliability problems). This makes sampling more efficient.

12. Methodology

13. Bayesian modelling formulation
Model’s parameters
Data
Posterior
Likelihood

14. Proposed method formulation

15. Example

16. SDOF
Simulated by assuming normal distribution with 𝜇=1000 𝑁 𝑚 and 𝜎=10 𝑁 𝑚 for K
𝑀=100 𝑘𝑔 then what is 𝐾 using the data in the table?

17. Inference without using the metamodel
Assuming uniform prior for K from Zero to 2000:

18. Inference without using the metamodel
MCMC:
Selecting the prior:
Assuming uniform prior for K from Zero to 𝑁 𝑚 :
A total of samples are generated and the first 2000 are discarded.

19. Inference without using the metamodel
𝜇 𝑘 = 𝑁 𝑚
95% 𝐻𝑖𝑔ℎ𝑒𝑠𝑡 𝑃𝑜𝑠𝑡𝑒𝑟𝑖𝑜𝑟 𝐷𝑒𝑛𝑠𝑖𝑡𝑦 𝐻𝑃𝐷 = 642 ,1428 𝑁 𝑚
𝜎=203 𝑁 𝑚

20. Proposed method

21. Proposed method Priors:
For 𝑐 1 : a uniform distribution with the lower bound of and the upper bound of 1.0.
For 𝜇 1 : a uniform distribution with the lower bound of zero and the upper bound of 2000.
For 𝜎 1 : a normal distribution with 𝜇=300 𝑁 𝑚 , and 𝜎=80 𝑁 𝑚
A total of samples were obtained, 4000 of them were considered burning samples and were discarded.

22. Proposed method

23. Proposed method Parameter Mean 95% HPD 𝜇 1 1021 𝑁 𝑚 [1011,1031] 𝑁 𝑚
[1011,1031] 𝑁 𝑚
𝜎 1
201 𝑁 𝑚
[190,210] 𝑁 𝑚
𝑐 1
0.051
[0.049,0.053]

24. Comparison
Using obtained mean and standard deviation, 95% HPD will be [627, 1414] 𝑁 𝑚 which is comparable with results without the metamodel, i.e.[642, 1428] 𝑁 𝑚 .

25. Future work

26. Future work
Considering different types of metamodels like polynomial, kriging, etc.
Development of sampling strategies.
Considering expensive models and study of computational power of method.

27. Reducing MCMC Computational Cost With a Two Layered Bayesian Approach
Thank you!
Reducing MCMC Computational Cost With a Two Layered Bayesian Approach
Ramin Madarshahian, Doctoral Candidate,
Juan M. Caicedo, Associate Professor