1. Graduate School of Information Sciences, Tohoku University
Physical Fluctuomatics Applied Stochastic Process 8th Markov chain Monte Carlo methods
Kazuyuki Tanaka
Graduate School of Information Sciences, Tohoku University
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

2. Horizon of Computation in Probabilistic Information Processing
Compensation of expressing uncertainty
using probability and statistics
It must be calculated by taking account of both events with high probability and events with low probability.
Computational Complexity
It is expected to break throw the computational complexity by introducing approximation algorithms.
Physical Fuctuomatics (Tohoku University)

3. What is an important point in computational complexity?
How should we treat the calculation of the summation over 2N configuration?
If it takes 1 second in the case of N=10, it takes 17 minutes in N=20, 12 days in N=30 and 34 years in N=40.
N fold loops
Markov Chain Monte Carlo Metod
Belief Propagation Method
This Talk
Next Talk
Physical Fuctuomatics (Tohoku University)

4. Generate uniform random numbers a and b in the interval [-1,1]
Calculation of the ratio of the circumference of a circle to its diameter by using random numbers (Monte Carlo Method) 1
n0 m0 -1 1
nn+1 -1
Generate uniform random numbers a and b in the interval [-1,1]
Count the number of points inside of the unit circle after plotting points randomly
a2+b2≤1 No
the ratio of the circumference of a circle to its diameter
Yes
mm+1
Accuracy is improved as the number of trials increases
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

5. Physics Fluctuomatics/Applied Stochastic Process (Tohoku University)
Law of Large Numbers
Let us suppose that random variables X1,X2,...,Xn are identical and mutual independent random variables with average m. Then we have
Central Limit Theorem
We consider a sequence of independent, identical distributed random variables, {X1,X2,...,Xn}, with average m and variance s2. Then the distribution of the random variable
tends to the Gauss distribution with average m and variance s2/n as n+.
Physics Fluctuomatics/Applied Stochastic Process　(Tohoku University)

6. Calculation of the ratio of the circumference of a circle to its diameter by using random numbers (Monte Carlo Method) 1
x- and y- coordinates of each random points is the average 0 and the variance ½. -1 1
From the central limit theorem, the sample average and the sample variance are 0 and 1/2n for n random points.
The width of probability density function decreases by according to 1/n1/2 as the number of points, n, increases. -1
Count the number of points m inside of the unit circle after plotting points n randomly
the ratio of the circumference of a circle to its diameter
Order of the error of the ratio of the circumference of a circle to its diameter is O(1/n1/2)
Accuracy is improved as the number of trials increases
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

7. Integral Computation by Monte Carlo Method1
n0 m0
nn+1 -1 1
Generate uniform random numbers a and b in the interval [-1,1] -1
Compute the value of f(x,y) at each point (x,y) after plotting points n inside of the green region randomly
mm + f(a,b)
Accuracy is O(1/n1/2)
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

8. Marginal Probability Marginal Probability
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

9. Important Point of Computations
Can we make an algorithm to generate |V| random vectors (x1,x2,…,x|V|) which are independent of each other?
The random numbers should be according to
Computational time generating one random numbers should be order of |V|.
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

10. Fundamental Stochastic Process: Markov Process
Transition Probability w(x|y)≥0 (x,y=0,1)
For any initial distribution P0(x),
Transition Matrix
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

11. Fundamental Stichastic Process: Markov Chain
Transition matrix can be diagonalized as
Limit Distribution
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

12. Fundamental Stochastic Process: Markov Process
Stationary Distribution or Equilibrium Distribution
In the Markov process, if there exists one unique limiting distribution,
it is an equilibrium distribution.
Even if there exists one equilibrium distribution, it is not always a limiting distribution.
Example
The stationary distribution is
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

13. Stationary Process and Detailed Balance in Markov Process
where
P1(x), P2(x), P3(x),…: Markov Chain
Detailed Balance
When the transition probability w(x|y) is chosen so as to satisfy the detailed balance, the Markov process provide us a stationary distribution P(x).
Stationary Distribution of Markov Process
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

14. Markov Chain Monte Carlo Method
Let us consider a joint probability distribution P(x1,x2,…,xL)
How to find the transition probability w(x|y) so as to satisfy
where
P1(x), P2(x), P3(x),…: Markov Process
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

15. Markov Chain Monte Carlo Method
Randomly generated
For sufficient large t, x[t], x[2t], x[3t], …, x[Nt] are independent of each other
Reject
How large number t ?
t: relaxation time
They can be regarded as samples from the given probability distribution P(x).
Accuracy O(1/t1/2)
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

16. Markov Chain Monte Carlo Method
Histgram Xi
Marginal Probability Distribution
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

17. Markov Chain Monte Carlo Method
V：Set of all the nodes
E：Set of all the neighbouring pairs of nodes
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

18. Markov Chain Monte Carlo Method
Markov Random Field
E：Set of all the neighbouring pairs of nodes
∂i：Set of all the neighbouring nodes of the node i
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

19. Markov Chain Monte Carlo Method
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

20. Markov Chain Monte Carlo Methodx
True
False
xi = ○ or ● x’
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

21. Sampling by Markov Chain Monte Carlo Method
Small p
Large p
Sampling by Markov Chain Monte Carlo Method
Disordered State
Ordered State
More is different.
Near Critical Point of p
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

22. Summary
Calculation of the ratio of the circumference of a circle to its diameter by using random numbers
Law of Large Numbers and Central Limit Theorem
Markov Chain Monte Carlo Method
Future Talks
9th Belief propagation
10th Probabilistic image processing by means of physical models
11th Bayesian network and belief propagation in statistical inference
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

23. Practice 8-1
When the probability distribution P(x) and the transition probability w(x’|x) satisfy the detailed balance
where
, prove that
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

24. Find the limit distribution defined by , where
Practice 8-2
Let us consider that the transition matrix of the present stochastic process is given as
Find the limit distribution defined by
, where
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)

25. Practice 8-3
Let us consider an undirected square grid graph with L=Lx×Ly nodes. The set of all the nodes is denoted by V={1,2,…,L} and the set of all the neighbouring pairs of nodes is denoted by E. A random variable Fi is assigned at each node i and takes every integer in the set {0,1,2,…,Q-1} . The joint probability distribution of the provability vector F=(F1,F2,…,FL)T is given as
Make a program which generate N mutual independent random vectors (f1,f2,…,fL)T randomly from the above joint probability distribution Pr{F1=f1,F2=f2,…,FL=fL} . For various values of positive numbers a, give numerical experiments.
Example of generated random vector in the case of Q=256, a=0.0005
Example of generated random vector in the case of Q=2, a=2
L=Lx×Ly
Physical Fluctuomatics / Applied Stochastic Process (Tohoku University)