1. Gibbs Sampling A little bit of theory Outline: -What is Markov chain
- -When will it be stationary? Properties ….
-Gibbs is a Markov chain
-P(z) is the stationary
For structured learning like structured perceptron, know how to inference solve the problem
Inference for everyone when using MRF
Question
- Ergoic: true considition. What is not?
- Slice sampling (not sure to mention or not)

2. Gibbs Sampling
Gibbs sampling from a distribution P(z) ( z = {z1,…,zN} )
𝒛 𝟎 = 𝑧 1 0 , 𝑧 2 0 ,⋯, 𝑧 𝑁 0
For t = 1 to T:
𝑧 1 𝑡 ~𝑃 𝑧 1 | 𝑧 2 = 𝑧 2 𝑡−1 , 𝑧 3 = 𝑧 3 𝑡−1 , 𝑧 4 = 𝑧 4 𝑡−1 ,⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1
𝑧 2 𝑡 ~𝑃 𝑧 2 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 3 = 𝑧 3 𝑡−1 , 𝑧 4 = 𝑧 4 𝑡−1 ,⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1
𝑧 3 𝑡 ~𝑃 𝑧 3 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 2 = 𝑧 2 𝑡 , 𝑧 4 = 𝑧 4 𝑡−1 ,⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1 …
Markov Chain: a stochastic process in which future states are independent of past states given the present state
𝑧 𝑁 𝑡 ~𝑃 𝑧 𝑁 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 2 = 𝑧 2 𝑡 , 𝑧 3 = 𝑧 3 𝑡 ,⋯, 𝑧 𝑁−1 = 𝑧 𝑁−1 𝑡
Output: 𝒛 𝒕 = 𝑧 1 𝑡 , 𝑧 2 𝑡 ,⋯, 𝑧 𝑁 𝑡
𝒛 𝟏 , 𝒛 𝟐 , 𝒛 𝟑 ,…, 𝒛 𝑻
As sampling from P(z)
Why?

3. Markov Chain Three cities A, B and C 1/6 1/2 1/2 1/2 1/6 2/3 1/2
Three cities A, B and C C
1/6
1/2
1/2
1/2
1/6 A B
2/3
1/2
Each day he decided which city he would visit with the probability distribution depending on which city he was in
Random Walk
Web pages ? Use on webpages, page rank
The traveler recorded the cities he visited each day. …… A B C A A
This is a Markov chain
state

4. Markov Chain With sufficient samples …… A : B : C = 0.6 : 0.2 : 0.2
(independent of the starting city)
10000 days
10000 days
10000 days
5915
2025
2060
6016
2002
1982
5946
2016
2038
0.6
0.2

5. Markov Chain 0.2 0.2 0.6 The distribution will not change. P(A)=0.6
0.2 C
P(A)=0.6
1/6
1/2
P(B)=0.2
1/2
1/2
P(C)=0.2
0.6
0.2
1/6 A B
Stationary
Distribution
2/3
1/2
Web pages ? Use on webpages, page rank
2/3
0.6
1/2
0.2
1/2
0.2
0.6
𝑃 𝑇 𝐴 𝐴 𝑃 𝐴 + 𝑃 𝑇 𝐴 𝐵 𝑃 𝐵 + 𝑃 𝑇 𝐴 𝐶 𝑃 𝐶 =𝑃 𝐴
𝑃 𝑇 𝐵 𝐴 𝑃 𝐴 + 𝑃 𝑇 𝐵 𝐵 𝑃 𝐵 + 𝑃 𝑇 𝐵 𝐶 𝑃 𝐶 =𝑃 𝐵
𝑃 𝑇 𝐶 𝐴 𝑃 𝐴 + 𝑃 𝑇 𝐶 𝐵 𝑃 𝐵 + 𝑃 𝑇 𝐶 𝐶 𝑃 𝐶 =𝑃 𝐶
The distribution will not change.

6. Unique stationary distribution
Markov Chain
A Markov Chain can have multiple stationary distributions. C A B 1
Reaching which stationary distribution depends on starting state
The Markov Chain fulfill some conditions will have unique stationary distribution.
Web pages ? Use on webpages, page rank
Irreducible, appreriodic
PT(s’|s) for any states s and s’ is not zero
Unique stationary distribution
(sufficient but not necessary condition)

7. Markov Chain from Gibbs Sampling
Gibbs sampling from a distribution P(z) ( z = {z1,…,zN} )
𝒛 𝟎 = 𝑧 1 0 , 𝑧 2 0 ,⋯, 𝑧 𝑁 0
For t = 1 to T:
𝑧 1 𝑡 ~𝑃 𝑧 1 | 𝑧 2 = 𝑧 2 𝑡−1 , 𝑧 3 = 𝑧 3 𝑡−1 , 𝑧 4 = 𝑧 4 𝑡−1 ,⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1
𝑧 2 𝑡 ~𝑃 𝑧 2 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 3 = 𝑧 3 𝑡−1 , 𝑧 4 = 𝑧 4 𝑡−1 ,⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1
𝑧 𝑁 𝑡 ~𝑃 𝑧 𝑁 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 2 = 𝑧 2 𝑡 , 𝑧 3 = 𝑧 3 𝑡 ,⋯, 𝑧 𝑁−1 = 𝑧 𝑁−1 𝑡 …
Output: 𝒛 𝒕 = 𝑧 1 𝑡 , 𝑧 2 𝑡 ,⋯, 𝑧 𝑁 𝑡
𝑧 3 𝑡 ~𝑃 𝑧 3 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 2 = 𝑧 2 𝑡 , 𝑧 4 = 𝑧 4 𝑡−1 ,⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1
Markov Chain: a stochastic process in which future states are independent of past states given the present state
This is a Markov Chain
𝒛 𝟏 , 𝒛 𝟐 , 𝒛 𝟑 ,…, 𝒛 𝑻
zt only depend on zt-1
state

8. Markov Chain from Gibbs Sampling
Gibbs sampling from a distribution P(z) ( z = {z1,…,zN} )
𝒛 𝟎 = 𝑧 1 0 , 𝑧 2 0 ,⋯, 𝑧 𝑁 0
For t = 1 to T:
𝑧 1 𝑡 ~𝑃 𝑧 1 | 𝑧 2 = 𝑧 2 𝑡−1 , 𝑧 3 = 𝑧 3 𝑡−1 , 𝑧 4 = 𝑧 4 𝑡−1 ,⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1
𝑧 2 𝑡 ~𝑃 𝑧 2 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 3 = 𝑧 3 𝑡−1 , 𝑧 4 = 𝑧 4 𝑡−1 ,⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1
𝑧 𝑁 𝑡 ~𝑃 𝑧 𝑁 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 2 = 𝑧 2 𝑡 , 𝑧 3 = 𝑧 3 𝑡 ,⋯, 𝑧 𝑁−1 = 𝑧 𝑁−1 𝑡 …
Output: 𝒛 𝒕 = 𝑧 1 𝑡 , 𝑧 2 𝑡 ,⋯, 𝑧 𝑁 𝑡
𝑧 3 𝑡 ~𝑃 𝑧 3 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 2 = 𝑧 2 𝑡 , 𝑧 4 = 𝑧 4 𝑡−1 ,⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1
Markov Chain: a stochastic process in which future states are independent of past states given the present state
Proof that the Markov chain has unique stationary distribution which is P(z).
𝒛 𝟏 , 𝒛 𝟐 , 𝒛 𝟑 ,…, 𝒛 𝑻

9. Markov Chain from Gibbs Sampling
Markov chain from Gibbs sampling has unique stationary distribution?
𝑃 𝑇 𝑧 ′ |𝑧 >0, for any z and z’
Yes
𝑧 1 𝑡 ~𝑃 𝑧 1 | 𝑧 2 = 𝑧 2 𝑡−1 , 𝑧 3 = 𝑧 3 𝑡−1 , ⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1
None of the conditional probability is zero
𝑧 2 𝑡 ~𝑃 𝑧 2 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 3 = 𝑧 3 𝑡−1 , ⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1
𝑧 3 𝑡 ~𝑃 𝑧 3 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 2 = 𝑧 2 𝑡 , ⋯, 𝑧 𝑁 = 𝑧 𝑁 𝑡−1 …
𝑧 𝑁 𝑡 ~𝑃 𝑧 𝑁 | 𝑧 1 = 𝑧 1 𝑡 , 𝑧 2 = 𝑧 2 𝑡 , ⋯, 𝑧 𝑁−1 = 𝑧 𝑁−1 𝑡
can be any zt

10. Markov Chain from Gibbs Sampling
Show that P(z) is a stationary distribution
𝑧 𝑃 𝑇 𝑧 ′ |𝑧 𝑃 𝑧
=𝑃 𝑧 ′
𝑃 𝑇 𝑧 ′ |𝑧
=𝑃 𝑧 1 ′ | 𝑧 2 , 𝑧 3 , 𝑧 4 ,⋯, 𝑧 𝑁
×𝑃 𝑧 2 ′ | 𝑧 1 ′ , 𝑧 3 , 𝑧 4 ,⋯, 𝑧 𝑁
×𝑃 𝑧 3 ′ | 𝑧 1 ′ , 𝑧 2 ′ , 𝑧 4 ,⋯, 𝑧 𝑁
Can be proofed,
Random systematic sweep …
×𝑃 𝑧 𝑁 ′ | 𝑧 1 ′ , 𝑧 2 ′ , 𝑧 3 ′ ,⋯, 𝑧 𝑁−1 ′
There is only one stationary distribution for Gibbs sampling, so we are done.

11. Thank you for your attention!