1. General Gibbs Distribution
Representation
Probabilistic
Graphical
Models
Markov Networks
General Gibbs Distribution

3. Consider a fully connected pairwise Markov network over X1,…,Xn where each Xi has d values. How many parameters does the network have?
O(dn)
O(nd)
O(n2d2)
O(nd)

5. Gibbs Distribution Parameters: a1 b1 c1 0.25 c2 0.35 b2 0.08 0.16 a2
0.05
0.07 a3
0.15
0.21
0.09
0.18
Parameters:

6. Gibbs Distribution

7. Markov Network Representation
P factorizes over H

8. Separation in Undirected Graph H
A trail between X and Y is active given Z
X and Y are separated in H given Z if

9. Independence Assumptions in H
The independencies implied by H
I(H) =
We say that H is an I-map (independence map) of P if
Define I(G)

10. Factorization  Independence
Theorem: If P factorizes over H then H is an I-map for P

12. Independence  Factorization
Hammersley-Clifford Theorem: If H is an I-map for P, and P is positive, then P factorizes over H

13. Which parameterization of P factorizes over the graph H?D B C
All of the above

14. Graph Structure & Factorization
Factorization not unique, but same independencies

15. Summary
Gibbs distribution represents distribution as a product of factors
Associated Markov network connects every pair of nodes that are in the same factor
Can read independencies that must hold in P from Markov network separation
Markov network structure doesn’t fully specify the factorization of P

16. END END END

22. The Chain Rule for Bayesian Nets
Intelligence
Difficulty
Grade
Letter
SAT
0.3
0.08
0.25
0.4 g2
0.02
0.9
i1,d0
0.7
0.05
i0,d1
0.5 g1 g3
0.2
i1,d1
i0,d0 l1 l0
0.99
0.1
0.01
0.6
0.95 s0 s1
0.8 i1 i0 d1 d0
P(D,I,G,S,L) =
P(D) P(I) P(G | I,D) P(L | G) P(S | I)

23. Suppose q is at a local minimum of a function
Suppose q is at a local minimum of a function. What will one iteration of gradient descent do?
Leave q unchanged.
Change q in a random direction.
Move q towards the global minimum of J(q).
Decrease q.

24. Fig. A corresponds to a=0.01, Fig. B to a=0.1, Fig. C to a=1.