1. Daphne Koller Message Passing Belief Propagation Algorithm Probabilistic Graphical Models Inference

2. Daphne Koller BD C A 2: B,C 1: A,B 3: C,D 4: A,D C A B D Cluster Graph

3. Daphne Koller Passing Messages 2: B,C 1: A,B 3: C,D 4: A,D C A B D  2,1 = 1  1 (A,B)  4 (A,D)  2 (B,C)  3 (C,D)  1,4 =  B  2,1 (B)   (A,B)

4. Daphne Koller Passing Messages 2: B,C 1: A,B 3: C,D 4: A,D C A B D  1,2 =  2,1 =  2,3 =  3,2 =  3,4 =  4,3 =  1,4 =  4,1 =  B  2,1 (B)   (A,B)  D  3,4 (D)   (A,D)  C  2,3 (C)   (C,D)  A  1,4 (A)   (A,D)  B  1,2 (B)   (B,C)  D  4,3 (D)   (C,D)  A  4,1 (A)   (A,B)  C  3,2 (C)   (B,C)

5. Daphne Koller Cluster Graphs Undirected graph such that: – nodes are clusters C i  {X 1,…,X n } – edge between C i and C j associated with sepset S i,j  C i  C j Given set of factors , we assign each  k to a cluster C  (k) s.t. Scope[  k ]  C  (k) Define

6. Daphne Koller Example Cluster Graph 1: A, B, C 3: B,D,F 2: B, C, D5: D, E 4: B, E B B B C D D E  1 =  1  2 =  2  3  6  4 =  5  5 =  4  3 =  7

7. Daphne Koller Different Cluster Graph 1: A, B, C 3: B,D,F 2: B, C, D5: D, E 4: B, E B B B,C D D E

8. Daphne Koller Message Passing 1: A, B, C 3: B,D,F 2: B, C, D5: D, E 4: B, E B B B C D D E

9. Daphne Koller Belief Propagation Algorithm Assign each factor  k  to a cluster C  (k) Construct initial potentials Initialize all messages to be 1 Repeat – Select edge (i,j) and pass message Compute

10. Daphne Koller Belief Propagation Run 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0 5 10 15 20 Iteration # P(a 1 ) True posterior

11. Daphne Koller Different BP Run

12. Daphne Koller Summary Graph of clusters connected by sepsets Adjacent clusters pass information to each other about variables in sepset – Message from i to j summarizes everything i knows, except information obtained from j Algorithm may not converge The resulting beliefs are pseudo-marginals Nevertheless, very useful in practice

13. Daphne Koller END END END