Presentation is loading. Please wait.

Presentation is loading. Please wait.

Approximating the Partition Function by Deleting and then Correcting for Model Edges Arthur Choi and Adnan Darwiche University of California, Los Angeles.

Published byMorgan French Modified over 5 years ago

Similar presentations

Presentation on theme: "Approximating the Partition Function by Deleting and then Correcting for Model Edges Arthur Choi and Adnan Darwiche University of California, Los Angeles."— Presentation transcript:

1 Approximating the Partition Function by Deleting and then Correcting for Model Edges
Arthur Choi and Adnan Darwiche University of California, Los Angeles

2 Edge Deletion: Idea Delete an edge Model M Model M'

3 Deleting an Equivalence Edge
j

4 Deleting an Equivalence Edge
j j

5 Deleting an Equivalence Edge
j

6 Deleting an Equivalence Edge
j

7 Deleting an Equivalence Edge
j

8 Deleting an Equivalence Edge
j

9 Edge Parameters: ED-BP
i j

10 Edge Parameters: ED-BP
i j Conditions on edge parameters imply an iterative algorithm: ED-BP

11 Edge Parameters: ED-BP
i j Yields a weaker notion of equivalence:

12 A Spectrum of Approximations
ED-BP networks: [CD06]

13 A Spectrum of Approximations
ED-BP networks: [CD06] Exact Inference

14 A Spectrum of Approximations
ED-BP networks: [CD06] Loopy BP marginals Exact Inference

15 A Spectrum of Approximations
ED-BP networks: [CD06] Loopy BP marginals Exact Inference partition function?

16 A Partition Function i j

17 An Easy Case: Delete a Single Edge
Prop.: If MI(Xi,Xj) = 0 in ED-BP network M', then: where i j

18 An Easy Case: Delete a Single Edge
Prop.: If MI(Xi,Xj) = 0 in ED-BP network M', then: where i With multiple edges deleted (ZERO-EC): j

19 Bethe Free Energy and ZERO-EC
Bethe free energy approximation: as a partition function approximation:

20 Bethe Free Energy is ZERO-EC
Bethe free energy approximation: as a partition function approximation: Theorem: The Bethe approximation is ZERO-EC when M' is a tree :

21 An Easy Case: Delete a Single Edge
Prop.: If MI(Xi,Xj) = 0 in ED-BP network M', then: where i With multiple edges deleted (ZERO-EC): j

22 An Easy Case: Delete a Single Edge
Prop.: For any edge in an ED-BP network M', then where i With multiple edges deleted (GENERAL-EC): j

23 Overview tree exact exact marginals marginals LBP IJGP

24 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP joingraph free energies zero-EC Bethe exact Z

25 Overview tree exact exact marginals marginals LBP IJGP
joingraph free energies zero-EC Bethe exact Z improved approximations (higher order EP/GBP energies) general-EC

26 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP joingraph free energies zero-EC Bethe exact Z recover edges general-EC

27 Edge Recovery: ZERO-EC
i j Recover edges with largest MI(Xi;Xj)

28 Edge Recovery: GENERAL-EC
j i t s Recover edges with largest MI(Xi,Xj; Xs,Xt)

29 Edge Recovery Bethe exact Z 6x6 grid EC-Z,rand 0.07 0.06 0.05
relative error 0.04 0.03 0.02 exact Z 0.01 edges recovered 25

30 Edge Recovery Bethe exact Z 6x6 grid EC-Z,rand EC-G,rand 0.07 0.06
0.05 relative error 0.04 0.03 0.02 exact Z 0.01 edges recovered 25

31 Edge Recovery Bethe exact Z 6x6 grid EC-Z,rand EC-G,rand 0.07 EC-Z,MI
0.06 0.05 relative error 0.04 0.03 0.02 exact Z 0.01 edges recovered 25

32 Edge Recovery Bethe exact Z 6x6 grid EC-Z,rand EC-G,rand 0.07 EC-Z,MI
EC-G,MI 0.06 0.05 relative error 0.04 0.03 0.02 exact Z 0.01 edges recovered 25

33 Edge Recovery Bethe exact Z 6x6 grid EC-Z,rand EC-G,rand 0.07 EC-Z,MI
EC-G,MI 0.06 EC-G,MI2 0.05 relative error 0.04 0.03 0.02 exact Z 0.01 edges recovered 25

34 Edge Recovery

35 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP/GBP joingraph free energies zero-EC Bethe exact Z recover edges general-EC

36 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP/GBP joingraph free energies zero-EC Bethe exact Z partial corrections: general-EC

37 Partial Correction pigs 1 ED-BP (Bethe) 0.8 0.6 relative error 0.4
general-EC 0.2 500 1000 1500 2000 2500 3000 time (ms)

38 Partial Correction pigs 1 ED-BP (Bethe) 0.8 0.6 relative error
soft-sep see AAAI’08 0.4 general-EC 0.2 500 1000 1500 2000 2500 3000 time (ms)

39 Partial Correction

40 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP/GBP joingraph free energies zero-EC Bethe exact Z general-EC

41 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP/GBP marginal corrections (AAAI'08) joingraph free energies zero-EC Bethe exact Z general-EC

42 Thanks!

Download ppt "Approximating the Partition Function by Deleting and then Correcting for Model Edges Arthur Choi and Adnan Darwiche University of California, Los Angeles."

Similar presentations

Exact Inference. Inference Basic task for inference: – Compute a posterior distribution for some query variables given some observed evidence – Sum out.

Exact Inference. Inference Basic task for inference: – Compute a posterior distribution for some query variables given some observed evidence – Sum out.
Join-graph based cost-shifting Alexander Ihler, Natalia Flerova, Rina Dechter and Lars Otten University of California Irvine Introduction Mini-Bucket Elimination.

Join-graph based cost-shifting Alexander Ihler, Natalia Flerova, Rina Dechter and Lars Otten University of California Irvine Introduction Mini-Bucket Elimination.
Exact Inference in Bayes Nets

Exact Inference in Bayes Nets
Online Performance Guarantees for Sparse Recovery Raja Giryes ICASSP 2011 Volkan Cevher.

Online Performance Guarantees for Sparse Recovery Raja Giryes ICASSP 2011 Volkan Cevher.
Loopy Belief Propagation a summary. What is inference? Given: –Observabled variables Y –Hidden variables X –Some model of P(X,Y) We want to make some.

Loopy Belief Propagation a summary. What is inference? Given: –Observabled variables Y –Hidden variables X –Some model of P(X,Y) We want to make some.
Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies Tamir Hazan, Amnon Shashua School of Computer Science.

Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies Tamir Hazan, Amnon Shashua School of Computer Science.
Introduction to Belief Propagation and its Generalizations. Max Welling Donald Bren School of Information and Computer and Science University of California.

Introduction to Belief Propagation and its Generalizations. Max Welling Donald Bren School of Information and Computer and Science University of California.
Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.

Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.
CS774. Markov Random Field : Theory and Application Lecture 04 Kyomin Jung KAIST Sep 15 2009.

CS774. Markov Random Field : Theory and Application Lecture 04 Kyomin Jung KAIST Sep
Original Tree: 9876543210 2000011001000 1000001100100 1700010101010 1200001111000 5231000001011 4000110010000 3500101011110 8441101001100 7751100000111.

Original Tree:
What is the next line of the proof? a). Let G be a graph with k vertices. b). Assume the theorem holds for all graphs with k+1 vertices. c). Let G be a.

What is the next line of the proof? a). Let G be a graph with k vertices. b). Assume the theorem holds for all graphs with k+1 vertices. c). Let G be a.
Global Approximate Inference Eran Segal Weizmann Institute.

Global Approximate Inference Eran Segal Weizmann Institute.
Belief Propagation, Junction Trees, and Factor Graphs

Belief Propagation, Junction Trees, and Factor Graphs
A Constraint Satisfaction Problem (CSP) is a combinatorial decision problem defined by a set of variables, a set of domain values for these variables,

A Constraint Satisfaction Problem (CSP) is a combinatorial decision problem defined by a set of variables, a set of domain values for these variables,
Genome evolution: a sequence-centric approach Lecture 6: Belief propagation.

Genome evolution: a sequence-centric approach Lecture 6: Belief propagation.
Bioinformatics and Graphical Models: Computation, approximation, and their value MSR: Nebojsa Jojic, Vladimir Jojic, Chris Meek, David Heckerman UW: Jim.

Bioinformatics and Graphical Models: Computation, approximation, and their value MSR: Nebojsa Jojic, Vladimir Jojic, Chris Meek, David Heckerman UW: Jim.
On the Power of Belief Propagation: A Constraint Propagation Perspective Rina Dechter Bozhena Bidyuk Robert Mateescu Emma Rollon.

On the Power of Belief Propagation: A Constraint Propagation Perspective Rina Dechter Bozhena Bidyuk Robert Mateescu Emma Rollon.
A Trainable Graph Combination Scheme for Belief Propagation Kai Ju Liu New York University.

A Trainable Graph Combination Scheme for Belief Propagation Kai Ju Liu New York University.
Genome Evolution. Amos Tanay 2009 Genome evolution: Lecture 8: Belief propagation.

Genome Evolution. Amos Tanay 2009 Genome evolution: Lecture 8: Belief propagation.
Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Similar presentations

Ads by Google