Presentation is loading. Please wait.

Presentation is loading. Please wait.

Approximating the Partition Function by Deleting and then Correcting for Model Edges Arthur Choi and Adnan Darwiche University of California, Los Angeles.

Similar presentations


Presentation on theme: "Approximating the Partition Function by Deleting and then Correcting for Model Edges Arthur Choi and Adnan Darwiche University of California, Los Angeles."— Presentation transcript:

1 Approximating the Partition Function by Deleting and then Correcting for Model Edges
Arthur Choi and Adnan Darwiche University of California, Los Angeles

2 Edge Deletion: Idea Delete an edge Model M Model M'

3 Deleting an Equivalence Edge
j

4 Deleting an Equivalence Edge
j j

5 Deleting an Equivalence Edge
j

6 Deleting an Equivalence Edge
j

7 Deleting an Equivalence Edge
j

8 Deleting an Equivalence Edge
j

9 Edge Parameters: ED-BP
i j

10 Edge Parameters: ED-BP
i j Conditions on edge parameters imply an iterative algorithm: ED-BP

11 Edge Parameters: ED-BP
i j Yields a weaker notion of equivalence:

12 A Spectrum of Approximations
ED-BP networks: [CD06]

13 A Spectrum of Approximations
ED-BP networks: [CD06] Exact Inference

14 A Spectrum of Approximations
ED-BP networks: [CD06] Loopy BP marginals Exact Inference

15 A Spectrum of Approximations
ED-BP networks: [CD06] Loopy BP marginals Exact Inference partition function?

16 A Partition Function i j

17 An Easy Case: Delete a Single Edge
Prop.: If MI(Xi,Xj) = 0 in ED-BP network M', then: where i j

18 An Easy Case: Delete a Single Edge
Prop.: If MI(Xi,Xj) = 0 in ED-BP network M', then: where i With multiple edges deleted (ZERO-EC): j

19 Bethe Free Energy and ZERO-EC
Bethe free energy approximation: as a partition function approximation:

20 Bethe Free Energy is ZERO-EC
Bethe free energy approximation: as a partition function approximation: Theorem: The Bethe approximation is ZERO-EC when M' is a tree :

21 An Easy Case: Delete a Single Edge
Prop.: If MI(Xi,Xj) = 0 in ED-BP network M', then: where i With multiple edges deleted (ZERO-EC): j

22 An Easy Case: Delete a Single Edge
Prop.: For any edge in an ED-BP network M', then where i With multiple edges deleted (GENERAL-EC): j

23 Overview tree exact exact marginals marginals LBP IJGP

24 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP joingraph free energies zero-EC Bethe exact Z

25 Overview tree exact exact marginals marginals LBP IJGP
joingraph free energies zero-EC Bethe exact Z improved approximations (higher order EP/GBP energies) general-EC

26 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP joingraph free energies zero-EC Bethe exact Z recover edges general-EC

27 Edge Recovery: ZERO-EC
i j Recover edges with largest MI(Xi;Xj)

28 Edge Recovery: GENERAL-EC
j i t s Recover edges with largest MI(Xi,Xj; Xs,Xt)

29 Edge Recovery Bethe exact Z 6x6 grid EC-Z,rand 0.07 0.06 0.05
relative error 0.04 0.03 0.02 exact Z 0.01 edges recovered 25

30 Edge Recovery Bethe exact Z 6x6 grid EC-Z,rand EC-G,rand 0.07 0.06
0.05 relative error 0.04 0.03 0.02 exact Z 0.01 edges recovered 25

31 Edge Recovery Bethe exact Z 6x6 grid EC-Z,rand EC-G,rand 0.07 EC-Z,MI
0.06 0.05 relative error 0.04 0.03 0.02 exact Z 0.01 edges recovered 25

32 Edge Recovery Bethe exact Z 6x6 grid EC-Z,rand EC-G,rand 0.07 EC-Z,MI
EC-G,MI 0.06 0.05 relative error 0.04 0.03 0.02 exact Z 0.01 edges recovered 25

33 Edge Recovery Bethe exact Z 6x6 grid EC-Z,rand EC-G,rand 0.07 EC-Z,MI
EC-G,MI 0.06 EC-G,MI2 0.05 relative error 0.04 0.03 0.02 exact Z 0.01 edges recovered 25

34 Edge Recovery

35 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP/GBP joingraph free energies zero-EC Bethe exact Z recover edges general-EC

36 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP/GBP joingraph free energies zero-EC Bethe exact Z partial corrections: general-EC

37 Partial Correction pigs 1 ED-BP (Bethe) 0.8 0.6 relative error 0.4
general-EC 0.2 500 1000 1500 2000 2500 3000 time (ms)

38 Partial Correction pigs 1 ED-BP (Bethe) 0.8 0.6 relative error
soft-sep see AAAI’08 0.4 general-EC 0.2 500 1000 1500 2000 2500 3000 time (ms)

39 Partial Correction

40 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP/GBP joingraph free energies zero-EC Bethe exact Z general-EC

41 joingraph free energies
Overview tree exact exact marginals marginals LBP IJGP/GBP marginal corrections (AAAI'08) joingraph free energies zero-EC Bethe exact Z general-EC

42 Thanks!


Download ppt "Approximating the Partition Function by Deleting and then Correcting for Model Edges Arthur Choi and Adnan Darwiche University of California, Los Angeles."

Similar presentations


Ads by Google