Presentation is loading. Please wait.

Presentation is loading. Please wait.

BP in Practice Message Passing Inference Probabilistic Graphical

Published byEdward Sims Modified over 5 years ago

Similar presentations

Presentation on theme: "BP in Practice Message Passing Inference Probabilistic Graphical"— Presentation transcript:

1 BP in Practice Message Passing Inference Probabilistic Graphical
Models Message Passing BP in Practice

2 Misconception Revisited
A D B C

3 Nonconvergent BP Run

4 Different Variants of BP
Synchronous BP: all messages are updated in parallel synchronous Time (seconds) 2 4 6 8 10 12 14 # messages converged Ising Grid x 100 11 11 12 12 13 13 21 21 22 22 23 23 31 31 32 32 33 33

5 Different Variants of BP
Asynchronous BP: Messages are updated one at a time asynchronous Time (seconds) 2 4 6 8 10 12 14 # messages converged Ising Grid x 100 asynchronous order 2 synchronous 11 12 13 21 22 23 31 32 33

6 Observations Convergence is a local property:
some messages converge soon others may never converge Synchronous BP converges considerably worse than asynchronous Message passing order makes a difference to extent and rate of convergence

7 Informed Message Scheduling
Tree reparameterization (TRP) Pick a tree and pass messages to calibrate 11 12 13 21 22 23 31 32 33

8 Informed Message Scheduling
Tree reparameterization (TRP) Pick a tree and pass messages to calibrate Residual belief propagation (RBP) Pass messages between two clusters whose beliefs over the sepset disagree the most

9 Smoothing (Damping) Messages
Dampens oscillations in messages

10

11 Summary To achieve BP convergence, two main tricks
Damping Intelligent message ordering Convergence doesn’t guarantee correctness Bad cases for BP – both convergence & accuracy: Strong potentials pulling in different directions Tight loops Some new algorithms have better convergence: Optimization-based view to inference

12 END END END

Download ppt "BP in Practice Message Passing Inference Probabilistic Graphical"

Similar presentations

Algorithms for MAP estimation in Markov Random Fields Vladimir Kolmogorov University College London Tutorial at GDR (Optimisation Discrète, Graph Cuts.

Algorithms for MAP estimation in Markov Random Fields Vladimir Kolmogorov University College London Tutorial at GDR (Optimisation Discrète, Graph Cuts.
Bayesian Belief Propagation

Bayesian Belief Propagation
Scaling Up Graphical Model Inference

Scaling Up Graphical Model Inference
Exact Inference. Inference Basic task for inference: – Compute a posterior distribution for some query variables given some observed evidence – Sum out.

Exact Inference. Inference Basic task for inference: – Compute a posterior distribution for some query variables given some observed evidence – Sum out.
Variational Methods for Graphical Models Micheal I. Jordan Zoubin Ghahramani Tommi S. Jaakkola Lawrence K. Saul Presented by: Afsaneh Shirazi.

Variational Methods for Graphical Models Micheal I. Jordan Zoubin Ghahramani Tommi S. Jaakkola Lawrence K. Saul Presented by: Afsaneh Shirazi.
Lauritzen-Spiegelhalter Algorithm

Lauritzen-Spiegelhalter Algorithm
Exact Inference in Bayes Nets

Exact Inference in Bayes Nets
Junction Trees And Belief Propagation. Junction Trees: Motivation What if we want to compute all marginals, not just one? Doing variable elimination for.

Junction Trees And Belief Propagation. Junction Trees: Motivation What if we want to compute all marginals, not just one? Doing variable elimination for.
ICCV 2007 tutorial Part III Message-passing algorithms for energy minimization Vladimir Kolmogorov University College London.

ICCV 2007 tutorial Part III Message-passing algorithms for energy minimization Vladimir Kolmogorov University College London.
Discrete Optimization in Computer Vision Nikos Komodakis Ecole des Ponts ParisTech, LIGM Traitement de l’information et vision artificielle.

Discrete Optimization in Computer Vision Nikos Komodakis Ecole des Ponts ParisTech, LIGM Traitement de l’information et vision artificielle.
Loopy Belief Propagation a summary. What is inference? Given: –Observabled variables Y –Hidden variables X –Some model of P(X,Y) We want to make some.

Loopy Belief Propagation a summary. What is inference? Given: –Observabled variables Y –Hidden variables X –Some model of P(X,Y) We want to make some.
Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies Tamir Hazan, Amnon Shashua School of Computer Science.

Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies Tamir Hazan, Amnon Shashua School of Computer Science.
3 March, 2003University of Glasgow1 Statistical-Mechanical Approach to Probabilistic Inference --- Cluster Variation Method and Generalized Loopy Belief.

3 March, 2003University of Glasgow1 Statistical-Mechanical Approach to Probabilistic Inference --- Cluster Variation Method and Generalized Loopy Belief.
Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.

Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.
Junction tree Algorithm 10-708:Probabilistic Graphical Models Recitation: 10/04/07 Ramesh Nallapati.

Junction tree Algorithm :Probabilistic Graphical Models Recitation: 10/04/07 Ramesh Nallapati.
Summary Problem: Exponential Performance Gap: Computer architectures transitioned from exponential frequency scaling to parallelism ending decades of free.

Summary Problem: Exponential Performance Gap: Computer architectures transitioned from exponential frequency scaling to parallelism ending decades of free.
Distributed Message Passing for Large Scale Graphical Models Alexander Schwing Tamir Hazan Marc Pollefeys Raquel Urtasun CVPR2011.

Distributed Message Passing for Large Scale Graphical Models Alexander Schwing Tamir Hazan Marc Pollefeys Raquel Urtasun CVPR2011.
Carnegie Mellon Focused Belief Propagation for Query-Specific Inference Anton Chechetka Carlos Guestrin 14 May 2010.

Carnegie Mellon Focused Belief Propagation for Query-Specific Inference Anton Chechetka Carlos Guestrin 14 May 2010.
Practical Belief Propagation in Wireless Sensor Networks Bracha Hod Based on a joint work with: Danny Dolev, Tal Anker and Danny Bickson The Hebrew University.

Practical Belief Propagation in Wireless Sensor Networks Bracha Hod Based on a joint work with: Danny Dolev, Tal Anker and Danny Bickson The Hebrew University.
Convergent and Correct Message Passing Algorithms Nicholas Ruozzi and Sekhar Tatikonda Yale University TexPoint fonts used in EMF. Read the TexPoint manual.

Convergent and Correct Message Passing Algorithms Nicholas Ruozzi and Sekhar Tatikonda Yale University TexPoint fonts used in EMF. Read the TexPoint manual.

Similar presentations

Ads by Google