Presentation is loading. Please wait.

Presentation is loading. Please wait.

Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 7th~10th Belief propagation Appendix Kazuyuki Tanaka Graduate School of Information.

Published byIrma Logan Modified over 8 years ago

Similar presentations

Presentation on theme: "Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 7th~10th Belief propagation Appendix Kazuyuki Tanaka Graduate School of Information."— Presentation transcript:

1 Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 7th~10th Belief propagation Appendix Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University kazu@smapip.is.tohoku.ac.jp http://www.smapip.is.tohoku.ac.jp/~kazu/

2 Physics Fluctuomatics (Tohoku University) 2 Textbooks Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese), Chapter 5. References H. Nishimori: Statistical Physics of Spin Glasses and Information Processing, ---An Introduction, Oxford University Press, 2001. H. Nishimori, G. Ortiz: Elements of Phase Transitions and Critical Phenomena, Oxford University Press, 2011. M. Mezard, A. Montanari: Information, Physics, and Computation, Oxford University Press, 2010.

3 Physics Fluctuomatics (Tohoku University) 3 Probabilistic Model for Ferromagnetic Materials

4 Physics Fluctuomatics (Tohoku University) 4 Probabilistic Model for Ferromagnetic Materials Prior probability prefers to the configuration with the least number of red lines. > > =

5 Physics Fluctuomatics (Tohoku University) 5 More is different in Probabilistic Model for Ferromagnetic Materials Disordered State Ordered State Sampling by Markov Chain Monte Carlo method Small p Large p More is different. Critical Point (Large fluctuation)

6 Physics Fluctuomatics (Tohoku University) 6 Fundamental Probabilistic Models for Magnetic Materials Since h is positive, the probablity of up spin is larger than the one of down spin ． +1 11 h ： External Field Variance Average

7 Physics Fluctuomatics (Tohoku University) 7 Fundamental Probabilistic Models for Magnetic Materials Since J is positive, (a 1,a 2 )=(+1,+1) and (  1,  1) have the largest probability ． J ： Interaction Variance Average +1 11 11 11 11

8 Physics Fluctuomatics (Tohoku University) 8 Fundamental Probabilistic Models for Magnetic Materials Translational Symmetry J J h h E ： Set of All the neighbouring Pairs of Nodes Problem: Compute

9 Physics Fluctuomatics (Tohoku University) 9 Fundamental Probabilistic Models for Magnetic Materials Problem: Compute Translational Symmetry J J h h Spontaneous Magnetization

10 Physics Fluctuomatics (Tohoku University) 10 Mean Field Approximation for Ising Model We assume that the probability for configurations satisfying i Jm h are large.

11 Physics Fluctuomatics (Tohoku University) 11 Mean Field Approximation for Ising Model Fixed Point Equation of m We assume that all random variables a i are independent of each other, approximately.

12 Physics Fluctuomatics (Tohoku University) 12 Fixed Point Equation and Iterative Method Fixed Point Equation

13 Physics Fluctuomatics (Tohoku University) 13 Fixed Point Equation and Iterative Method Fixed Point Equation Iterative Method

14 Physics Fluctuomatics (Tohoku University) 14 Fixed Point Equation and Iterative Method Fixed Point Equation Iterative Method

15 Physics Fluctuomatics (Tohoku University) 15 Fixed Point Equation and Iterative Method Fixed Point Equation Iterative Method

16 Physics Fluctuomatics (Tohoku University) 16 Fixed Point Equation and Iterative Method Fixed Point Equation Iterative Method

17 Physics Fluctuomatics (Tohoku University) 17 Fixed Point Equation and Iterative Method Fixed Point Equation Iterative Method

18 Physics Fluctuomatics (Tohoku University) 18 Fixed Point Equation and Iterative Method Fixed Point Equation Iterative Method

19 Physics Fluctuomatics (Tohoku University) 19 Marginal Probability Distribution in Mean Field Approximation i Jm h Jm ： Mean Field

20 Physics Fluctuomatics (Tohoku University) 20 Advanced Mean Field Method h h h Bethe Approximation Kikuchi Method (Cluster Variation Meth) ： Effective Field Fixed Point Equation for  J

21 Physics Fluctuomatics (Tohoku University) 21 Average of Ising Model on Square Grid Graph (a)Mean Field Approximation (b)Bethe Approximation (c)Kikuchi Method (Cluster Variation Method) (d)Exact Solution （ L. Onsager ， C.N.Yang ） J J h h

22 Physics Fluctuomatics (Tohoku University) 22 Model Representation in Statistical Physics Gibbs Distribution Partition Function Free Energy Energy Function

23 Physics Fluctuomatics (Tohoku University) 23 Gibbs Distribution and Free Energy Gibbs Distribution Variational Principle of Free Energy Functional F[Q] under Normalization Condition for Q(a) Free Energy Functional of Trial Probability Distribution Q(a) Free Energy

24 Physics Fluctuomatics (Tohoku University) 24 Explicit Derivation of Variantional Principle for Minimization of Free Energy Functional Normalization Condition

25 Physics Fluctuomatics (Tohoku University) 25 Kullback-Leibler Divergence and Free Energy

26 Physics Fluctuomatics (Tohoku University) 26 Interpretation of Mean Field Approximation as Information Theory and Marginal Probability Distributions Q i (a i ) are determined so as to minimize D[Q|P] Minimization of Kullback-Leibler Divergence between

27 Physics Fluctuomatics (Tohoku University) 27 Interpretation of Mean Field Approximation as Information Theory Problem: Compute Translational Symmetry J J h h Magnetization

28 Physics Fluctuomatics (Tohoku University) 28 Kullback-Leibler Divergence in Mean Field Approximation for Ising Model

29 Physics Fluctuomatics (Tohoku University) 29 Minimization of Kullback-Leibler Divergence and Mean Field Equation Fixed Point Equations for {Q i |  i  V} Variation i Set of all the neighbouring nodes of the node i

30 Physics Fluctuomatics (Tohoku University) 30 Orthogonal Functional Representation of Marginal Probability Distribution of Ising Model

31 Physics Fluctuomatics (Tohoku University) 31 Conventional Mean Field Equation in Ising Model Fixed Point Equation J J Translational Symmetry h h

32 Physics Fluctuomatics (Tohoku University) 32 Interpretation of Bethe Approximation (1) Translational Symmetry J J h h Compute and

33 Interpretation of Bethe Approximation (2) Free Energy KL Divergence 33 Physics Fluctuomatics (Tohoku University)

34 Interpretation of Bethe Approximation (3) Bethe Free Energy Free Energy KL Divergence 34 Physics Fluctuomatics (Tohoku University)

35 Interpretation of Bethe Approximation (4) 35 Physics Fluctuomatics (Tohoku University)

36 Interpretation of Bethe Approximation (5) Lagrange Multipliers to ensure the constraints 36 Physics Fluctuomatics (Tohoku University)

37 Interpretation of Bethe Approximation (6) Extremum Condition 37 Physics Fluctuomatics (Tohoku University)

38 Interpretation of Bethe Approximation (7) Extremum Condition 38 Physics Fluctuomatics (Tohoku University)

39 Interpretation of Bethe Approximation (8) 1 42 5 3 Extremum Condition 39 Physics Fluctuomatics (Tohoku University) 1 4 5 3 2 687

40 Interpretation of Bethe Approximation (9) 1 42 5 3 4 1 53 2 687 Message Update Rule 40 Physics Fluctuomatics (Tohoku University)

41 Interpretation of Bethe Approximation (10) 1 3 42 5 1 4 5 3 2 6 8 7 1 42 53 = Message Passing Rule of Belief Propagation It corresponds to Bethe approximation in the statistical mechanics. 41 Physics Fluctuomatics (Tohoku University)

42 Interpretation of Bethe Approximation (11) 42 Physics Fluctuomatics (Tohoku University) Translational Symmetry

43 Physics Fluctuomatics (Tohoku University) 43 Summary Statistical Physics and Information Theory Probabilistic Model of Ferromagnetism Mean Field Theory Gibbs Distribution and Free Energy Free Energy and Kullback-Leibler Divergence Interpretation of Mean Field Approximation as Information Theory Interpretation of Bethe Approximation as Information Theory

Download ppt "Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 7th~10th Belief propagation Appendix Kazuyuki Tanaka Graduate School of Information."

Similar presentations

Loopy Belief Propagation a summary. What is inference? Given: –Observabled variables Y –Hidden variables X –Some model of P(X,Y) We want to make some.

Loopy Belief Propagation a summary. What is inference? Given: –Observabled variables Y –Hidden variables X –Some model of P(X,Y) We want to make some.
Introduction to Belief Propagation and its Generalizations. Max Welling Donald Bren School of Information and Computer and Science University of California.

Introduction to Belief Propagation and its Generalizations. Max Welling Donald Bren School of Information and Computer and Science University of California.
3 March, 2003University of Glasgow1 Statistical-Mechanical Approach to Probabilistic Inference --- Cluster Variation Method and Generalized Loopy Belief.

3 March, 2003University of Glasgow1 Statistical-Mechanical Approach to Probabilistic Inference --- Cluster Variation Method and Generalized Loopy Belief.
Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.

Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.
Graduate School of Information Sciences, Tohoku University

Graduate School of Information Sciences, Tohoku University
CS774. Markov Random Field : Theory and Application Lecture 04 Kyomin Jung KAIST Sep 15 2009.

CS774. Markov Random Field : Theory and Application Lecture 04 Kyomin Jung KAIST Sep
1 Bayesian Image Modeling by Generalized Sparse Markov Random Fields and Loopy Belief Propagation Kazuyuki Tanaka GSIS, Tohoku University, Sendai, Japan.

1 Bayesian Image Modeling by Generalized Sparse Markov Random Fields and Loopy Belief Propagation Kazuyuki Tanaka GSIS, Tohoku University, Sendai, Japan.
The Markov property Discrete time: A time symmetric version: A more general version: Let A be a set of indices >k, B a set of indices

The Markov property Discrete time: A time symmetric version: A more general version: Let A be a set of indices >k, B a set of indices
Monte Carlo Simulation of Ising Model and Phase Transition Studies

Monte Carlo Simulation of Ising Model and Phase Transition Studies
Computer vision: models, learning and inference Chapter 10 Graphical Models.

Computer vision: models, learning and inference Chapter 10 Graphical Models.
24 November, 2011National Tsin Hua University, Taiwan1 Mathematical Structures of Belief Propagation Algorithms in Probabilistic Information Processing.

24 November, 2011National Tsin Hua University, Taiwan1 Mathematical Structures of Belief Propagation Algorithms in Probabilistic Information Processing.
Monte Carlo Simulation of Ising Model and Phase Transition Studies By Gelman Evgenii.

Monte Carlo Simulation of Ising Model and Phase Transition Studies By Gelman Evgenii.
Some Surprises in the Theory of Generalized Belief Propagation Jonathan Yedidia Mitsubishi Electric Research Labs (MERL) Collaborators: Bill Freeman (MIT)

Some Surprises in the Theory of Generalized Belief Propagation Jonathan Yedidia Mitsubishi Electric Research Labs (MERL) Collaborators: Bill Freeman (MIT)
1 物理フラクチュオマティクス論 Physical Fluctuomatics 応用確率過程論 Applied Stochastic Process 第 5 回グラフィカルモデルによる確率的情報処理 5th Probabilistic information processing by means of.

1 物理フラクチュオマティクス論 Physical Fluctuomatics 応用確率過程論 Applied Stochastic Process 第 5 回グラフィカルモデルによる確率的情報処理 5th Probabilistic information processing by means of.
Physics Fluctuomatics / Applied Stochastic Process (Tohoku University) 1 Physical Fluctuomatics Applied Stochastic Process 7th “More is different” and.

Physics Fluctuomatics / Applied Stochastic Process (Tohoku University) 1 Physical Fluctuomatics Applied Stochastic Process 7th “More is different” and.
1 October, 2007 ALT&DS2007 (Sendai, Japan ） 1 Introduction to Probabilistic Image Processing and Bayesian Networks Kazuyuki Tanaka Graduate School of Information.

1 October, 2007 ALT&DS2007 (Sendai, Japan ） 1 Introduction to Probabilistic Image Processing and Bayesian Networks Kazuyuki Tanaka Graduate School of Information.
Lecture 11: Ising model Outline: equilibrium theory d = 1

Lecture 11: Ising model Outline: equilibrium theory d = 1
1 Physical Fluctuomatics 5th and 6th Probabilistic information processing by Gaussian graphical model Kazuyuki Tanaka Graduate School of Information Sciences,

1 Physical Fluctuomatics 5th and 6th Probabilistic information processing by Gaussian graphical model Kazuyuki Tanaka Graduate School of Information Sciences,
3 September, 2009 SSP2009, Cardiff, UK 1 Probabilistic Image Processing by Extended Gauss-Markov Random Fields Kazuyuki Tanaka Kazuyuki Tanaka, Muneki.

3 September, 2009 SSP2009, Cardiff, UK 1 Probabilistic Image Processing by Extended Gauss-Markov Random Fields Kazuyuki Tanaka Kazuyuki Tanaka, Muneki.
Physics Fluctuomatics / Applied Stochastic Process (Tohoku University) 1 Physical Fluctuomatics Applied Stochastic Process 2nd Probability and its fundamental.

Physics Fluctuomatics / Applied Stochastic Process (Tohoku University) 1 Physical Fluctuomatics Applied Stochastic Process 2nd Probability and its fundamental.

Similar presentations

Ads by Google