28 February, 2003University of Glasgow1 Cluster Variation Method and Probabilistic Image Processing -- Loopy Belief Propagation -- Kazuyuki Tanaka Graduate.

Slides:

Advertisements

Similar presentations

Motivating Markov Chain Monte Carlo for Multiple Target Tracking

Advertisements

Graphical Models and Applications CNS/EE148 Instructors: M.Polito, P.Perona, R.McEliece TA: C. Fanti.

Exact Inference in Bayes Nets

Introduction to Belief Propagation and its Generalizations. Max Welling Donald Bren School of Information and Computer and Science University of California.

3 March, 2003University of Glasgow1 Statistical-Mechanical Approach to Probabilistic Inference --- Cluster Variation Method and Generalized Loopy Belief.

Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.

Markov Networks.

Graduate School of Information Sciences, Tohoku University

1 Bayesian Image Modeling by Generalized Sparse Markov Random Fields and Loopy Belief Propagation Kazuyuki Tanaka GSIS, Tohoku University, Sendai, Japan.

24 November, 2011National Tsin Hua University, Taiwan1 Mathematical Structures of Belief Propagation Algorithms in Probabilistic Information Processing.

Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields Yong-Joong Kim Dept. of Computer Science Yonsei.

1 物理フラクチュオマティクス論 Physical Fluctuomatics 応用確率過程論 Applied Stochastic Process 第 5 回グラフィカルモデルによる確率的情報処理 5th Probabilistic information processing by means of.

Physics Fluctuomatics / Applied Stochastic Process (Tohoku University) 1 Physical Fluctuomatics Applied Stochastic Process 7th “More is different” and.

Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 7th~10th Belief propagation Appendix Kazuyuki Tanaka Graduate School of Information.

1 October, 2007 ALT&DS2007 (Sendai, Japan ） 1 Introduction to Probabilistic Image Processing and Bayesian Networks Kazuyuki Tanaka Graduate School of Information.

1 Physical Fluctuomatics 5th and 6th Probabilistic information processing by Gaussian graphical model Kazuyuki Tanaka Graduate School of Information Sciences,

3 September, 2009 SSP2009, Cardiff, UK 1 Probabilistic Image Processing by Extended Gauss-Markov Random Fields Kazuyuki Tanaka Kazuyuki Tanaka, Muneki.

Physics Fluctuomatics / Applied Stochastic Process (Tohoku University) 1 Physical Fluctuomatics Applied Stochastic Process 9th Belief propagation Kazuyuki.

Markov Random Fields Probabilistic Models for Images

Physical Fuctuomatics (Tohoku University) 1 Physical Fluctuomatics Applied Stochastic Process 1st Review of probabilistic information processing Kazuyuki.

10 December, 2008 CIMCA2008 (Vienna) 1 Statistical Inferences by Gaussian Markov Random Fields on Complex Networks Kazuyuki Tanaka, Takafumi Usui, Muneki.

September 2007 IW-SMI2007, Kyoto 1 A Quantum-Statistical-Mechanical Extension of Gaussian Mixture Model Kazuyuki Tanaka Graduate School of Information.

Tokyo Institute of Technology, Japan Yu Nishiyama and Sumio Watanabe Theoretical Analysis of Accuracy of Gaussian Belief Propagation.

Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 12th Bayesian network and belief propagation in statistical inference Kazuyuki Tanaka.

14 October, 2010LRI Seminar 2010 (Univ. Paris-Sud)1 Statistical performance analysis by loopy belief propagation in probabilistic image processing Kazuyuki.

29 December, 2008 National Tsing Hua University, Taiwan 1 Introduction to Probabilistic Image Processing and Bayesian Networks Kazuyuki Tanaka Graduate.

Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 7th~10th Belief propagation Kazuyuki Tanaka Graduate School of Information Sciences,

Phisical Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 4th Maximum likelihood estimation and EM algorithm Kazuyuki Tanaka Graduate School.

Approximate Inference: Decomposition Methods with Applications to Computer Vision Kyomin Jung ( KAIST ) Joint work with Pushmeet Kohli (Microsoft Research)

Lecture 2: Statistical learning primer for biologists

Physical Fuctuomatics (Tohoku University) 1 Physical Fluctuomatics 1st Review of probabilistic information processing Kazuyuki Tanaka Graduate School of.

Graduate School of Information Sciences, Tohoku University

Efficient Belief Propagation for Image Restoration Qi Zhao Mar.22,2006.

Markov Networks: Theory and Applications Ying Wu Electrical Engineering and Computer Science Northwestern University Evanston, IL 60208

Physical Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 13th Quantum-mechanical extensions of probabilistic information processing Kazuyuki.

Bayesian Brain Probabilistic Approaches to Neural Coding 1.1 A Probability Primer Bayesian Brain Probabilistic Approaches to Neural Coding 1.1 A Probability.

30 November, 2005 CIMCA2005, Vienna 1 Statistical Learning Procedure in Loopy Belief Propagation for Probabilistic Image Processing Kazuyuki Tanaka Graduate.

Learning Machine Learning Further Introduction from Eleftherios inspired mainly from Ghahramani’s and Bishop’s lectures.

29 June, 2006 Kyoto University 1 画像処理における確率伝搬法と EM アルゴリズムの統計的性能評価東北大学大学院情報科学研究科田中和之 Reference 田中和之 : ガウシアングラフィカルモデルにもとづく確率的情報処理におけ.

ICPR2004 (24 July, 2004, Cambridge) 1 Probabilistic image processing based on the Q-Ising model by means of the mean- field method and loopy belief propagation.

10 October, 2007 University of Glasgow 1 EM Algorithm with Markov Chain Monte Carlo Method for Bayesian Image Analysis Kazuyuki Tanaka Graduate School.

Graduate School of Information Sciences, Tohoku University

マルコフ確率場の統計的機械学習の数理とデータサイエンスへの展開 Statistical Machine Learning in Markov Random Field and Expansion to Data Sciences 田中和之東北大学大学院情報科学研究科 Kazuyuki Tanaka.

Physical Fluctuomatics 13th Quantum-mechanical extensions of probabilistic information processing Kazuyuki Tanaka Graduate School of Information Sciences,

Statistical-Mechanical Approach to Probabilistic Image Processing -- Loopy Belief Propagation and Advanced Mean-Field Method -- Kazuyuki Tanaka and Noriko.

Graduate School of Information Sciences, Tohoku University

Sublinear Computational Time Modeling in Statistical Machine Learning Theory for Markov Random Fields Kazuyuki Tanaka GSIS, Tohoku University, Sendai,

Graduate School of Information Sciences, Tohoku University, Japan

Graduate School of Information Sciences, Tohoku University

Graduate School of Information Sciences, Tohoku University

Markov Networks.

Graduate School of Information Sciences Tohoku University, Japan

Graduate School of Information Sciences, Tohoku University

Cluster Variation Method for Correlation Function of Probabilistic Model with Loopy Graphical Structure Kazuyuki Tanaka Graduate School of Information.

Graduate School of Information Sciences, Tohoku University

Graduate School of Information Sciences

Physical Fluctuomatics 7th~10th Belief propagation

Expectation-Maximization & Belief Propagation

Graduate School of Information Sciences, Tohoku University

Advanced Mean Field Methods in Quantum Probabilistic Inference

Probabilistic image processing and Bayesian network

Probabilistic image processing and Bayesian network

Graduate School of Information Sciences, Tohoku University

Cluster Variation Method for Correlation Function of Probabilistic Model with Loopy Graphical Structure Kazuyuki Tanaka Graduate School of Information.

Graduate School of Information Sciences, Tohoku University

Markov Networks.

Graduate School of Information Sciences, Tohoku University

Graduate School of Information Sciences, Tohoku University

Kazuyuki Tanaka Graduate School of Information Sciences

Presentation transcript:

28 February, 2003University of Glasgow1 Cluster Variation Method and Probabilistic Image Processing -- Loopy Belief Propagation -- Kazuyuki Tanaka Graduate School of Information Sciences Tohoku University

28 February, 2003University of Glasgow2 Probabilistic Model and Image Restoration Original Image Degraded Image Transmission Noise

28 February, 2003University of Glasgow3 Image Restoration and Magnetic Material Restored images are determined from a priori information and given data. Ordered states are determined from interactions and external fields. Feature detection from the data and image processing by means of filters. Interpretation and prediction of physical property by means of physical model. Similarity of Mathematical Structure Regular lattice consisting of a lot of nodes. Neighbouring spin interactions and Markov random field

28 February, 2003University of Glasgow4 Massive Information Processing and Probabilistic Information Processing Computational Complexity. Approximation algorithms for massive information processing by means of advanced mean-field methods. Application of the cluster variation method (Bethe/Kikuchi method) to massive information processing Cluster Variation Method is equivalent to a generalized loopy belief propagation for probabilistic inference in the artificial intelligence.

28 February, 2003University of Glasgow5 Important point in the application of cluster variation method to probabilistic image processing Design of iterative algorithms for probabilistic inference based on cluster variation method (Computer Science). Hyperparameter estimation (Statistics). Cooperative phenomena in probabilistic models and probabilistic information processing (Physics).

28 February, 2003University of Glasgow6 Degradation Process and A Priori Probability in Binary Image Restoration Degradation Process (Binary Symmetric Channel) A Priori Probability

28 February, 2003University of Glasgow7 A Priori Probability in Binary Image Restoration

28 February, 2003University of Glasgow8 Bayes Formula and A Posteriori Probability Maximization of Posterior Marginal

28 February, 2003University of Glasgow9 A Posteriori Probability in Binary Image Restoration

28 February, 2003University of Glasgow10 Kullback-Leibler divergence

28 February, 2003University of Glasgow11 Kullback-Leibler Divergence

28 February, 2003University of Glasgow12 Basic Framework of Pair Approximation in Cluster Variation Method

28 February, 2003University of Glasgow13 Propagation Rule of Pair Approximation in Cluster Variation Method Update Rule is reduced to Loopy Belief Propagation

28 February, 2003University of Glasgow14 One-Body Marginal Probability of Pair Approximation in CVM

28 February, 2003University of Glasgow15 Two-Body Marginal Probability of Pair Approximation in CVM

28 February, 2003University of Glasgow16 Message Propagation Rule of Pair Approximation in CVM

28 February, 2003University of Glasgow17 Binary Image Restoration Original images are generated by Monte Carlo simulations in the a priori probability. Original Image Degraded Image (p=0.2) Restored Image

28 February, 2003University of Glasgow18 Binary Image Restoration Original ImageDegraded ImageRestored Image

28 February, 2003University of Glasgow19 Hyperparameter Estimation Maximization of Marginal Likelihood

28 February, 2003University of Glasgow20 Binary Image Restoration Original images are generated by Monte Carlo simulations in the a priori probability. Original Image Degraded Image (p=0.2) Restored Image

28 February, 2003University of Glasgow21 Binary Image Restoration Original Image Degraded Image Mean Field Approximation Pair Approximation in CVM Hyperparameters are determined so as to maximize the marginal likelihood.

28 February, 2003University of Glasgow22 Multi-Valued Image Restoration Degradation Process

28 February, 2003University of Glasgow23 A Priori Probability in Multi-Valued Image Restoration Q-state Ising Model Q-state Potts Model Kronecker Delta

28 February, 2003University of Glasgow24 Multi-Valued Image Restoration (Q=4) Hyperparameters are determined so as to maximize the marginal likelihood. Degraded Image Restored Image Original Image 4-state Ising Model 4-state Potts Model

28 February, 2003University of Glasgow25 Multi-Valued Image Restoration (Q=4) Hyperparameters are determined so as to maximize the marginal likelihood. Degraded Image （ 3p=0.3 ） 4-state Potts Model Original Image 4-state Ising Model

28 February, 2003University of Glasgow26 Summary Probabilistic Image Processing by Bayes Formula Cluster Variation Method and Loopy Belief Propagation Binary Image Restoration Multi-Valued Image Restoration

28 February, 2003University of Glasgow27 Other Practical Applications Edge Detection Segmentation Texture Analysis Image Compression Motion Detection Color Image

28 February, 2003University of Glasgow28 Other Theoretical Works Hyperparameter Estimation by EM algorithm Statistical Performance Estimation and Spin Glass Theory Replica method Inequality of Statistical Quantity Line Field Generalized Loopy Belief Propagation and Cluster Variation Method Information Geometry and Cluster Variation Method