1. 28 February, 2003University of Glasgow1 Cluster Variation Method and Probabilistic Image Processing -- Loopy Belief Propagation -- Kazuyuki Tanaka Graduate School of Information Sciences Tohoku University kazu@statp.is.tohoku.ac.jp http://www.statp.is.tohoku.ac.jp/~kazu/index-e.html

2. 28 February, 2003University of Glasgow2 Probabilistic Model and Image Restoration Original Image Degraded Image Transmission Noise

3. 28 February, 2003University of Glasgow3 Image Restoration and Magnetic Material Restored images are determined from a priori information and given data. Ordered states are determined from interactions and external fields. Feature detection from the data and image processing by means of filters. Interpretation and prediction of physical property by means of physical model. Similarity of Mathematical Structure Regular lattice consisting of a lot of nodes. Neighbouring spin interactions and Markov random field

4. 28 February, 2003University of Glasgow4 Massive Information Processing and Probabilistic Information Processing Computational Complexity. Approximation algorithms for massive information processing by means of advanced mean-field methods. Application of the cluster variation method (Bethe/Kikuchi method) to massive information processing Cluster Variation Method is equivalent to a generalized loopy belief propagation for probabilistic inference in the artificial intelligence.

5. 28 February, 2003University of Glasgow5 Important point in the application of cluster variation method to probabilistic image processing Design of iterative algorithms for probabilistic inference based on cluster variation method (Computer Science). Hyperparameter estimation (Statistics). Cooperative phenomena in probabilistic models and probabilistic information processing (Physics).

6. 28 February, 2003University of Glasgow6 Degradation Process and A Priori Probability in Binary Image Restoration Degradation Process (Binary Symmetric Channel) A Priori Probability

7. 28 February, 2003University of Glasgow7 A Priori Probability in Binary Image Restoration

8. 28 February, 2003University of Glasgow8 Bayes Formula and A Posteriori Probability Maximization of Posterior Marginal

9. 28 February, 2003University of Glasgow9 A Posteriori Probability in Binary Image Restoration

10. 28 February, 2003University of Glasgow10 Kullback-Leibler divergence

11. 28 February, 2003University of Glasgow11 Kullback-Leibler Divergence

12. 28 February, 2003University of Glasgow12 Basic Framework of Pair Approximation in Cluster Variation Method

13. 28 February, 2003University of Glasgow13 Propagation Rule of Pair Approximation in Cluster Variation Method Update Rule is reduced to Loopy Belief Propagation

14. 28 February, 2003University of Glasgow14 One-Body Marginal Probability of Pair Approximation in CVM

15. 28 February, 2003University of Glasgow15 Two-Body Marginal Probability of Pair Approximation in CVM

16. 28 February, 2003University of Glasgow16 Message Propagation Rule of Pair Approximation in CVM

17. 28 February, 2003University of Glasgow17 Binary Image Restoration Original images are generated by Monte Carlo simulations in the a priori probability. Original Image Degraded Image (p=0.2) Restored Image

18. 28 February, 2003University of Glasgow18 Binary Image Restoration Original ImageDegraded ImageRestored Image

19. 28 February, 2003University of Glasgow19 Hyperparameter Estimation Maximization of Marginal Likelihood

20. 28 February, 2003University of Glasgow20 Binary Image Restoration Original images are generated by Monte Carlo simulations in the a priori probability. Original Image Degraded Image (p=0.2) Restored Image

21. 28 February, 2003University of Glasgow21 Binary Image Restoration Original Image Degraded Image Mean Field Approximation Pair Approximation in CVM Hyperparameters are determined so as to maximize the marginal likelihood.

22. 28 February, 2003University of Glasgow22 Multi-Valued Image Restoration Degradation Process

23. 28 February, 2003University of Glasgow23 A Priori Probability in Multi-Valued Image Restoration Q-state Ising Model Q-state Potts Model Kronecker Delta

24. 28 February, 2003University of Glasgow24 Multi-Valued Image Restoration (Q=4) Hyperparameters are determined so as to maximize the marginal likelihood. Degraded Image Restored Image Original Image 4-state Ising Model 4-state Potts Model

25. 28 February, 2003University of Glasgow25 Multi-Valued Image Restoration (Q=4) Hyperparameters are determined so as to maximize the marginal likelihood. Degraded Image （ 3p=0.3 ） 4-state Potts Model Original Image 4-state Ising Model

26. 28 February, 2003University of Glasgow26 Summary Probabilistic Image Processing by Bayes Formula Cluster Variation Method and Loopy Belief Propagation Binary Image Restoration Multi-Valued Image Restoration

27. 28 February, 2003University of Glasgow27 Other Practical Applications Edge Detection Segmentation Texture Analysis Image Compression Motion Detection Color Image

28. 28 February, 2003University of Glasgow28 Other Theoretical Works Hyperparameter Estimation by EM algorithm Statistical Performance Estimation and Spin Glass Theory Replica method Inequality of Statistical Quantity Line Field Generalized Loopy Belief Propagation and Cluster Variation Method Information Geometry and Cluster Variation Method