1. 30 November, 2005 CIMCA2005, Vienna 1 Statistical Learning Procedure in Loopy Belief Propagation for Probabilistic Image Processing Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University kazu@smapip.is.tohoku.ac.jphttp://www.smapip.is.tohoku.ac.jp/~kazu/ References K. Tanaka, H. Shouno, M. Okada and D. M. Titterington: Accuracy of the Bethe approximation for hyperparameter estimation in probabilistic image processing, J. Phys. A, vol.37, pp.8675-8695 (2004).

2. 30 November, 2005 CIMCA2005, Vienna 2 Bayesian Network and Belief Propagation Probabilistic Information Processing Probabilistic Model Bayes Formula Belief Propagation J. Pearl: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Morgan Kaufmann, 1988). Bayesian Network Graphical Model

3. 30 November, 2005CIMCA2005, Vienna3 How should we treat the calculation of the summation over 256 N configurations? It is very hard to calculate it exactly except some special cases. Formulation for approximate algorithm Accuracy of the approximate algorithm Belief Propagation

4. 30 November, 2005CIMCA2005, Vienna4 Contents 1.Introduction 2.Bayesian Image Analysis and Gaussian Graphical Model 3.Belief Propagation 4.Concluding Remarks

5. 30 November, 2005CIMCA2005, Vienna5 Bayesian Image Analysis Original ImageDegraded Image Transmission Noise

6. 30 November, 2005CIMCA2005, Vienna6 Bayesian Image Analysis Degradation Process Original ImageDegraded Image Transmission Additive White Gaussian Noise

7. 30 November, 2005CIMCA2005, Vienna7 Bayesian Image Analysis A Priori Probability Standard Images Generate Similar?

8. 30 November, 2005CIMCA2005, Vienna8 Bayesian Image Analysis A Posteriori Probability Gaussian Graphical Model

9. 30 November, 2005CIMCA2005, Vienna9 Bayesian Image Analysis Original Image Degraded Image A Priori Probability A Posteriori Probability Degraded Image Pixels

10. 30 November, 2005CIMCA2005, Vienna10 Hyperparameter Determination by Maximization of Marginal Likelihood Marginalization Original Image Marginal Likelihood Degraded Image

11. 30 November, 2005CIMCA2005, Vienna11 Maximization of Marginal Likelihood by EM (Expectation Maximization) Algorithm Marginal Likelihood Incomplete Data Equivalent Q-Function

12. 30 November, 2005CIMCA2005, Vienna12 Maximization of Marginal Likelihood by EM (Expectation Maximization) Algorithm Marginal Likelihood Q-Function Iterate the following EM-steps until convergence: EM Algorithm A. P. Dempster, N. M. Laird and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. Roy. Stat. Soc. B, 39 (1977).

13. 30 November, 2005CIMCA2005, Vienna13 Contents 1.Introduction 2.Bayesian Image Analysis and Gaussian Graphical Model 3.Belief Propagation 4.Concluding Remarks

14. 30 November, 2005CIMCA2005, Vienna14 Probabilistic Model on a Graph with Loops Marginal Probability

15. 30 November, 2005CIMCA2005, Vienna15 Belief Propagation 1 42 5 3 1 4 53 2 6 8 7 Message Update Rule

16. 30 November, 2005CIMCA2005, Vienna16 Message Passing Rule of Belief Propagation Fixed Point Equations for Massage 1 3 42 5

17. 30 November, 2005CIMCA2005, Vienna17 Fixed Point Equation and Iterative Method Fixed Point Equation Iterative Method

18. 30 November, 2005CIMCA2005, Vienna18 Image Restoration by Gaussian Graphical Model Original Image Degraded Image MSE: 1529 MSE: 1512 EM Algorithm with Belief Propagation

19. 30 November, 2005CIMCA2005, Vienna19 Comparison of Belief Propagation with Exact Results in Gaussian Graphical Model MSE Belief Propagation 3270.00061136.302-5.19201 Exact3150.00075937.919-5.21444 MSE 2600.00057433.998-5.15241 Exact2360.00065234.975-5.17528

20. 30 November, 2005CIMCA2005, Vienna20 Image Restoration by Gaussian Graphical Model Original Image MSE:315 MSE: 325 MSE: 545 MSE: 447 MSE: 411 MSE: 1512 Degraded Image Belief Propagation Lowpass Filter Median Filter Exact Wiener Filter

21. 30 November, 2005CIMCA2005, Vienna21 Original Image MSE236 MSE: 260 MSE: 372 MSE: 244 MSE: 224 MSE: 1529 Degraded Image Belief Propagation Lowpass Filter Median Filter Exact Wiener Filter Image Restoration by Gaussian Graphical Model

22. 30 November, 2005CIMCA2005, Vienna22 Contents 1.Introduction 2.Bayesian Image Analysis and Gaussian Graphical Model 3.Belief Propagation 4.Concluding Remarks

23. 30 November, 2005CIMCA2005, Vienna23 Summary Formulation of belief propagation Accuracy of belief propagation in Bayesian image analysis by means of Gaussian graphical model (Comparison between the belief propagation and exact calculation)