1. 1 物理フラクチュオマティクス論 Physical Fluctuomatics 応用確率過程論 Applied Stochastic Process 第 5 回グラフィカルモデルによる確率的情報処理 5th Probabilistic information processing by means of graphical model 東北大学 大学院情報科学研究科 応用情報科学専攻 田中 和之 (Kazuyuki Tanaka) kazu@smapip.is.tohoku.ac.jp http://www.smapip.is.tohoku.ac.jp/~kazu/ 物理フラクチュオマティクス論 ( 東北大 )

2. 2 今回の講義の講義ノート 田中和之著： 確率モデルによる画像処理技術入門， 森北出版， 2006 ． 物理フラクチュオマティクス論 ( 東北大 )

3. 3 Contents 1.Introduction 2.Probabilistic Image Processing 3.Gaussian Graphical Model 4.Statistical Performance Analysis 5.Concluding Remarks

4. 物理フラクチュオマティクス論 ( 東北大 ) 4 Contents 1.Introduction 2.Probabilistic Image Processing 3.Gaussian Graphical Model 4.Statistical Performance Analysis 5.Concluding Remarks

5. 物理フラクチュオマティクス論 ( 東北大 ) 5 Markov Random Fields for Image Processing S. Geman and D. Geman (1986): IEEE Transactions on PAMI Image Processing for Markov Random Fields (MRF) (Simulated Annealing, Line Fields) J. Zhang (1992): IEEE Transactions on Signal Processing Image Processing in EM algorithm for Markov Random Fields (MRF) (Mean Field Methods) Markov Random Fields are One of Probabilistic Methods for Image processing.

6. 物理フラクチュオマティクス論 ( 東北大 ) 6 Markov Random Fields for Image Processing In Markov Random Fields, we have to consider not only the states with high probabilities but also ones with low probabilities. In Markov Random Fields, we have to estimate not only the image but also hyperparameters in the probabilistic model.  We have to perform the calculations of statistical quantities repeatedly. Hyperparameter Estimation Statistical Quantities Estimation of Image We can calculate statistical quantities by adopting the Gaussian graphical model as a prior probabilistic model and by using Gaussian integral formulas.

7. 物理フラクチュオマティクス論 ( 東北大 ) 7 Purpose of My Talk Review of formulation of probabilistic model for image processing by means of conventional statistical schemes. Review of probabilistic image processing by using Gaussian graphical model (Gaussian Markov Random Fields) as the most basic example. K. Tanaka: Statistical-Mechanical Approach to Image Processing (Topical Review), J. Phys. A: Math. Gen., vol.35, pp.R81-R150, 2002. Section 2 and Section 4 are summarized in the present talk.

8. 物理フラクチュオマティクス論 ( 東北大 ) 8 Contents 1.Introduction 2.Probabilistic Image Processing 3.Gaussian Graphical Model 4.Statistical Performance Analysis 5.Concluding Remarks

9. 物理フラクチュオマティクス論 ( 東北大 ) 9 Image Representation in Computer Vision Digital image is defined on the set of points arranged on a square lattice. The elements of such a digital array are called pixels. We have to treat more than 100,000 pixels even in the digital cameras and the mobile phones.

10. 物理フラクチュオマティクス論 ( 東北大 ) 10 Image Representation in Computer Vision At each point, the intensity of light is represented as an integer number or a real number in the digital image data. A monochrome digital image is then expressed as a two- dimensional light intensity function and the value is proportional to the brightness of the image at the pixel.

11. 物理フラクチュオマティクス論 ( 東北大 ) 11 Noise Reduction by Conventional Filters 192 202 190 202 219 120 100 218 110 192 202 190 202 173 120 100 218 110 It is expected that probabilistic algorithms for image processing can be constructed from such aspects in the conventional signal processing. Markov Random FieldsProbabilistic Image Processing Algorithm Smoothing Filters The function of a linear filter is to take the sum of the product of the mask coefficients and the intensities of the pixels.

12. 物理フラクチュオマティクス論 ( 東北大 ) 12 Bayes Formula and Bayesian Network Posterior Probability Bayes Rule Prior Probability Event A is given as the observed data. Event B corresponds to the original information to estimate. Thus the Bayes formula can be applied to the estimation of the original information from the given data. A B Bayesian Network Data-Generating Process

13. 物理フラクチュオマティクス論 ( 東北大 ) 13 Image Restoration by Probabilistic Model Original Image Degraded Image Transmission Noise Assumption 1: The degraded image is randomly generated from the original image by according to the degradation process. Assumption 2: The original image is randomly generated by according to the prior probability. Bayes Formula

14. 物理フラクチュオマティクス論 ( 東北大 ) 14 Image Restoration by Probabilistic Model Degraded Image i f i : Light Intensity of Pixel i in Original Image Position Vector of Pixel i g i : Light Intensity of Pixel i in Degraded Image i Original Image The original images and degraded images are represented by f = {f i } and g = {g i }, respectively.

15. 物理フラクチュオマティクス論 ( 東北大 ) 15 Probabilistic Modeling of Image Restoration Random Fields fifi gigi fifi gigi or Assumption 1: A given degraded image is obtained from the original image by changing the state of each pixel to another state by the same probability, independently of the other pixels.

16. 物理フラクチュオマティクス論 ( 東北大 ) 16 Probabilistic Modeling of Image Restoration Random Fields Assumption 2: The original image is generated according to a prior probability. Prior Probability consists of a product of functions defined on the neighbouring pixels. ij Product over All the Nearest Neighbour Pairs of Pixels

17. 物理フラクチュオマティクス論 ( 東北大 ) 17 Prior Probability for Binary Image = = > pp i j Probability of Neigbouring Pixel ij It is important how we should assume the function  (f i,f j ) in the prior probability. We assume that every nearest-neighbour pair of pixels take the same state of each other in the prior probability.

18. 物理フラクチュオマティクス論 ( 東北大 ) 18 Prior Probability for Binary Image Prior probability prefers to the configuration with the least number of red lines. Which state should the center pixel be taken when the states of neighbouring pixels are fixed to the white states? ？ > = = > pp i j Probability of Nearest Neigbour Pair of Pixels

19. 物理フラクチュオマティクス論 ( 東北大 ) 19 Prior Probability for Binary Image Which state should the center pixel be taken when the states of neighbouring pixels are fixed as this figure? ？-？？-？ = = > pp > > = Prior probability prefers to the configuration with the least number of red lines.

20. 物理フラクチュオマティクス論 ( 東北大 ) 20 What happens for the case of large umber of pixels? p lnp Disordered State Critical Point (Large fluctuation) small plarge p Covariance between the nearest neghbour pairs of pixels Sampling by Marko chain Monte Carlo Ordered State Patterns with both ordered states and disordered states are often generated near the critical point.

21. 物理フラクチュオマティクス論 ( 東北大 ) 21 Pattern near Critical Point of Prior Probability ln p similar small plarge p Covariance between the nearest neghbour pairs of pixels We regard that patterns generated near the critical point are similar to the local patterns in real world images.

22. 物理フラクチュオマティクス論 ( 東北大 ) 22 Contents 1.Introduction 2.Probabilistic Image Processing 3.Gaussian Graphical Model 4.Statistical Performance Analysis 5.Concluding Remarks

23. 物理フラクチュオマティクス論 ( 東北大 ) 23 Bayesian Image Analysis by Gaussian Graphical Model Patterns are generated by MCMC. Markov Chain Monte Carlo Method Prior Probability E:Set of all the nearest-neighbour pairs of pixels V:Set of all the pixels

24. 物理フラクチュオマティクス論 ( 東北大 ) 24 Bayesian Image Analysis by Gaussian Graphical Model Histogram of Gaussian Random Numbers Degraded image is obtained by adding a white Gaussian noise to the original image. Degradation Process is assumed to be the additive white Gaussian noise. V: Set of all the pixels Original Image f Gaussian Noise n Degraded Image g

25. 物理フラクチュオマティクス論 ( 東北大 ) 25 Bayesian Image Analysis Original Image Degraded Image Prior Probability Posterior Probability Degradation Process Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability. E ： Set of all the nearest neighbour pairs of pixels V ： Set of All the pixels

26. 物理フラクチュオマティクス論 ( 東北大 ) 26 Estimation of Original Image We have some choices to estimate the restored image from posterior probability. In each choice, the computational time is generally exponential order of the number of pixels. Thresholded Posterior Mean (TPM) estimation Maximum posterior marginal (MPM) estimation Maximum A Posteriori (MAP) estimation (1) (2) (3)

27. 物理フラクチュオマティクス論 ( 東北大 ) 27 Statistical Estimation of Hyperparameters Marginalized with respect to F Original Image Marginal Likelihood Degraded Image Hyperparameters  are determined  so as to maximize the marginal likelihood Pr{G=g| ,  } with respect to , 

28. 物理フラクチュオマティクス論 ( 東北大 ) 28 Bayesian Image Analysis A Posteriori Probability Gaussian Graphical Model

29. 物理フラクチュオマティクス論 ( 東北大 ) 29 Average of Posterior Probability Gaussian Integral formula

30. 物理フラクチュオマティクス論 ( 東北大 ) 30 Bayesian Image Analysis by Gaussian Graphical Model Multi-Dimensional Gaussian Integral Formula Posterior Probability Average of the posterior probability can be calculated by using the multi- dimensional Gauss integral Formula |V|x|V| matrix E:Set of all the nearest-neghbour pairs of pixels V:Set of all the pixels

31. 物理フラクチュオマティクス論 ( 東北大 ) 31 Statistical Estimation of Hyperparameters Marginalized with respect to F Original Image Marginal Likelihood Degraded Image

32. 物理フラクチュオマティクス論 ( 東北大 ) 32 Calculations of Partition Function (A is a real symmetric and positive definite matrix.) Gaussian Integral formula

33. 物理フラクチュオマティクス論 ( 東北大 ) 33 Exact expression of Marginal Likelihood in Gaussian Graphical Model Multi-dimensional Gauss integral formula We can construct an exact EM algorithm.

34. 物理フラクチュオマティクス論 ( 東北大 ) 34 Bayesian Image Analysis by Gaussian Graphical Model Iteration Procedure in Gaussian Graphical Model

35. 物理フラクチュオマティクス論 ( 東北大 ) 35 Image Restoration by Markov Random Field Model and Conventional Filters MSE Statistical Method315 Lowpass Filter (3x3)388 (5x5)413 Median Filter (3x3)486 (5x5)445 (3x3) Lowpass (5x5) Median MRF Original Image Degraded Image RestoredImage V:Set of all the pixels

36. 物理フラクチュオマティクス論 ( 東北大 ) 36 Contents 1.Introduction 2.Probabilistic Image Processing 3.Gaussian Graphical Model 4.Statistical Performance Analysis 5.Concluding Remarks

37. 物理フラクチュオマティクス論 ( 東北大 ) 37 Performance Analysis Posterior Probability Estimated Results Observed Data Sample Average of Mean Square Error Signal Additive White Gaussian Noise

38. 物理フラクチュオマティクス論 ( 東北大 ) 38 Statistical Performance Analysis Additive White Gaussian Noise Posterior Probability Restored Image Original Image Degraded Image Additive White Gaussian Noise

39. 物理フラクチュオマティクス論 ( 東北大 ) 39 Statistical Performance Analysis

40. 物理フラクチュオマティクス論 ( 東北大 ) 40 Statistical Performance Estimation for Gaussian Markov Random Fields = 0

41. 物理フラクチュオマティクス論 ( 東北大 ) 41 Statistical Performance Estimation for Gaussian Markov Random Fields   =40 

42. 物理フラクチュオマティクス論 ( 東北大 ) 42 Contents 1.Introduction 2.Probabilistic Image Processing 3.Gaussian Graphical Model 4.Statistical Performance Analysis 5.Concluding Remarks

43. 物理フラクチュオマティクス論 ( 東北大 ) 43 Summary Formulation of probabilistic model for image processing by means of conventional statistical schemes has been summarized. Probabilistic image processing by using Gaussian graphical model has been shown as the most basic example.

44. References K. Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese). K. Tanaka: Statistical-Mechanical Approach to Image Processing (Topical Review), J. Phys. A, 35 (2002). A. S. Willsky: Multiresolution Markov Models for Signal and Image Processing, Proceedings of IEEE, 90 (2002). 物理フラクチュオマティクス論 ( 東北大 ) 44

45. 物理フラクチュオマティクス論 ( 東北大 ) 45 Problem 5-1: Derive the expression of the posterior probability Pr{F=f|G=g, ,  } by using Bayes formulas Pr{F=f|G=g, ,  } =Pr{G=g|F=f,  }Pr{F=f,  }/Pr{G=g| ,  }. Here Pr{G=g|F=f,  } and Pr{F=f,  } are assumed to be as follows: [Answer]

46. 物理フラクチュオマティクス論 ( 東北大 ) 46 Problem 5-2: Show the following equality.

47. 物理フラクチュオマティクス論 ( 東北大 ) 47 Problem 5-3: Show the following equality.

48. 物理フラクチュオマティクス論 ( 東北大 ) 48 Problem 5-4: Show the following equalities by using the multi-dimensional Gaussian integral formulas.

49. 物理フラクチュオマティクス論 ( 東北大 ) 49 Problem 5-5: Derive the extremum conditions for the following marginal likelihood Pr{G=g  } with respect to the hyperparameters  and . [Answer]

50. 物理フラクチュオマティクス論 ( 東北大 ) 50 Problem 5-6: Derive the extremum conditions for the following marginal likelihood Pr{G=g  } with respect to the hyperparameters  and . [Answer]

51. 物理フラクチュオマティクス論 ( 東北大 ) 51 Problem 5-7: Make a program that generate a degraded image by the additive white Gaussian noise. Generate some degraded images from a given standard images by setting  =10,20,30,40 numerically. Calculate the mean square error (MSE) between the original image and the degraded image. Histogram of Gaussian Random Numbers F i  G i ~N(0,40 2 ) Original Image Gaussian Noise Degraded Image Sample Program: http://www.morikita.co.jp/soft/84661/ K.Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006.

52. 物理フラクチュオマティクス論 ( 東北大 ) 52 Problem 5-8: Make a program of the following procedure in probabilistic image processing by using the Gaussian graphical model and the additive white Gaussian noise. Algorithm: Repeat the following procedure until convergence Sample Program: http://www.morikita.co.jp/soft/84661/ K.Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006.