Presentation is loading. Please wait.

Presentation is loading. Please wait.

Graduate School of Information Sciences, Tohoku University

Similar presentations


Presentation on theme: "Graduate School of Information Sciences, Tohoku University"— Presentation transcript:

1 Graduate School of Information Sciences, Tohoku University
Physical Fluctuomatics 4th Maximum likelihood estimation and EM algorithm Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University Phisical Fluctuomatics (Tohoku University)

2 Phisical Fluctuomatics (Tohoku University)
Textbooks Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 4. Phisical Fluctuomatics (Tohoku University)

3 Statistical Machine Learning and Model Selection
Inference of Probabilistic Model by using Data Model Selection Statistical Machine Learning Maximum Likelihood Complete Data and Incomplete data EM algorithm Extension Implement by Belief Propagation and Markov Chain Monte Carlo method Akaike Information Criteria, Akaike Bayes Information Criteria, etc. Phisical Fluctuomatics (Tohoku University)

4 Maximum Likelihood Estimation
Parameter Phisical Fluctuomatics (Tohoku University)

5 Maximum Likelihood Estimation
Parameter 1 2 3 4 5 6 7 8 Phisical Fluctuomatics (Tohoku University)

6 Maximum Likelihood Estimation
Data Parameter 1 2 3 4 5 6 7 8 Data Phisical Fluctuomatics (Tohoku University)

7 Maximum Likelihood Estimation
Data Parameter 1 2 3 4 5 6 7 8 Data Histogram Phisical Fluctuomatics (Tohoku University)

8 Maximum Likelihood Estimation
Data Parameter 1 2 3 4 5 6 7 8 Data Histogram Phisical Fluctuomatics (Tohoku University)

9 Maximum Likelihood Estimation
Data Parameter 1 2 3 4 5 6 7 8 Data Histogram Phisical Fluctuomatics (Tohoku University)

10 Maximum Likelihood Estimation
Data Parameter Probability density function for data with average μ and variance σ2 is regarded as likelihood function for average μ and variance σ2 when data is given. Phisical Fluctuomatics (Tohoku University)

11 Maximum Likelihood Estimation
Data Parameter Probability density function for data with average μ and variance σ2 is regarded as likelihood function for average μ and variance σ2 when data is given. Extremum Condition Phisical Fluctuomatics (Tohoku University)

12 Maximum Likelihood Estimation
Data Parameter Probability density function for data with average μ and variance σ2 is regarded as likelihood function for average μ and variance σ2 when data is given. Extremum Condition Sample Deviation Sample Average Phisical Fluctuomatics (Tohoku University)

13 Maximum Likelihood Estimation
Data Parameter Probability density function for data with average μ and variance σ2 is regarded as likelihood function for average μ and variance σ2 when data is given. Extremum Condition Sample Deviation Sample Average Histogram Phisical Fluctuomatics (Tohoku University)

14 Phisical Fluctuomatics (Tohoku University)
Maximum Likelihood is unknown Hyperparameter Data Marginal Likelihood Extremum Condition Parameter Bayes Formula Phisical Fluctuomatics (Tohoku University)

15 Probabilistic Model for Signal Processing
Noise i fi i gi Transmission Source Signal Observable Data Bayes Formula Phisical Fluctuomatics (Tohoku University)

16 Prior Probability for Source Signal
j E:Set of all the links One dimensional Data Image Data 1 2 3 4 5 1 2 3 4 6 7 8 9 21 22 23 24 5 10 25 11 12 13 14 16 17 18 19 15 20 = 1 2 X 2 3 X 3 4 X 4 5 Phisical Fluctuomatics (Tohoku University)

17 Data Generating Process
Additive White Gaussian Noise V:Set of all the nodes Phisical Fluctuomatics (Tohoku University)

18 Probabilistic Model for Signal Processing
Parameter データ i fi i gi Hyperparameter Posterior Probability Phisical Fluctuomatics (Tohoku University)

19 Maximum Likelihood in Signal Processing
Incomplete Data Parameter Data Marginal Likelihood Hyperparameter Extremum Condition Phisical Fluctuomatics (Tohoku University)

20 Maximum Likelihood and EM algorithm
Incomplete Data Parameter Data Marginal Likelihood Q function Hyperparameter Expectation Maximization (EM) algorithm provide us one of Extremum Points of Marginal Likelihood. Extemum Condition Phisical Fluctuomatics (Tohoku University)

21 Model Selection in One Dimensional Signal
Expectation Maximization (EM) Algorithm 127 255 100 200 Original Signal Degraded Signal Estimated Signal 0.04 0.03 α(t) 0.02 0.01 α(0)=0.0001, σ(0)=100 Phisical Fluctuomatics (Tohoku University)

22 Model Selection in Noise Reduction
MSE 327 36.30 Estimate of Original Image Original Image Degraded Image EM algorithm and Belief Propagation α(0)=0.0001 σ(0)=100 MSE 260 34.00 Phisical Fluctuomatics (Tohoku University)

23 Phisical Fluctuomatics (Tohoku University)
Summary Maximum Likelihood and EM algorithm Statistical Inference by Gaussian Graphical Model Phisical Fluctuomatics (Tohoku University)

24 Phisical Fluctuomatics (Tohoku University)
Practice 3-1 ,  , ,  Let us suppose that data {gi |i=0,1,...,N-1} are generated by according to the following probability density function: , where Prove that estimates for average m and variance s2 of the maximum likelihood , are given as Phisical Fluctuomatics (Tohoku University)


Download ppt "Graduate School of Information Sciences, Tohoku University"

Similar presentations


Ads by Google