1. Fundamentals of estimation and detection in signals and images
Teaching team :
Matthieu Boffety
François Goudail

2. Layout and objective of the course
1. Basics of estimation theory
How extract information from a noisy signal in an optimal way ?
2. Basics of detection theory
How choose between two hypotheses in an optimal way ?
A few new mathematics
But mainly a new way of seeing phyiscs …

3. Organization of the course
Lectures : 5 slots (15h)
Printed document « Introduction to estimation and detection theory »
Books : Ph. Réfrégier, « Noise theory and application to physics »
S. Kay, « Fundamentals of statistical signal processing », vol. I and II
Lectures : blackboard + slides + exercises
Labworks (Matlab) : 5 slots (15h)
Signal processing is an experimental science !
Project (Matlab) : write a scientific paper !

4. FED : Fundamentals of estimation and detection
IM3D : Images, mouvement, 3D
PAI: Programmation pour les activités de l’ingénieur

5. Evaluation Project : write a scientific paper (to be handed on 21/10)
1/3 of the final mark
Project : write a scientific paper
(to be handed on 21/10)
Written examination
(19/10, 2h, no documents)
1/3 of the final mark
Labworks :
on-site evaluation for all labworks
reports (6 pages max) on labworks 2, 4 and 5 to be handed in and evaluated.
1/3 of the final mark

6. I. Estimation theory 1. The problematic of estimation
2. Measuring the quality of an estimator
3. Estimation methods
Moment method
Maximum likelihood estimator
4. Cramer-Rao Lower Bound – Efficiency of an estimator
5. Robust estimation

7. I. Estimation theory 1. The problematic of estimation
2. Measuring the quality of an estimator
3. Estimation methods
Moment method
Maximum likelihood estimator
4. Cramer-Rao Lower Bound – Efficiency of an estimator
5. Robust estimation

8. 1. The problematic of estimation
One observes a series of N measurements assumed to be produced by a random process. They are gathered in a set called a sample.
Observed sample : deterministic vector
Realization of
Random sample : random vector
Statistic :
Likelihood :

9. The likelihood
i.i.d. sample :

10. Estimation problem : example
Let us consider a i.i.d. sample with exponential PDF:
Our objective is to estimate the parameter q. Propose a method ?
The mean and the variance of an exponential random variable are :
Solution 1:
Solution 2:
What is the better solution ?
Need for a criterion to evaluate the quality of an estimator

11. II. Estimation theory 1. The problematic of estimation
2. Measuring the quality of an estimator
3. Estimation methods
Moment method
Maximum likelihood estimator
4. Cramer-Rao Lower Bound – Efficiency of an estimator
5. Robust estimation

12. Quality of an estimator
: True value of the parameter
: estimator (statistic)
Quality criterion : Mean Square Error
MSE can also be written as:
where:

13. Pdf of Standard deviation of the estimator : sT
Bias of the estimator : bT

14. Examples Example 1 : arithmetic mean (as an estimator of the mean)
unbiased
Example 2 : empirical variance (as an estimator of the variance)
biased
Gaussian sample:

15. Estimation problem : example
Let us consider a i.i.d. sample with exponential PDF:
Our objective is to estimate the parameter q. Propose a method ?
The mean and the variance of an exponential random variable are :
Solution 1:
Solution 2:
What is the better estimator ?

16. Pdf of two estimators
Exponential sample, N=100, estimated parameter : mean
True parameter : q0=1
Histograms estimated on 105 trials

17. Examples Example 3 : histogram (as an estimate of the PDF) Estimate :
from the sample :
Estimator :
unbiased
Bad estimation if Pk small

18. II. Estimation theory 1. The problematic of estimation
2. Measuring the quality of an estimator
3. Estimation methods
Moment method
Maximum likelihood estimator
4. Cramer-Rao Lower Bound – Efficiency of an estimator
5. Robust estimation

19. Moment estimator Assume that the sample has three parameters :
Consider the three first moments
The solution of the following system is the moment estimator :
Statistical moments
Empirical
moments
Example 1 : Gaussian sample – q=(m,s)

20. Moment estimator Assume that the sample has three parameters :
Consider the three first moments
The solution of the following system is the moment estimator :
Statistical moments
Empirical
moments
Example 2 : Gamma sample - q=(m, L)

21. Reduction of speckle noise
Spatial averaging of an image:
I(i,j) = intensity backscattered at pixel (i,j) of the image.
In the presence of speckle, I(i,j) is an exponential RV with mean <I>.
Averaging on a region:
Mean :
Variance :

22. Density of is the sum of NR exponential RV
What is its probability density ?
-> Gamma density of order NR (L)
L =1
L =10
As L increases, the probability density of the noise
becomes « thinner » (the variance decreases),
tends to a Gaussian (CLT)
L =2

23. Reduction of speckle noise
Gamma noise, L=9
P(I) I
Raw image
After spatial averaging on a 3x3 neighborhood
P(I) I
Exponential speckle

24. Moment estimator Assume that the sample has three parameters :
Consider the three first moments
Solution of the following system is the moment estimator :
Statistical moments
Empirical
moments
Example 2 : Gamma sample - q=(m, L)
L =1
L =2
L =10

25. II. Estimation theory 1. The problematic of estimation
2. Measuring the quality of an estimator
3. Estimation methods
Moment method
Maximum likelihood estimator
4. Cramer-Rao Lower Bound – Efficiency of an estimator
5. Robust estimation

26. The likelihood
i.i.d. sample :

27. Example : i.i.d. Gaussian sample
Marginal PDF:
Loglikelihood ?

28. Maximum likelihood (ML) estimator

29. Examples Example 1 : i.i.d. white Gaussian sample
For this estimation problem, the ML estimator is identical to the moment estimator

30. Examples Example 2 : Gamma sample
The ML estimator of m is identical to the moment estimator
The ML estimator of L is solution of the following equation:
with:
The ML estimator of L is different from the moment estimator

31. Examples Example 3 : i.i.d. correlated Gaussian sample
Objective : estimate m and G

32. II. Estimation theory 1. The problematic of estimation
2. Measuring the quality of an estimator
3. Estimation methods
Moment method
Maximum likelihood estimator
4. Cramer-Rao Lower Bound – Efficiency of an estimator
5. Robust estimation

33. Cramèr-Rao lower bound
Example : i.i.d. exponential sample

35. Small CRLB : good precision
Large CRLB : weak precision

36. Asymptotic efficiency of the ML estimator

37. Conditions of strict efficiency of the ML estimator

38. II. Estimation theory 1. The problematic of estimation
2. Mesuring the quality of an estimator
3. Estimation methods
Moment method
Maximum likelihood estimator
4. Cramer-Rao Lower Bound – Efficiency of an estimator
5. Robust estimation

39. Gaussian
Cauchy
a=0
5 realizations

40. Gaussian
Cauchy
a=0.01
5 realizations

41. Robust estimation
Empirical mean
Variance
Médian