1. Labwork 3

2. Solving an estimation problem
4 steps
1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise)
-> one needs to take into account the physics of signal acquisition
2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB
3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …)
4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

3. Solving an estimation problem
4 steps
1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise)
-> one needs to take into account the physics of signal acquisition
2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB
3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …)
4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

4. Lab 3 and project : PALM microscopy
Continuous signal for 1 fluorophore (in 1D to simplify the problem):
with
Hypothesis: Gaussian PSF of parameter sr q

5. Lab 3 and project : PALM microscopy
Continuous signal for 1 fluorophore (in 1D to simplify the problem):
with
Hypothesis: Gaussian PSF of parameter sr q D D

6. Lab 3 and project : PALM microscopy
Continuous signal for 1 fluorophore (in 1D to simplify the problem):
with
Measured signal : has to take into account pixel integration
with
and bi a white Gaussan noise of variance s2 si q D D

7. Solving an estimation problem
4 steps
1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise)
-> one needs to take into account the physics of signal acquisition
2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB
3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …)
4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

8. Lab 3 and project : PALM microscopy
Expression of the log-likelihood
Expression of the CRLB when a is assumed known

9. Lab 3 and project : PALM microscopy
Variation of the CRLB as a function of q
Period = D= 2 µm

10. TP 3 and project : PALM microscopy
Most favorable case si 2 4 …
D=2 µm
q=46µm D
Most defavorable case si 2 4 …
D=2 µm
q=47µm D

11. TP 3 and project : PALM microscopy
Most favorable case
q=46µm
q=46.5µm
q=47µm
Most defavorable case
q=47.5µm
q=48µm
D=2 µm

12. TP 3 and project : PALM microscopy
D=2 µm
Most favorable case
q=46µm
q=46.1µm
q=46.2µm
Most defavorable case
q=47 µm
q=47.1µm
q=47.2µm

13. Solving an estimation problem
4 steps
1. Choose a relevant model for the observed signal (parameter to estimate + statistical properties s of the noise)
-> one needs to take into account the physics of signal acquisition
2. Characterize the potential performance and the relevant parameters by computing, when it is possible, the CRLB
3. Determine one (or several) estimator(s): one begins with the ML, and if untractable, find other solutions (moments, …)
4. Study the performance of the estimator (bias, variance), in general by using Monte Carlo simulations.

14. Following of Lab 3 ML estimator unbiased and efficient ?
With Monte Carlo simulations
Consider a is unknown (nuisance parameter):
Expression of the CRLB
Expression of the ML estimator (subpixel estimation)
Check efficiency of ML estimator with Monte Carlo simulations
Find the value of w that « minimizes » the CRLB
Optimize the magnification of the optical system
(ratio PSF size / pixel size)

15. Layout of the project TP 3 : Estimation precision of q
Algorithm for subpixel estimation of q in a 1D signal
Optimization of the magnifcation (ratio size of PSF / size of pixel)
Lecture 4 :
How to write a scientific paper ?
Example : Libres Savoirs, article_astro.pdf
TP 5 :
Peak detection in an image
« By yourself » :
ML algorithm for subpixel estimation of q in an 2D image
Report under the form of a scientific paper

16. Read the text …

17. IV. The matched filter
1. Estimation in presence of additive Gaussian noise
2. Cramer-Rao Lower Bound and matched filter
Application example : delay estimation
3. Nuisance parameters
4. Estimation the presence of signal-dependent Poisson noise
Application to astronomical image registration

18. Estimation under additive Gaussian noise
Signal model :
Gaussian noise
One assumes that: =1
White noise of variance s2 ?
Expression of the loglikelihood :

19. Example : 1D vector Gaussian white noise K=512 Correlated noise 4 4 4

20. Estimation under additive Gaussian noise
Correlated noise of covariance matrix G
Expression of the loglikelihood :
Define the new vectors:
Whitening
Loglikelihood :
After whitening, the case of correlated noise is equivalent to the case of white noise

21. Estimation under additive Gaussian noise
Signal model :
One assumes that: =1
Expression of the loglikelihood (white noise):

22. Ambiguity function « True » signal Ambiguity function :
Noise of variance s2
Expression of the loglikelihood :

23. Ambiguity function

24. Cramer-Rao Lower Bound
avec
In our case, the loglikelihood equals ambiguity function + noise. One shows that :
SNR
with
SNR

25. Curvature of ambiguity function
Estimation is all the more precise that the ambiguity function is narrow.
SNR
Curvature of the ambiguity function

26. Mean-square estimator
Matched filter
ML estimator :
Optimal only if noise is Gaussian, additive and white !
Mean-square estimator
Simplification :
Matched filter

27. Matched filter
In the presence of correlated noise ?
replace s and r(q) with :
The covariance matrix of the noise has to be taken into account in the expression of the matched filter

28. Example : time delay estimation
Estimation of the translation of a continuous time signal :
Ambiguity function :
Autocorrelation of r(t) du
Second derivative :
The autocorrelation function of r(q) must be as « narrow » as possible.

29. Example : time delay estimation
An alternative way of expressing the curvature: ?
One defines :

30. Uncertainty relations
One defines the normalized signal
Mean time
Mean frequency
One defines:
and
and
Spectral spread
Temporal spread
Using Cauchy-Schwarz inequality, one shows that :

31. Example : time delay estimation
CRLB :
SNR
Precision depends on :
- SNR
spectral width of the signal
= curvature of the autocorrelation function

32. Example : time delay estimation
Example of Gaussian pulse :
SNR
To have precise estimation, the pulse must be narrow
Is it always the case ?

33. Example : time delay estimation
Gaussian pulses of different widths

34. Example : time delay estimation
Spectra (modulus of FT)

35. Example : time delay estimation
Autocorrelation functions

36. Example : time delay estimation
« Chirp »

37. Example : time delay estimation
Autocorrelation function

38. Example : time delay estimation
Spectrum (modulus of FT)

39. Example : time delay estimation
« Chirp »

40. Example : time delay estimation
Autocorrelation function

41. Example : time delay estimation
Spectrum (modulus of FT)

42. Example : radar Distance estimation: d= c (to-te) / 2
Emitted signal
(modulated by a « carrier ») te
Received signal (attenuation in 1/d4) d
Distance estimation:
Matched filter
d= c (to-te) / 2
The parameter to estimate is t0
Equivalent principle for sonars, optical telemeters, … t0

43. Pulse compression - Example
Signal without noise – 3 pulses : chirp, C=50 t3 t1 t2

44. Pulse compression - Example
Noisy signal ~ 20 dB t3 t1 t2

45. Pulse compression - Example
After matched filter: t3 t1 t2
Modern radar : T=31 µs, Df=15,5 MHz => C=480 !

46. Example : time delay estimation
Coded pulse (Barker code) :

47. Example : position estimation in an image
r(x,y)

48. Example : position estimation in an image
r(x-t,y-h)
Signal model :
r(x,y) t h
Estimate (t,h) ?
Ambiguity function ?
2D autocorrelation

49. CRLB for vectorial parameters

50. Example : position estimation in an image
Fisher Matrix
r(x-t,y-h)
r(x,y) t h

51. Example : position estimation in an image
Inverse of Fisher Matrix
r(x-t,y-h)
r(x,y) t h
CRLB
If :

52. Example : Gaussian spot
Parallel to reference axes :
One has :
CRLB ?
Direction in which estimation variance is larger ? smaller ?

53. Example : Gaussian spot
General case :
with :
Direction in which estimation variance is larger ? smaller ?

54. Example : Gaussian spot
General case :
with :
Numerical criterion for « global » estimation precision ?

55. Other interpretation of the Fisher matrix

56. Matching problem

57. What are the best blocks for registration ?
Matching problem
What are the best blocks for registration ?

58. Corners

59. Criterion for best blocks detection
“Harris points”
Criterion for best blocks detection

60. Nuisance parameter If a is unknown ? : ML estimate of a :
One « injects » the estimate of a into the loglikelihood:

61. Delay estimation in the presence of Poisson noise
Additive noise
Poisson noise ?
s is a Poisson random vector of mean ar(q)
Loglikelihood
« Natural » normalization:
Additive noise
ML estimator :