1. 3.7 Adaptive filtering Joonas Vanninen Antonio Palomino Alarcos

2. Adaptive filtering Linear filtering does not take into account the local features of the image – Causes for example blurring of the edges An improvement: change the filter parameters according to the local statistics – Change the shape and the size of the neighborhood – Suppress the filtering if there are features that we want to preserve in the neighborhood

3. Filters The local LMMSE filter The noise-updating repeated Wiener filter The adaptive 2D LMS filter The adaptive rectangular window LMS filter The adaptive-neighborhood filter

4. The local LMMSE filter Local Linear Minimum Mean Squared Error filter, also known as the adaptive Wiener filter Assumes that the image is corrupted by additive noise Minimizes the local mean squared error by applying a linear operator to each pixel in the image The local mean and variance are estimated from a rectangular window around the pixel

5. LLMMSE estimation g is the corrupted image μ g is the local mean σ g 2 is the local variance σ η 2 is the variance Constant if noise is signal-independent Varies if noise is signal-dependent In the latter case it should be estimated locally with the knowledge of the type of the noise

6. MSE=1595 MSE=360 MSE=324

7. Interpretation of the equation If the area processed is uniform, the second term is small – Estimate is near the local mean of the noisy image – Noise is reduced If there is a edge in the processing window, the variance of the image is larger than the variance of the noise – Estimate is closer to the actual noisy value – Edge doesn’t get blurred

8. A refined version If the local variance is high, it is assumed that there is an edge in the processing window The edge is assumed to be straight in a small window The direction of the edge is calculated – Gradient operator – 8 different directions The processing window is divided to two uniform sub-areas over the edge The statistics in the sub-area that holds processed pixel are used Noise is reduced near the edges without causing blurring

9. Sub-areas

10. The noise-updating repeated Wiener Filter An iterative application of the LLMMSE filter The variance of noise used in the LLMMSE estimate formula is updated in each iteration The noise is reduced even near the edges To avoid blurring, a smaller window size may be choosen for each iteration

11. The Adaptative 2D LMS Filter

12. Why? Wiener filter has to be used with stationary and statistically independent signal and noise models Adaptative 2D LMS algorithm circunvent this problem – It takes into account the nonstationarity of the given image

13. 2D LMS Algorithm Same concept: fixed-window Wiener filter New idea: filter coefficients vary depending upon the image characteristics – Algorithm based on the steepest descent method – It tracks the variations in the local statistics adapting to the changes in the images

14. Estimate of the original pixel value f(m,n): – Convolution – w l (p,q) Ξ causal FIR filter – g(m,n) Ξ noise-corrupted input image

15. Updating for the filter coefficients New coefficients determined by minimazing MSE between f(m,n) and the estimation Steepest descent method: – μ controls the rate of convergence and filter stability. – Error is estimated using an approximation to the original signal d(m,n) – d(m,n) obtained by decorrelation from the input image g(m,n) 2D delay operator of (1,1) samples, use the previous pixel as an estimate

16. Advantages of the Algorithm It does not require a priori information about: – Image – Noise statistics – Correlation properties It does not require averaging, differentiation and matrix operations

17. MSE=1590 MSE=395

18. The Adaptative Rectangular Window LMS Filter

19. ARW LMS Algorithm Same concept: 2D LMS filter based on standard Wiener filter New Idea: use of an adaptative-sized rectangular window Additional assumption: image processes have zero mean

20. Implementation Taking into account that now we have zero mean noise, following estimate is derived: σ f 2 is the variance of the original image (estimated) Idea: globally nonstationary process can be considered locally stationary and ergodic over small regions Target: to identify the size of a stationary square region for each pixel in the image – Sample statistics can approximate the a posteriori pareameters needed

21. Updating the window size Large window It may include pixels form other ensembles Small window The statistics needed are poorly estimated ARW lengths (Lr and Lc) are varied using a signal activity parameter: A similar signal activity parameter is defined in the colum direction

22. Updating the window size (II) If S is large N is decremented If S is small N is incremented In order to make this decision, S is compared to a threshold T as follows: Threshold is defined as: κ controls the rate at which T changes

23. The adaptive-neighborhood filter Uses a variable-size, variable-shape neighborhood determined individually for every pixel Neighborhood contains only spatially connected pixels that are similar to the seed This way the estimated statistics are likely to be closer to the true statistics of the signal Adaptive neighbourhoods should not mix the pixels of an object with the pixels of the background → no blurring of the edges

24. Region growing The absolute difference between each of the 8- neighbor pixels of the region and the seed is calculated If the value is under a threshold T, pixel is included in the region The process is iterated until there are no new pixels Additionally a background region is grown by expanding the foreground boundary by a prespecified number of pixels The regions are grown for each pixel in the image

25. An example of region growing The same neighborhood can be used for every pixel in the area with the same gray-value

26. Adaptive-neighborhood mean and median filters The mean and median values of adaptive neighborhoods can be used to filter noise They provide a larger population to compute local statistics than fixed 3 x 3 or 5 x 5 windows Edge distortion is pervented because the neighbourhood does not cross object boundaries

27. Adaptive-neighborhood noise subtraction (ANNS) Used for removing additive signal-independent noise Estimates the noise value in the seed pixel with an adaptive neighborhood and subtracts it to obtain an estimate of the original The local statistics are estimated as in the ARW-LMS – filter

28. Implementation A maximum foreground limit of Q pixels is set. At first, the tolerance is set to the full dynamic range An estimate of the variance of the uncorrupted signal is calculated from the neighborhood It is compared to the noise variance, and if σ f 2 > 2σ η 2, it is assumed that the neighbourhood contains a significant feature If so, the tolerance is modified by T = 2 σ f 2

29. Implementation (II) If the foreground has only one pixel, the neighborhood is enlarged to include the backround around it (3 x 3) Flat regions → σ f 2 << 2σ η 2 → foreground will grow to Q Busy regions → σ f 2 >> 2σ η 2 → tolerance will be reduced

30. Conclusion Use of local statistics in an adaptive filter is a powerful approach to remove noise – while retaining the edges in the images with minimal distortion Some of the implicit assumptions may not apply well to the image or noise processes – It is common to try several previously established techniques It is quite difficult to compare the results provided by different filters – Generally MSE or RMS error is used – In real applications, it is important to obtain an assessment of the results by a specialist

31. Quote “You can only cure retail but you can prevent wholesale” Brock Chisholm

32. …any question?