1. EE465: Introduction to Digital Image Processing1 Limitation of Imaging Technology Two plagues in image acquisition Noise interference Blur (motion, out-of-focus, hazy weather) Difficult to obtain high-quality images as imaging goes Beyond visible spectrum Micro-scale (microscopic imaging) Macro-scale (astronomical imaging)

2. What is Noise? Wiki definition: noise means any unwanted signal One person’s signal is another one’s noise Noise is not always random and randomness is an artificial term Noise is not always bad (see stochastic resonance example in the next slide) EE465: Introduction to Digital Image Processing2

3. Stochastic Resonance EE465: Introduction to Digital Image Processing3 no noiseheavy noise light noise

4. EE465: Introduction to Digital Image Processing4 Image Denoising Where does noise come from? Sensor (e.g., thermal or electrical interference) Environmental conditions (rain, snow etc.) Why do we want to denoise? Visually unpleasant Bad for compression Bad for analysis

5. EE465: Introduction to Digital Image Processing5 electrical interference Noisy Image Examples thermal imaging ultrasound imaging physical interference

6. EE465: Introduction to Digital Image Processing6 (Ad-hoc) Noise Modeling Simplified assumptions Noise is independent of signal Noise types Independent of spatial location Impulse noise Additive white Gaussian noise Spatially dependent Periodic noise

7. EE465: Introduction to Digital Image Processing7 Noise Removal Techniques Linear filtering Nonlinear filtering Recall Linear system

8. EE465: Introduction to Digital Image Processing8 Image Denoising Introduction Impulse noise removal Median filtering Additive white Gaussian noise removal 2D convolution and DFT Periodic noise removal Band-rejection and Notch filter

9. EE465: Introduction to Digital Image Processing9 Impulse Noise (salt-pepper Noise) Definition Each pixel in an image has the probability of p/2 (0<p<1) being contaminated by either a white dot (salt) or a black dot (pepper) with probability of p/2 with probability of 1-p noisy pixels clean pixels X: noise-free image, Y: noisy image Note: in some applications, noisy pixels are not simply black or white, which makes the impulse noise removal problem more difficult

10. EE465: Introduction to Digital Image Processing10 Numerical Example P=0.1 128 128 128 128 128 128 128 128 128 128 X 128 128 255 0 128 128 128 128 128 128 128 128 128 128 0 128 128 128 128 0 128 128 128 128 128 128 128 128 128 128 128 128 0 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 0 128 128 128 128 255 128 128 128 128 128 128 128 128 128 128 128 128 128 255 128 128 128 128 128 128 128 255 128 128 Y Noise level p=0.1 means that approximately 10% of pixels are contaminated by salt or pepper noise (highlighted by red color)

11. EE465: Introduction to Digital Image Processing11 MATLAB Command >Y = IMNOISE(X,'salt & pepper',p) Notes:  The intensity of input images is assumed to be normalized to [0,1]. If X is double, you need to do normalization first, i.e., X=X/255; If X is uint8, MATLAB would do the normalization automatically  The default value of p is 0.05 (i.e., 5 percent of pixels are contaminated)  imnoise function can produce other types of noise as well (you need to change the noise type ‘ salt & pepper ’ )

12. EE465: Introduction to Digital Image Processing12 Impulse Noise Removal Problem Noisy image Y filtering algorithm Can we make the denoised image X as close to the noise-free image X as possible? ^ X ^ denoised image

13. EE465: Introduction to Digital Image Processing13 Median Operator Given a sequence of numbers {y 1, …,y N } Mean: average of N numbers Min: minimum of N numbers Max: maximum of N numbers Median: half-way of N numbers Example sorted

14. EE465: Introduction to Digital Image Processing14 y(n) W=2T+1 1D Median Filtering …… Note: median operator is nonlinear MATLAB command: x=median(y(n-T:n+T));

15. EE465: Introduction to Digital Image Processing15 Numerical Example T=1: Boundary Padding

16. EE465: Introduction to Digital Image Processing16 x(m,n) W:(2T+1)-by-(2T+1) window 2D Median Filtering MATLAB command: x=medfilt2(y,[2*T+1,2*T+1]);

17. EE465: Introduction to Digital Image Processing17 Numerical Example 225 225 225 226 226 226 226 226 225 225 255 226 226 226 225 226 226 226 225 226 0 226 226 255 255 226 225 0 226 226 226 226 225 255 0 225 226 226 226 255 255 225 224 226 226 0 225 226 226 225 225 226 255 226 226 228 226 226 225 226 226 226 226 226 0 225 225 226 226 226 226 226 225 225 226 226 226 226 226 226 225 226 226 226 226 226 226 226 226 226 225 225 226 226 226 226 225 225 225 225 226 226 226 226 225 225 225 226 226 226 226 226 226 226 226 226 226 226 226 226 Y X ^ Sorted: [0, 0, 0, 225, 225, 225, 226, 226, 226]

18. EE465: Introduction to Digital Image Processing18 Image Example P=0.1 Noisy image YX ^ denoised image 3-by-3 window

19. EE465: Introduction to Digital Image Processing19 Image Example (Con ’ t) 3-by-3 window5-by-5 window clean noisy (p=0.2)

20. EE465: Introduction to Digital Image Processing20 Reflections What is good about median operation? Since we know impulse noise appears as black (minimum) or white (maximum) dots, taking median effectively suppresses the noise What is bad about median operation? It affects clean pixels as well Noticeable edge blurring after median filtering

21. EE465: Introduction to Digital Image Processing21 Idea of Improving Median Filtering Can we get rid of impulse noise without affecting clean pixels? Yes, if we know where the clean pixels are or equivalently where the noisy pixels are How to detect noisy pixels? They are black or white dots

22. EE465: Introduction to Digital Image Processing22 Median Filtering with Noise Detection Noisy image Y x=medfilt2(y,[2*T+1,2*T+1]); Median filtering Noise detection C=(y==0)|(y==255); xx=c.*x+(1-c).*y; Obtain filtering results

23. EE465: Introduction to Digital Image Processing23 Image Example clean noisy (p=0.2) w/o noise detection with noise detection

24. EE465: Introduction to Digital Image Processing24 Image Denoising Introduction Impulse noise removal Median filtering Additive white Gaussian noise removal 2D convolution and DFT Periodic noise removal Band-rejection and Notch filter

25. EE465: Introduction to Digital Image Processing25 Additive White Gaussian Noise Definition Each pixel in an image is disturbed by a Gaussian random variable With zero mean and variance  2 X: noise-free image, Y: noisy image Note: unlike impulse noise situation, every pixel in the image contaminated by AWGN is noisy

26. EE465: Introduction to Digital Image Processing26 Numerical Example  2 =1 XY 128 128 128 128 128 128 128 128 128 128 129 127 129 126 126 128 126 128 128 129 129 128 128 127 128 128 128 129 129 127 127 128 128 129 127 126 129 129 129 128 127 127 128 127 129 127 129 128 129 130 127 129 127 129 130 128 129 128 129 128 128 128 129 129 128 128 130 129 128 127 127 126

27. EE465: Introduction to Digital Image Processing27 MATLAB Command >Y = IMNOISE(X, ’ gaussian',m,v) >Y = X+m+randn(size(X))*v; or Note: rand() generates random numbers uniformly distributed over [0,1] randn() generates random numbers observing Gaussian distribution N(0,1)

28. EE465: Introduction to Digital Image Processing28 Image Denoising Noisy image Y filtering algorithm Question: Why not use median filtering? Hint: the noise type has changed. X ^ denoised image

29. EE465: Introduction to Digital Image Processing29 f(n) h(n) g(n) - Linearity - Time-invariant property Linear convolution 1D Linear Filtering See review section

30. EE465: Introduction to Digital Image Processing30 forward inverse Note that the input signal is a discrete sequence while its FT is a continuous function time-domain convolutionfrequency-domain multiplication Fourier Series

31. EE465: Introduction to Digital Image Processing31 Filter Examples LP HP w |H(w)| Low-pass (LP) h(n)=[1,1] |h(w)|=2cos(w/2) High-pass (LP) h(n)=[1,-1] |h(w)|=2sin(w/2)

32. EE465: Introduction to Digital Image Processing32 forward transforminverse transform Properties - periodic - conjugate symmetric 1D Discrete Fourier Transform Proof:

33. EE465: Introduction to Digital Image Processing33 Matrix Representation of 1D DFT Re Im DFT:

34. EE465: Introduction to Digital Image Processing34 Fast Fourier Transform (FFT)* Invented by Tukey and Cooley in 1965 Basic idea: divide-and-conquer Reduce the complexity of N-point DFT from O(N 2 ) to O(Nlog 2 N) N/2-point DFT N-point DFT

35. EE465: Introduction to Digital Image Processing35 Filtering in the Frequency Domain convolution in the time domain is equivalent to multiplication in the frequency domain f(n) h(n) g(n)F(k) H(k) G(k) DFT

36. EE465: Introduction to Digital Image Processing36 2D Linear Filtering f(m,n) h(m,n) g(m,n) 2D convolution MATLAB function: C = CONV2(A, B)

37. EE465: Introduction to Digital Image Processing37 2D Filtering=Two Sequential 1D Filtering Just as we have observed with 2D transform, 2D (separable) filtering can be viewed as two sequential 1D filtering operations: one along row direction and the other along column direction The order of filtering does not matter h 1 : 1D filter

38. EE465: Introduction to Digital Image Processing38 Numerical Example h 1 (m)=[1,1], h 1 (n)=[1,-1] 1D filter MATLAB command: >h1=[1,1];h2=[1,-1]; >conv2(h1,h2) >conv2(h2,h1)

39. EE465: Introduction to Digital Image Processing39 Fourier Series (2D case) Note that the input signal is discrete while its FT is a continuous function spatial-domain convolutionfrequency-domain multiplication

40. EE465: Introduction to Digital Image Processing40 Filter Examples Low-pass (LP) h 1 (n)=[1,1] |h 1 (w)|=2cos(w/2) 1D h(n)=[1,1;1,1] |h(w 1,w 2 )|=4cos(w 1 /2)cos(w 2 /2) 2D w1w1 w2w2 |h(w 1,w 2 )|

41. EE465: Introduction to Digital Image Processing41 Image DFT Example Original ray image X choice 1: Y=fft2(X)

42. EE465: Introduction to Digital Image Processing42 Image DFT Example (Con ’ t) choice 1: Y=fft2(X)choice 2: Y=fftshift(fft2(X)) Low-frequency at the centerLow-frequency at four corners FFTSHIFT Shift zero-frequency component to center of spectrum.

43. EE465: Introduction to Digital Image Processing43 Gaussian Filter FT >h=fspecial( ‘ gaussian ’, HSIZE,SIGMA);MATLAB code:

44. EE465: Introduction to Digital Image Processing44 (  =1) PSNR=24.4dB Image Example PSNR=20.2dB noisy (  =25) denoised (  =1.5) PSNR=22.8dB Matlab functions: imfilter, filter2

45. 45 Gaussian Filter=Heat Diffusion Linear Heat Flow Equation: scale A Gaussian filter with zero mean and variance of t Isotropic diffusion:

46. 46 Basic Idea of Nonlinear Diffusion* x y I(x,y) image I image I viewed as a 3D surface (x,y,I(x,y)) Diffusion should be anisotropic instead of isotropic

47. Experimental Results EE465: Introduction to Digital Image Processing47 (Gaussian filtering) PSNR=24.4dBPSNR=20.2dB noisy (  =25) linear diffusion (TV filtering) PSNR=27.5dB nonlinear diffusion

48. Hammer-Nail Analogy EE465: Introduction to Digital Image Processing 48 Gaussian filter median filter salt-pepper/ impulse noise Gaussian noise periodic noise ???

49. EE465: Introduction to Digital Image Processing49 Image Denoising Introduction Impulse noise removal Median filtering Additive white Gaussian noise removal 2D convolution and DFT Periodic noise removal Band-rejection and Notch filter

50. EE465: Introduction to Digital Image Processing50 Periodic Noise Source: electrical or electromechanical interference during image acquistion Characteristics Spatially dependent Periodic – easy to observe in frequency domain Processing method Suppressing noise component in frequency domain

51. EE465: Introduction to Digital Image Processing51 Image Example spatial Frequency (note the four pairs of bright dots)

52. EE465: Introduction to Digital Image Processing52 Band Rejection Filter w1w1 w2w2

53. EE465: Introduction to Digital Image Processing53 Image Example Before filtering After filtering

54. Advanced Denoising Techniques* EE465: Introduction to Digital Image Processing54 Basic idea: from linear diffusion (equivalent to Gaussian filtering) to nonlinear diffusion (with implicit edge-stopping criterion)