1. Image Processing Frequency Filtering Instructor: Juyong Zhang juyong@ustc.edu.cn http://staff.ustc.edu.cn/~juyong

2. Convolution Property of the Fourier Transform The Fourier Transform of a convolution equals the product of the Fourier Transforms. Similarly, the Fourier Transform of a convolution is the product of the Fourier Transforms * = convolution · = multiplication 5/5/20152

3. Convolution via Fourier Transform Image & Mask Transforms Pixel-wise Product Inverse Transform 5/5/20153

4. 1.Read the image from a file into a variable, say I. 2.Read in or create the convolution mask, h. 3.Compute the sum of the mask: s = sum(h(:)); 4.If s == 0, set s = 1; 5.Replace h with h = h/s; 6.Create: H = zeros(size(I)); 7.Copy h into the middle of H. 8.Shift H into position: H = ifftshift(H); 9.Take the 2D FT of I and H : FI=fft2(I); FH=fft2(H); 10.Pointwise multiply the FTs: FJ=FI.*FH; 11.Compute the inverse FT: J = real(ifft2(FJ)); How to Convolve via FT in Matlab For color images you may need to do each step for each band separately. The mask is usually 1-band 5/5/20154

5. Coordinate Origin of the FFT Center = (floor(R/2)+1, floor(C/2)+1) Center = (floor(R/2)+1, floor(C/2)+1) Even Odd Image OriginWeight Matrix OriginImage OriginWeight Matrix Origin After FFT shiftAfter IFFT shiftAfter FFT shiftAfter IFFT shift 5/5/20155

6. 564 897 231 123 456 789 Matlab’s fftshift and ifftshift J = fftshift(I): I ( 1, 1 )  J (  R/2  + 1,  C/2  + 1 ) I = ifftshift(J): J (  R/2  + 1,  C/2  + 1 )  I ( 1, 1 ) where  x  = floor(x) = the largest integer smaller than x. 123 456 789 564 897 231 5/5/20156

7. Blurring: Averaging / Lowpass Filtering Blurring results from: l Pixel averaging in the spatial domain: – Each pixel in the output is a weighted average of its neighbors. – Is a convolution whose weight matrix sums to 1. l Lowpass filtering in the frequency domain: – High frequencies are diminished or eliminated – Individual frequency components are multiplied by a nonincreasing function of  such as 1/  = 1/  (u 2 +v 2 ). The values of the output image are all non-negative. 5/5/20157

8. Sharpening: Differencing / Highpass Filtering Sharpening results from adding to the image, a copy of itself that has been: l Pixel-differenced in the spatial domain: – Each pixel in the output is a difference between itself and a weighted average of its neighbors. – Is a convolution whose weight matrix sums to 0. l Highpass filtered in the frequency domain: – High frequencies are enhanced or amplified. – Individual frequency components are multiplied by an increasing function of  such as  =  (u 2 +v 2 ), where  is a constant. The values of the output image positive & negative. 5/5/20158

9. Convolution Property of the Fourier Transform The Fourier Transform of a convolution equals the product of the Fourier Transforms. Similarly, the Fourier Transform of a convolution is the product of the Fourier Transforms * = convolution · = multiplication Recall: 5/5/20159

10. 10 Ideal Lowpass Filter

11. Fourier Domain Rep. Spatial Representation Central Profile Image size: 512x512 FD filter radius: 16 Image size: 512x512 FD filter radius: 16 Multiply by this, or … … convolve by this 5/5/201511

12. Spatial Representation Central Profile Ideal Lowpass Filter Image size: 512x512 FD filter radius: 8 Image size: 512x512 FD filter radius: 8 Multiply by this, or … … convolve by this 5/5/201512 Fourier Domain Rep.

13. Power Spectrum and Phase of an Image Consider the image below: Original Image Power Spectrum Phase 5/5/201513

14. Ideal LPF in FD Original Image Power Spectrum Ideal Lowpass Filter Image size: 512x512 FD filter radius: 16 Image size: 512x512 FD filter radius: 16 5/5/201514

15. Filtered Power Spectrum Ideal Lowpass Filter Image size: 512x512 FD filter radius: 16 Image size: 512x512 FD filter radius: 16 5/5/201515 Filtered Image Original Image

16. 5/5/201516 Ideal Highpass Filter

17. Fourier Domain Rep. Spatial Representation Central Profile Image size: 512x512 FD notch radius: 16 Image size: 512x512 FD notch radius: 16 Multiply by this, or … … convolve by this 5/5/201517

18. Ideal HPF in FD Original Image Power Spectrum Ideal Highpass Filter Image size: 512x512 FD notch radius: 16 Image size: 512x512 FD notch radius: 16 5/5/201518

19. Original Image Filtered Image* Filtered Power Spectrum Ideal Highpass Filter Image size: 512x512 FD notch radius: 16 Image size: 512x512 FD notch radius: 16 5/5/201519 *signed image: 0 mapped to 128

20. Filtered Image* Positive Pixels Negative Pixels Ideal Highpass Filter Image size: 512x512 FD notch radius: 16 Image size: 512x512 FD notch radius: 16 *signed image: 0 mapped to 128 5/5/201520

21. 5/5/201521 Ideal Bandpass Filter

22. 5/5/201522 A bandpass filter is created by (1)subtracting a FD radius  2 lowpass filtered image from a FD radius  1 lowpass filtered image, where  2 <  1, or (2)convolving the image with a mask that is the difference of the two lowpass masks. FD LP mask with radius  1 FD LP mask with radius  2 FD BP mask  1 -  2 - =

23. Ideal Bandpass Filter 5/5/201523 *signed image: 0 mapped to 128 image LPF radius  1 image LPF radius  2 image BPF radii  1,  2 *

24. Ideal Bandpass Filter original image* filter power spectrum filtered image 5/5/201524 *signed image: 0 mapped to 128

25. A Different Ideal Bandpass Filter original image filter power spectrum filtered image* 5/5/201525 *signed image: 0 mapped to 128

26. 5/5/201526 The Optimal Filter

27. The Uncertainty Relation FT spacefrequency A small object in space has a large frequency extent and vice-versa. 5/5/201527 spacefrequency FT

28. The Uncertainty Relation Recall: a symmetric pair of impulses in the frequency domain becomes a sinusoid in the spatial domain. A symmetric pair of lines in the frequency domain becomes a sinusoidal line in the spatial domain. 5/5/201528 space frequency  small extent   large extent  IFT space frequency  large extent   small extent   large extent  IFT

29. Ideal Filters Do Not Produce Ideal Results A sharp cutoff in the frequency domain… …causes ringing in the spatial domain. IFT 5/5/201529

30. Ideal Filters Do Not Produce Ideal Results Ideal LPF Blurring the image above with an ideal lowpass filter… …distorts the results with ringing or ghosting. 5/5/201530

31. Optimal Filter: The Gaussian The Gaussian filter optimizes the uncertainty relation. It provides the sharpest cutoff with the least ringing. IFT 5/5/201531

32. One-Dimensional Gaussian 5/5/201532

33. Two-Dimensional Gaussian If  and  are different for r & c … …or if  and  are the same for r & c. r c R = 512, C = 512  = 257,  = 64 5/5/201533

34. Gaussian LPF With a gaussian lowpass filter, the image above … … is blurred without ringing or ghosting. Optimal Filter: The Gaussian 5/5/201534

35. Compare with an “Ideal” LPF Ideal LPF Blurring the image above with an ideal lowpass filter… …distorts the results with ringing or ghosting. 5/5/201535

36. 5/5/201536 Gaussian Lowpass Filter

37. Fourier Domain Rep. Spatial Representation Central Profile Image size: 512x512 SD filter sigma = 8 Image size: 512x512 SD filter sigma = 8 Gaussian Lowpass Filter Multiply by this, or … … convolve by this 5/5/201537

38. Fourier Domain Rep. Spatial Representation Central Profile Image size: 512x512 SD filter sigma = 2 Image size: 512x512 SD filter sigma = 2 Multiply by this, or … … convolve by this 5/5/201538 Gaussian Lowpass Filter

39. Gaussian LPF in FD Original Image Power Spectrum Image size: 512x512 SD filter sigma = 8 Image size: 512x512 SD filter sigma = 8 5/5/201539 Gaussian Lowpass Filter

40. Filtered Image Original Image Filtered Power Spectrum Image size: 512x512 SD filter sigma = 8 Image size: 512x512 SD filter sigma = 8 5/5/201540 Gaussian Lowpass Filter

41. 5/5/201541 Gaussian Highpass Filter

42. Fourier Domain Rep. Spatial Representation Central Profile Image size: 512x512 FD notch sigma = 8 Image size: 512x512 FD notch sigma = 8 Multiply by this, or … … convolve by this 5/5/201542

43. Gaussian HPF in FD Original Image Power Spectrum Gaussian Highpass Filter Image size: 512x512 FD notch sigma = 8 Image size: 512x512 FD notch sigma = 8 5/5/201543

44. Filtered Image* Filtered Power Spectrum Gaussian Highpass Filter Image size: 512x512 FD notch sigma = 8 Image size: 512x512 FD notch sigma = 8 *signed image: 0 mapped to 128 Original Image 5/5/201544

45. Negative Pixels Positive Pixels Gaussian Highpass Filter Image size: 512x512 FD notch sigma = 8 Image size: 512x512 FD notch sigma = 8 Filtered Image* 5/5/201545 *signed image: 0 mapped to 128

46. Another Gaussian Highpass Filter original image filter power spectrum filtered image * 5/5/201546 *signed image: 0 mapped to 128

47. 5/5/201547 Gaussian Bandpass Filter

48. 5/5/201548 A bandpass filter is created by (1)subtracting a FD radius  2 lowpass filtered image from a FD radius  1 lowpass filtered image, where  2 <  1, or (2)convolving the image with a mask that is the difference of the two lowpass masks. FD LP mask with radius  1 FD LP mask with radius  2 FD BP mask  1 -  2 - =

49. Ideal Bandpass Filter original image filter power spectrum filtered image* 5/5/201549 *signed image: 0 mapped to 128

50. Gaussian Bandpass Filter 5/5/201550 *signed image: 0 mapped to 128 image LPF radius  1 image LPF radius  2 image BPF radii  1,  2 *

51. Gaussian Bandpass Filter Fourier Domain Rep. Spatial Representation Central Profile Image size: 512x512 sigma = 2 - sigma = 8 Image size: 512x512 sigma = 2 - sigma = 8 5/5/201551

52. Gaussian BPF in FD Original Image Power Spectrum Gaussian Bandpass Filter Image size: 512x512 sigma = 2 - sigma = 8 Image size: 512x512 sigma = 2 - sigma = 8 5/5/201552

53. Original Image Filtered Image * Filtered Power Spectrum Gaussian Bandpass Filter Image size: 512x512 sigma = 2 - sigma = 8 Image size: 512x512 sigma = 2 - sigma = 8 5/5/201553 *signed image: 0 mapped to 128

54. Filtered Image* Negative Pixels Filtered Image Positive Pixels Gaussian Bandpass Filter Image size: 512x512 sigma = 2 - sigma = 8 Image size: 512x512 sigma = 2 - sigma = 8 5/5/201554 *signed image: 0 mapped to 128

55. 5/5/201555 Ideal vs. Gaussian Filters

56. Original Image Ideal HPF * Ideal LPF Ideal Lowpass and Highpass Filters 5/5/201556 *signed image: 0 mapped to 128

57. Original Image Gaussian HPF* Gaussian LPF 5/5/201557 *signed image: 0 mapped to 128 Gaussian Lowpass and Highpass Filters

58. Original Image Gaussian BPF * Ideal BPF * 5/5/201558 *signed image: 0 mapped to 128 Ideal and Gaussian Bandpass Filters

59. Original Image Ideal BPF* Gaussian BPF* 5/5/201559 *signed image: 0 mapped to 128 Gaussian and Ideal Bandpass Filters

60. 5/5/201560 Effects on Power Spectrum

61. Power Spectrum and Phase of an Image original image power spectrum phase 5/5/201561

62. Power Spectrum and Phase of a Blurred Image 5/5/201562 blurred image power spectrum phase

63. Power Spectrum and Phase of an Image original image power spectrum phase 5/5/201563

64. Power Spectrum and Phase of a Sharpened Image 5/5/201564 power spectrum phase sharpened image

65. 5 May 201565 Thanks!