1. Images and Filters CSEP 576 Ali Farhadi Many slides from Steve Seitz and Larry Zitnick

2. What is an image?

5. F() = Image Operations (functions of functions)

6. F() = Image Operations (functions of functions)

7. F() = Image Operations (functions of functions) 0.1 0 0.8 0.9 0.2 0.4 0.3 0.6 0 0 0.1 0.5 0.9 0.2 0.4 0.3 0.6 0 0 0.1 0.9 0.2 0.4 0.3 0.6 0 0 0.1 0.5

8. F(, ) = Image Operations (functions of functions) 0.23

9. Local image functions F() =

10. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 0 0000000000 0000000000 000 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Credit: S. Seitz Image filtering 111 111 111

11. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 010 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz

12. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 01020 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz

13. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 0102030 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz

14. 0102030 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz

15. 0102030 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz ?

16. 0102030 50 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz ?

17. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 0102030 2010 0204060 4020 0306090 6030 0 5080 906030 0 5080 906030 0203050 604020 102030 2010 00000 Image filtering 111 111 111 Credit: S. Seitz

18. What does it do? Replaces each pixel with an average of its neighborhood Achieve smoothing effect (remove sharp features) 111 111 111 Slide credit: David Lowe (UBC) Box Filter

19. Smoothing with box filter

20. Practice with linear filters 000 010 000 Original ? Source: D. Lowe

21. Practice with linear filters 000 010 000 Original Filtered (no change) Source: D. Lowe

22. Practice with linear filters 000 100 000 Original ? Source: D. Lowe

23. Practice with linear filters 000 100 000 Original Shifted left By 1 pixel Source: D. Lowe

24. Practice with linear filters Original 111 111 111 000 020 000 - ? (Note that filter sums to 1) Source: D. Lowe

25. Practice with linear filters Original 111 111 111 000 020 000 - Sharpening filter - Accentuates differences with local average Source: D. Lowe

26. Sharpening Source: D. Lowe

27. Other filters 01 -202 01 Vertical Edge (absolute value) Sobel

28. Other filters -2 000 121 Horizontal Edge (absolute value) Sobel

29. Basic gradient filters 000 10 000 0 0 000 010 10 or Horizontal GradientVertical Gradient 10 or

30. Gaussian filter * = Input image f Filter h Output image g Compute empirically

31. Gaussian vs. mean filters What does real blur look like?

32. Spatially-weighted average 0.003 0.013 0.022 0.013 0.003 0.013 0.059 0.097 0.059 0.013 0.022 0.097 0.159 0.097 0.022 0.013 0.059 0.097 0.059 0.013 0.003 0.013 0.022 0.013 0.003 5 x 5,  = 1 Slide credit: Christopher Rasmussen Important filter: Gaussian

33. Smoothing with Gaussian filter

34. Smoothing with box filter

35. Gaussian filters What parameters matter here? Variance of Gaussian: determines extent of smoothing Source: K. Grauman

36. Smoothing with a Gaussian … Parameter σ is the “scale” / “width” / “spread” of the Gaussian kernel, and controls the amount of smoothing. Source: K. Grauman

37. First and second derivatives * =

38. Original First Derivative xSecond Derivative x, y What are these good for?

39. Subtracting filters OriginalSecond Derivative Sharpened

40. for some Combining filters * * 00000 00000 0010 00000 00000 00000 00 00 0 4 0 00 00 00000 = = It’s also true:

41. Combining Gaussian filters More blur than either individually (but less than ) * = ?

42. Separable filters * = Compute Gaussian in horizontal direction, followed by the vertical direction. Not all filters are separable. Freeman and Adelson, 1991 Much faster!

43. Sums of rectangular regions 243 239240225206185188218211206216225 242239218110673134152213206208221 24324212358948213277108208 215 23521711521224323624713991209208211 23320813122221922619611474208213214 23221713111677150695652201228223 232 18218618417915912393232235 23223620115421613312981175252241240 2352382301281721386563234249241245 23723624714359781094255248247251 2342372451935533115144213255253251 2482451611281491091386547156239255 1901073910294731145817751137 233233148168203179432717128 172612160255 1092226193524 How do we compute the sum of the pixels in the red box? After some pre-computation, this can be done in constant time for any box. This “trick” is commonly used for computing Haar wavelets (a fundemental building block of many object recognition approaches.)

44. Sums of rectangular regions The trick is to compute an “integral image.” Every pixel is the sum of its neighbors to the upper left. Sequentially compute using:

45. Sums of rectangular regions AB CD Solution is found using: A + D – B - C

46. Spatially varying filters Some filters vary spatially. Useful for deblurring. Durand, 02

47. * * * input output Same Gaussian kernel everywhere. Slides courtesy of Sylvian Paris Constant blur

48. * * * input output The kernel shape depends on the image content. Slides courtesy of Sylvian Paris Bilateral filter Maintains edges when blurring!

49. Borders What to do about image borders: blackfixed periodic reflected