1. CSC508 Convolution Operators

2. CSC508 Convolution Arguably the most fundamental operation of computer vision It’s a neighborhood operator –Similar to the median and outlier processes we discussed last week Utilizes a pattern of weights defined over the neighborhood –Also known as A filter A kernel

3. CSC508 Convolution Mathematically defined as The resultant image The input image neighborhood The kernel or filter u x v The size of the kernel H is convolved with F yielding R

4. CSC508 Convolution But what does it really mean? –It’s a “multiply/accumulate” operation 238237234227223216 229227224220225221 205212221220225220 177192213207212217 164180211208209215 190194220212210219 i j 012 345 678 Kernel Image

5. CSC508 Consider 1-Dimensional Input

6. CSC508 A 1-Dimensional Kernel

7. CSC508 Convolution in 1-Dimension

8. CSC508 1-Dimension Output

9. CSC508 Convolution We “slide” the kernel over the entire image –In two dimensions we just slide the kernel top to bottom and left to right Eventually it gets centered over every pixel –Note, we typically do this operation so that all pixels are handled in parallel (simultaneously) –It’s a fairly “expensive” operation What about the edges of the image? –Various techniques can be employed

10. CSC508 Convolution at the Edges Just ignore the edges –Initialize the resultant image to 0 (or something else) Use only the parts of the kernel that are within the image Enlarge the input image on all sides by reflection

11. CSC508 Enlarging by Reflection 238 237234227223216 238 237234227223216 229 227224220225221 205 212221220225220 177 192213207212217 164 180211208209215 190 194220212210219 190 194220212210219 Original image Assume a 3x3 kernel A larger kernel requires additional reflection

12. CSC508 Why Reflect? Why not just pad with 0 (or some other value)? We’ll see later when we look at edge detection

13. CSC508 Some Interesting Kernels Neighborhood averaging –A simple blurring function –Note that the results of the convolution may go out of range –Solution is to normalize the kernel –Various techniques Divide each kernel by the sum of all values Divide the result by the sum of all values 11111 11111 11111 11111 11111

14. CSC508 Neighborhood Averaging

15. CSC508 Some Interesting Kernels Emboss –Not really useful for computer vision but interesting none the less –Why don’t we need to normalize this kernel? –But, we do need to make sure the output does not go below 0 (since a kernel value is negative) 00 000 001

16. CSC508 Emboss

17. CSC508 Some Interesting Kernels Laplacian –A simple gradient detection mask –What is the range of the result? [-1020..1020] Need to do something about this (clamp or scale) 010 1-41 010

18. CSC508 Laplacian

19. CSC508 Some Interesting Kernels You can set the kernel weights to any values you want –Anything that will give you the desired effect –Selecting “meaningful” weights is an art

20. CSC508 Gaussian Kernel (Filter) The Gaussian filter is a smoothing or blurring filter Width is the number of pixels covered by the filter Sigma is the standard deviation of the Gaussian curve in pixels

21. Coding the Gaussian Use odd number of rows and columns in the kernel (e.g. 3x3, 5x5, 7x7…) Loop over every location in the kernel matrix –Translate integer indices from [0..width-1] to floating point [– width/2..+width/2] –This makes the floating point coordinate of the central value (0.0, 0.0) Perform the calculation given with the translated loop indices as the x and y values When done, normalize the kernel coefficients by dividing each coefficient by the sum of all coefficients CSC508

22. Gaussian Filter Sigma 1.0, Filter Width 7

23. CSC508 Gaussian Filter Sigma 1.6, Filter Width 7

24. CSC508 Gaussian Filter Sigma 1.6, Filter Width 15

25. CSC508 Gaussian is Separable Two 1D Gaussians produces the same results a one 2D Gaussian –First convolve the original image horizontally –Then convolve the horizontal results vertically –This speeds things up dramatically Reduces the total number of multiplies and additions

26. CSC508 Gaussian Filter – 1D

27. CSC508 Gaussian Filter – 1D Sigma 1.6, Filter Width 15

28. CSC508 Gaussian Filter Why would we want to blur an image? It will help us to extract gradient features (edges)

29. CSC508 Edge Detection What is an edge? –An intensity gradient –That is, a change of intensity within a localized region of the image (a neighborhood) The edge

30. CSC508 Edge Detection Is there an edge here? There is definitely a gradient but no edge –At least not over a small neighborhood

31. CSC508 Edge Detection Edges are described by magnitude which is related to the intensity on either side of the edge Weaker edge Strong edge

32. CSC508 Edge Detection Edges are described by orientation (direction) –Orientation is determined by the angle of the gradient and the intensity on either side of the gradient

33. CSC508 Edge Detection Various techniques –Differential operator Sobel –Templates Nevatia-Babu –Procedural Marr-Hildreth (Laplacian-Gaussian) Canny

34. CSC508 Sobel Edge Operator Starts with two convolution kernels 01 -202 01 -2 000 121 Perform two convolutions on the original image resulting in two intermediate images

35. CSC508 Sobel Edge Operator From the two convolved images you can now compute edge magnitude and direction –The magnitude will have to be scaled to 0..255 Unscaled values will be both + and - –The direction is typically scaled to a small number of bins i.e. 0..8 or 0..16

36. CSC508 Sobel Edge Operator Input Magnitude Encoded Direction If we were to zoom in on the corners we’d see other edge orientations present This edge is not missing, it’s just the same color as the background

37. CSC508 Nevatia-Babu Template Operator Six edge-oriented convolution kernels (templates) -100 0100 -100 0100 -100 0100 -100 0100 -100 0100 -10032100 -1007892100 -100 0100 -100 -9278100 -100 -32100 -3278100 -100-92092100 -100 -7832 -100 100 00000 -100 100 78-32 100920-92-100 32-78-100 100 32-100 100 92-78-100 100 0-100 10078-92-100 100-32-100 0-deg 30-deg60-deg 90-deg120-deg 150-deg

38. CSC508 Nevatia-Babu Template Operator Convolve each image pixel with all six kernels Select the mask that produces the maximum output –Assign the magnitude to the output of the maximal mask –Assign the direction to the orientation of the maximal mask –Direction is further modified by the sign of the maximal mask As you might imagine, this is a very time consuming operation

39. CSC508 Things To Do Programming homework assignment –Gaussian Convolution Reflect the edges of the image Implement as both a 2D convolution and 2 1D convolutions “prove” that they are equivalent through code demonstrations –Sobel Edge Operator You may write in any programming language you choose Deliverables: –Zipped images in email –Email the source code to reinhart@clunet.edu with the subject linereinhart@clunet.edu CSC508 PROGRAM 2 Due beginning of class in two weeks –(late assignments will be penalized 10%) –I will post test images Reading for Next Week –Still in chapter 8 –We’ll talk about Non-maximal edge suppression Edge following Hysteresis –We’ll experiment with two edge detection algorithms Marr-Hildreth (Laplacian-Gaussian/Difference of Gaussians) Canny (Differential)