1. CSE 6367 Computer Vision Image Operations and Filtering “You cannot teach a man anything, you can only help him find it within himself.” ― Galileo GalileiGalileo Galilei Farhad Kamangar Computer Science and Engineering Department The University of Texas at Arlington

2. What Is An Image? Grayscale image: –Think of an image as a function f( x, y ) is the intensity at position ( x, y ) In practice, an image only is defined over a finite A color image is three functions

3. Images as A Matrix Digital images:: Sample the 2D space on a regular grid 0.25 0.32 0.940.73 0.740.52 0.54 R G B row column

4. Neighborhood operations If we shuffle all pixels in an image then the image can not be recognized (However the histogram of the image will not change) Image information is stored in local neighborhoods. We can perform image operations on local neighborhoods. Example: Replace each pixel with the average of all the neighboring pixels.

5. Moving Average 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 20 10 10010 2000010 20030 10 Source: S. Seitz f( x, y ) g( x, y )

6. Correlation Filtering an image means replacing each pixel in the original image with a linear combination of that pixel and its neighbors. The filter “template” or “kernel” or “mask” H[u,v] is the values for the weights in the linear combination. The kernel size is 2k+1 by 2k+1 and the center element is assumed to be element (0,0): g( x, y ) h(u,v) “kernel” or “mask” f( x, y )

7. Convolution Same as cross-correlation, except that the kernel is “flipped” (horizontally and vertically) This is called a convolution operation:

8. Convolution vs Correlation Cross correlation is: Not commutative Associative No Identity Convolution is: Commutative Associative Identity

9. Moving Average as Correlation 20 10 10010 2000010 20030 10 g( x, y ) 1/9 h(u,v) “kernel” or “mask” f( x, y ) cross-correlation

10. What is the gradient? No Change in horizontal direction Change in vertical direction

11. Gradient

12. How do we calculate gradient on a digital image F(x,y)? We can subtract a pixel value from it’s neighbor. Question: Which neighbor?

13. Gradient Can we compute gradients with cross correlation? Correlation with [-.5, 0,.5] Correlation with [-.5; 0;.5] for derivative in y direction.

14. Gradient Convolution Masks

15. Roberts operator

16. Sobel Common Sobel operators are:

17. Prewit Operator

18. What is an Edge? Edge is Where Change Occurs The changes due to noise are not edges Edges in real images are not perfectly sharp Change is measured by derivative Biggest change, derivative has maximum magnitude Or 2 nd derivative is zero.

19. Edges Edge detection is the process of finding edges in an image A contour a curve that passes through neighboring edge points Edge following is the process of searching the image to determine contours

20. Derivative Around an Edge If the edge is noisy then derivative is high everywhere. Smooth before taking derivative.

21. Implementation of 1D Edge Detection

22. Efficient Implementation of Edge Detection

23. Use of Second Derivative Edge points are located at the peaks of of the first derivative peaks of the first derivative A threshold is used to detect an edge. Alternative approach is to use the zero crossing of the second derivative. The Laplacian may be used as a second derivative. The Laplacian is symmetric in all directions and is invariant to rotation in the image.

24. Laplacian Operator Two dimensional equivalent of the second derivative For digital images Mask

25. Laplacian of Gaussian The second derivative creates zero crossings in the presence of noise. The image should be smoothed before edge detection The Laplacian of a Gaussian LoG smoothes image by combining Gaussian filtering and Laplacian operator

26. Laplacian of Gaussian Laplacian of Gaussian operator Zero-crossings is the edge position

27. Discrete LoG

28. DoG vs LoG Difference of two Gaussians (DoG) can approximate Laplacian of Gaussian many filters with varying σ DoG is implemented by subtracting two Gaussian filters with different σ

29. Edge detection by subtraction Laplacian of Gaussian Gaussian delta function

30. Kirsch edge patterns

31. Canny Edge Detector Smooth the image to remove the noise (Gaussian filter) Find the gradients of the image Apply non-maximum suppression Apply double threshold Select edge pixels by hysteresis (suppress all edges that are weak and not connected to strong edges).

32. Canny Edge Detector Find points along the curve where the gradient magnitude is biggest.

33. Non-Maxima Supression Find points along the curve where the gradient magnitude is biggest.