1. Edge Detection and Geometric Primitive Extraction Jinxiang Chai

2. Outline Edge detection Geometric primitive extraction

3. Edge Detection What are edges in this image?

4. Edge Detection What are edges in this image?

5. Origin of Edges Edges are caused by a variety of factors depth discontinuity surface color discontinuity illumination discontinuity surface normal discontinuity

6. Edge Detection One of the most important vision problems - really easy for human - really difficult for computers - fundamental for computer vision (object recognition, 3D reconstruction, etc.) How to tell a pixel is on the edge?

7. Characterizing edges An edge is a place of rapid change in the image intensity function image intensity function (along horizontal scanline) first derivative edges are related to derivative

8. Gradient A vector variable - Direction of the maximum growth of the function - Magnitude of the growth - Perpendicular to the edge direction

9. Gradients

10. How to Calculate Gradient? How to compute ? Intensity Pixel column

11. The Good ole’ Taylor Series Subtracting the second from the first we obtain or…

12. Discrete Gradient Estimation For discrete functions, we can use the first order approximation of the gradient where h corresponds to the step size

13. Discrete Gradient Estimation For discrete functions, we can use the first order approximation of the gradient where h corresponds to the step size For our purposes, h corresponds to the width of 1 pixel =>

14. Discrete Gradient Estimation So how can we compute the image gradient efficiently? - Using our good old friend convolution! - Dropped off the “divide by 2” for speed considerations. - This only scales the gradient.

15. Taking the Discrete Derivative abs()

16. Basic Edge Detection Step 1 INPUT IMAGE 1) Edge Enhancement Horizontal [-1 0 1] Vertical [-1 0 1] T But isn’t edge detection susceptible to noise?

17. Basic Edge Detection Steps 1-2 INPUT IMAGE 2) Edge Enhancement Horizontal [-1 0 1] Vertical [-1 0 1] T 1) Noise Smoothing

18. Discrete Gradient Estimation Remember that the gradient is a vector and we have calculated the coefficients in the x and y directions at each point in the image After convolving, we get the magnitude of the gradient from at each point (pixel) from In practice, we often sum the absolute values of the components for computational efficiency

19. Basic Edge Detection (cont’d) INPUT IMAGE 1) Noise Smoothing 2) Edge Enhancement Horizontal [-1 0 1] Vertical [-1 0 1] T “GRADIENT” IMAGE

20. Basic Edge Detection (cont’d) INPUT IMAGE 1) Noise Smoothing 2) Edge Enhancement Horizontal [-1 0 1] Vertical [-1 0 1] T “GRADIENT” IMAGE What does the gradient image mean?

21. Basic Edge Detection (cont’d) INPUT IMAGE 1) Noise Smoothing 2) Edge Enhancement Horizontal [-1 0 1] Vertical [-1 0 1] T “GRADIENT” IMAGE What does the gradient image mean? - Magnitude of intensity changes around each pixel

22. Discrete Gradient Estimation So how do we segment the edges from the rest of the scene? - Thresholding!

23. The Effects of Thresholding While edge features are independent of illumination, the edge strength is not! Results from threshold values of 50 and 100

24. Basic Edge Detection Summary INPUT IMAGE 1) Noise Smoothing EDGE IMAGE 2) Edge Enhancement Horizontal [-1 0 1] Vertical [-1 0 1] T “GRADIENT” IMAGE 3)Threshold

25. The effects of Filtering Noise Threshold 20 Gaussian SmoothingUnsmoothed Edges Threshold 50

26. Sobel Edge Detection One of the dominant edge detection schemes uses the Sobel operators Convolving each of these with the original image generates horizontal and vertical gradient images that are combined as before

27. Sobel Edge Detection One of the dominant edge detection schemes uses the Sobel operators - Can be approximated as a derivative of Gaussian - First Gaussian smoothing and then compute derivatives

28. Sobel Edge Detection One of the dominant edge detection schemes uses the Sobel operators - Can be approximated as a derivative of Gaussian - First Gaussian smoothing and then compute derivatives Why?

29. Sobel Edge Detection One of the dominant edge detection schemes uses the Sobel operators - Can be approximated as a derivative of Gaussian - First Gaussian smoothing and then compute derivatives - In practice we may still need to smooth for noise

30. Robert and Prewitt Edge Detectors The Prewitt is similar to the Sobel, but uses a different kernel Roberts was an early edge detector kernel

31. Sobel Edge Detector Java Applet

32. Second Derivative Edge Detector Sobel Operator can produce thick edges; ideally we are looking for infinitely thin boundaries

33. Second Derivative Edge Detector Sobel Operator can produce thick edges; ideally we are looking for infinitely thin boundaries An alternative approach is to look for local extrema in the first derivative A peak in the first derivative corresponds what in the second derivative?

34. Second Derivative Edge Detector Sobel Operator can produce thick edges; ideally we are looking for infinitely thin boundaries An alternative approach is to look for local extrema in the first derivative A peak in the first derivative corresponds what in the second derivative?

35. Localization with Laplacian An equivalent measure of the second derivative in 2D is the Laplacian: Numerically, we approximate the Laplacian using the following filtering kernel: Zeros crossings of the filter corresponds to positions of maximum gradient. - can be used for detecting edges - might be sensitive to noise - need to filter the image

36. Laplacian of Gaussian In the LoG, there are two methods which are mathematically equivalent: - Convolve the image with a gaussian smoothing filter and compute the Laplacian of the result - Convolve the image with the linear filter that is the Laplacian of the Gaussian filter

37. The Laplacian of Gaussian Kernel

38. The Laplacian of Gaussian Filter Kernel

39. To avoid detection of insignificant edges, only the zero crossings whose corresponding first derivative is above some threshold, are selected as edge point. Laplacian Edge Detector Java applet: Click here here

40. Edge Detection Matlab functions for edge detection: - BW = edge(I,'sobel') % Sobel detector - BW = edge(I,'prewitt') % Prewitt detector - BW = edge(I,'roberts') % Robert detector - BW = edge(I,'log') % Laplacian of Gaussian detector - BW = edge(I,'canny') % canny detector

41. Outline Edge detection Geometric primitive extraction

42. Finding straight lines One solution: try many possible lines and see how many points each line passes through Hough transform provides a fast way to do this

43. Outline of Hough Transform 1.Create a grid of parameter values 2.Each point votes for a set of parameters, incrementing those values in grid 3.Find maximum or local maxima in grid

44. Finding lines using Hough transform Using m,b parameterization Using r, theta parameterization –Using oriented gradients Practical considerations –Bin size –Smoothing

45. Hough transform An early type of voting scheme General outline: –Discretize parameter space into bins –For each feature point in the image, put a vote in every bin in the parameter space that could have generated this point –Find bins that have the most votes P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959 Image space Hough parameter space

46. Parameter space representation A line in the image corresponds to a point in Hough space Image spaceHough parameter space Source: S. Seitz

47. Parameter space representation What does a point (x 0, y 0 ) in the image space map to in the Hough space? Image spaceHough parameter space

48. Parameter space representation What does a point (x 0, y 0 ) in the image space map to in the Hough space? –Answer: the solutions of b = –x 0 m + y 0 –This is a line in Hough space Image spaceHough parameter space

49. Parameter space representation Where is the line that contains both (x 0, y 0 ) and (x 1, y 1 )? Image spaceHough parameter space (x 0, y 0 ) (x 1, y 1 ) b = –x 1 m + y 1

50. Parameter space representation Where is the line that contains both (x 0, y 0 ) and (x 1, y 1 )? –It is the intersection of the lines b = –x 0 m + y 0 and b = –x 1 m + y 1 Image spaceHough parameter space (x 0, y 0 ) (x 1, y 1 ) b = –x 1 m + y 1

51. Problems with the (m,b) space: –Unbounded parameter domain –Vertical lines require infinite m Parameter space representation

52. Problems with the (m,b) space: –Unbounded parameter domain –Vertical lines require infinite m Alternative: polar representation Parameter space representation Each point will add a sinusoid in the ( ,  ) parameter space

53. Algorithm outline Initialize accumulator H to all zeros For each edge point (x,y) in the image For θ = 0 to 180 ρ = x cos θ + y sin θ H(θ, ρ) = H(θ, ρ) + 1 end end Find the value(s) of (θ, ρ) where H(θ, ρ) is a local maximum –The detected line in the image is given by ρ = x cos θ + y sin θ ρ θ

54. features votes Basic illustration

55. Square Circle Other shapes

56. Several lines

57. A more complicated image http://ostatic.com/files/images/ss_hough.jpg

58. featuresvotes Effect of noise

59. featuresvotes Effect of noise Peak gets fuzzy and hard to locate

60. Dealing with noise Choose a good grid / discretization –Too coarse: large votes obtained when too many different lines correspond to a single bucket –Too fine: miss lines because some points that are not exactly collinear cast votes for different buckets Increment neighboring bins (smoothing in accumulator array) Try to get rid of irrelevant features –Take only edge points with significant gradient magnitude

61. Incorporating image gradients Recall: when we detect an edge point, we also know its gradient direction But this means that the line is uniquely determined! Modified Hough transform: For each edge point (x,y) θ = gradient orientation at (x,y) ρ = x cos θ + y sin θ H(θ, ρ) = H(θ, ρ) + 1 end

62. Line detection Matlab functions for line detection: [H, theta, rho] = hough(BW)

63. Hough transform for Other Shapes How many dimensions will the parameter space have?

64. Hough transform for Other Shapes How many dimensions will the parameter space have? (x0,y0, r)(x0,y0, a,b,theta)

65. Things to remember Edge detector - First order method: smooth- >gradients->thresholding - Second order method: zero crossings for LOG Hough transform = points vote for shape parameters