1. Computer Vision Image Features Instructor: Dr. Sherif Sami Lecture 4

2. 2 Image Features Image features – may appear in two contexts: −Global properties of the image (average gray level, etc) – global features −Parts of the image with special properties (line, circle, textured region) – local features Here, assume second context for image features: −Local, meaningful, detectable parts of the image Detection of image features −Detection algorithms – produce feature descriptors −Example – line segment descriptor: coordinates of mid-point, length, orientation

3. 3 Edge Detection Definition of edges −Edges are significant local changes of intensity in an image. −Edges typically occur on the boundary between two different regions in an image.

4. 4 Edge Detection Examples:

5. 5 Edge Detection Goal of edge detection −Produce a line drawing of a scene from an image of that scene. −Important features can be extracted from the edges of an image (e.g., corners, lines, curves). −These features are used by higher-level computer vision algorithms (e.g., segmentation, recognition).

6. 6 Edge Detection Not that easy:

7. 7 Edge Detection What causes intensity changes? −Geometric events −object boundary (discontinuity in depth and/or surface color and texture) −surface boundary (discontinuity in surface orientation and/or surface color and texture) −Non-geometric events −specularity −shadows (from other objects or from the same object) −inter-reflections

8. 8 Edge Detection Edge descriptors −Edge normal: unit vector in the direction of maximum intensity change. −Edge direction: unit vector to perpendicular to the edge normal. −Edge position or center: the image position at which the edge is located. −Edge strength: related to the local image contrast along the normal.

9. 9 Modeling Intensity Changes Edges can be modeled according to their intensity profiles: Step edge: −the image intensity abruptly changes from one value to one side of the discontinuity to a different value on the opposite side. Ramp edge: −a step edge where the intensity change is not instantaneous but occurs over a finite distance.

10. 10 Modeling Intensity Changes Ridge edge: −the image intensity abruptly changes value but then returns to the starting value within some short distance −generated usually by lines

11. 11 Modeling Intensity Changes Roof edge: −a ridge edge where the intensity change is not instantaneous but occurs over a finite distance −generated usually by the intersection of surfaces

12. 12 Edge Detection The four steps of edge detection: −Smoothing: suppress as much noise as possible, without destroying the true edges. −Enhancement: apply a filter that responds to edges in the image −Detection: determine which edge pixels should be discarded as noise and which should be retained (usually, thresholding provides the criterion used for detection). −Localization: determine the exact location of an edge (sub-pixel resolution might be required for some applications, that is, estimate the location of an edge to better than the spacing between pixels). Edge thinning and linking are usually required in this step.

13. 13 Edge Detection Edge detection using derivatives −Calculus describes changes of continuous functions using derivatives. −An image is a 2D function, so operators describing edges are expressed using partial derivatives. −Points which lie on an edge can be detected by either: −detecting local maxima or minima of the first derivative −detecting the zero-crossing of the second derivative

14. 14 Edge Detection Edge detection using derivatives – cont.

15. 15 Edge Detection Differencing 1D signals: −To compute the derivative of a signal, we approximate the derivative by finite differences Computing the 1 st derivative:

16. 16 Edge Detection Computing the 1 st derivative – cont. Backward difference Forward difference Central difference

17. 17 Edge Detection Computing the 1 st derivative – cont. −Examples using the edge models and the mask [ -1 0 1] (centered about x):

18. 18 Edge Detection Computing the 2 nd derivative: −This approximation is centered about x + 1 −By replacing x + 1 by x we obtain:

19. 19 Edge Detection Computing the 2 nd derivative – cont.

20. 20 Edge Detection Computing the 2 nd derivative – cont. −Examples using the edge models:

21. 21 Edge Detection Image derivatives: Image I

22. 22 Edge Detection Image derivatives: Image I

23. 23 Derivatives and Noise Derivatives are strongly affected by noise −obvious reason: image noise results in pixels that look very different from their neighbors −The larger the noise - the stronger the response What is to be done? −Neighboring pixels look alike −Pixel along an edge look alike −Image smoothing should help −Force pixels different to their neighbors (possibly noise) to look like neighbors

24. 24 Derivatives and Noise Increasing noise −Need to perform image smoothing as a preliminary step −Generally – use Gaussian smoothing

25. 25 Edge Detection Gradient operators −Roberts −Prewitt −Sobel Gradient of Gaussian (Canny) Laplacian of Gaussian (Marr-Hildreth) Facet Model Based Edge Detector (Haralick) Possible detectors:

26. 26 Edge Detection Using the Gradient Definition of the gradient: To save computations, the magnitude of gradient is usually approximated by:

27. 27 Edge Detection Using the Gradient Properties of the gradient: −The magnitude of gradient provides information about the strength of the edge −The direction of gradient is always perpendicular to the direction of the edge Main idea: −Compute derivatives in x and y directions −Find gradient magnitude −Threshold gradient magnitude

28. 28 Edge Detection Using the Gradient Estimating the gradient with finite differences −Approximation by finite differences:

29. 29 Edge Detection Using the Gradient Using pixel-coordinate notation (remember: j corresponds to the x direction and i to the negative y direction):

30. 30 Edge Detection Using the Gradient Example: −Suppose we want to approximate the gradient magnitude at z 5 We can implement  I/  x and  I/  y using the following masks: Note: M x is the approximation at (i, j + 1/2) and M y is the approximation at (i + 1/2, j)

31. 31 Edge Detection Using the Gradient The Roberts edge detector This approximation can be implemented by the following masks: Note: M x and M y are approximations at (i + 1/2, j + 1/2)

32. 32 Edge Detection Using the Gradient The Prewitt edge detector −The partial derivatives can be computed by: Note: M x and M y are approximations at (i, j)) −Consider the arrangement of pixels about the pixel (i, j): −The constant c implies the emphasis given to pixels closer to the center of the mask. −Setting c = 1, we get the Prewitt operator:

33. 33 Edge Detection Using the Gradient The Sobel edge detector Note: M x and M y are approximations at (i, j)) −Setting c = 2, we get the Sobel operator:

34. 34 Edge Detection Using the Gradient Main steps in edge detection using masks:

35. 35 Edge Detection Using the Gradient (an example using the Prewitt edge detector - don’t divide by 2)

36. 36 Edge Detection Using the Gradient Example:

37. 37 Edge Detection Using the Gradient Example – cont.:

38. 38 Edge Detection Using the Gradient Example – cont.:

39. 39 Edge Detection Using the Gradient

40. 40 Edge Detection Using the Gradient

41. 41 Edge Detection Using the Gradient Isotropic property of gradient magnitude: −The magnitude of gradient is an isotropic operator (it detects edges in any direction !!)

42. 42 Edge Detection Practical issues: −Differential masks act as high-pass filters – tend to amplify noise. −Reduce the effects of noise - first smooth with a low-pass filter. 1)The noise suppression-localization tradeoff −a larger filter reduces noise, but worsens localization (i.e., it adds uncertainty to the location of the edge) and vice-versa.

43. 43 Edge Detection 2)How should we choose the threshold?

44. 44 Edge Detection 3)Edge thinning and linking −required to obtain good contours

45. 45 Edge Detection −Good detection: the optimal detector must minimize the probability of false positives (detecting spurious edges caused by noise), as well as that of false negatives (missing real edges) −Good localization: the edges detected must be as close as possible to the true edges. −Single response constraint: the detector must return one point only for each true edge point; that is, minimize the number of local maxima around the true edge Criteria for optimal edge detection:

46. 46 Edge Detection Examples: True edge Poor robustness to noise Poor localization Too many responses

47. 47 The Canny Edge Detector −This is probably the most widely used edge detector in computer vision. −Canny has shown that the first derivative of the Gaussian closely approximates the operator that optimizes the product of signal-to- noise ratio and localization. −His analysis is based on "step-edges" corrupted by "additive Gaussian noise". The Canny edge detector:

48. 48 The Canny Edge Detector

49. 49 The Canny Edge Detector The derivative of the Gaussian:

50. 50 The Canny Edge Detector Canny – smoothing and derivatives:

51. 51 The Canny Edge Detector Canny – gradient magnitude: imagegradient magnitude

52. 52 Edge Detection −To find the edge points, we need to find the local maxima of the gradient magnitude. −Broad ridges must be thinned so that only the magnitudes at the points of greatest local change remain. −All values along the direction of the gradient that are not peak values of a ridge are suppressed. Non-maxima suppression

53. 53 Edge Detection Non-maxima suppression – cont.

54. 54 Edge Detection Non-maxima suppression – cont. −What are the neighbors? −Look along gradient normal −Quantization of normal directions: 0 1 2 3

55. 55 Edge Detection Canny – Non-maxima suppression: gradient magnitudethinned

56. 56 Edge Detection −The output of non-maxima suppression still contains the local maxima created by noise. −Can we get rid of them just by using a single threshold? −if we set a low threshold, some noisy maxima will be accepted too. −if we set a high threshold, true maxima might be missed (the value of true maxima will fluctuate above and below the threshold, fragmenting the edge). Hysteresis thresholding / Edge linking −A more effective scheme is to use two thresholds: −a low threshold t l −a high threshold t h −usually, t h ≈ 2t l

57. 57 Edge Detection Hysteresis thresholding / Edge linking – cont. −The algorithm performs edge linking as a by-product of double- thresholding !!

58. 58 Edge Detection Canny – Hysteresis thresholding / Edge linking thinned

59. 59 Edge Detection

60. 60 Edge Detection