1. Segmentation (Section 10.2)
CS474/674 – Prof. Bebis

2. Segmentation
The goal is to partition an image into regions (e.g., separate objects from background.
Segmentation results could determine the eventual success or failure of image analysis.
In general, this is a very difficult problem!

3. Segmentation (cont’d)
Improve segmentation results by
Introducing knowledge about the application domain
Controlling the environment (e.g., industrial applications)
Selecting special sensors to enhance the objects of interest (e.g., use thermal-infrared imaging for target recognition)

4. Segmentation Approaches
Edge-based approaches
Use the boundaries of regions to segment the image.
Detect abrupt changes in intensity (discontinuities).
Region-based approaches
Use similarity among pixels to find different regions.

5. Definition of Edges
Edges are significant local changes of intensity in an image.
Edges typically occur on the boundary between two different regions in an image.

6. Goal of Edge Detection
Produce a line “drawing” of a scene from an image of that scene.
Important features can be extracted from the edges of an image (e.g., corners, lines, curves).
These features are used by higher-level computer vision algorithms (e.g., recognition).

7. What Causes Intensity Changes?
Geometric events
Discontinuities in boundary or surface orientation and/or surface color and texture.
Non-geometric events
Illumination changes
Specularities
Shadows
Inter-reflections

8. Edge Descriptors
Edge normal: unit vector in the direction of maximum intensity change.
Edge direction: unit vector to perpendicular to the edge normal.
Edge position: the image position at which the edge is located.
Edge strength: related to the local image contrast along the normal.

9. Modeling Intensity Changes
Step edge: the image intensity abruptly changes from one value on one side of the discontinuity to a different value on the opposite side.

10. Modeling Intensity Changes (cont’d)
Ramp edge: a step edge where the intensity change is not instantaneous but occur over a finite distance.

11. Modeling Intensity Changes (cont’d)
Ridge edge: the image intensity abruptly changes value but then returns to the starting value within some short distance (i.e., usually generated by lines).

12. Modeling Intensity Changes (cont’d)
Roof edge: a ridge edge where the intensity change is not instantaneous but occur over a finite distance (i.e., usually generated by the intersection of two surfaces).

13. Main Steps in Edge Detection
(1) Smoothing: suppress as much noise as possible, without destroying true edges.
(2) Enhancement: apply a filter to enhance the quality of the edges in the image
i.e., sharpening

14. Main Steps in Edge Detection (cont’d)
(3) Detection: determine which edge pixels should be discarded as noise and which should be retained
i.e., threshold edge strength
(4) Localization: determine the exact location of an edge
i.e., sub-pixel resolution might be required for some applications, that is, estimate the location of an edge to better than the spacing between pixels.

15. Edge Detection Using Derivatives
Points that lie on an edge can
be detected by:
(1) detecting local maxima or
minima of the first derivative.
(2) detecting the zero-crossings
of the second derivative.

16. Edge Detection Using 1st Derivative – 1D
(centered at x)
upward step edge
ramp edge
downward step edge
roof edge

17. Edge Detection Using 2nd Derivative – 1D
Replace x+1 with x (i.e., centered at x):

18. Edge Detection Using 2nd Derivative – 1D (cont’d)

19. Edge Detection Using 2nd Derivative -1D (cont’d)
upward step edge
downward step edge
ramp edge
roof edge

20. Edge Detection Using 1st Derivative – 2D Gradient
Gradient is defined as follows:
Approximate gradient using finite differences:

21. Edge Detection Using 1st Derivative – 2D Gradient (cont’d)
In pixel-coordinate notation, j corresponds to the x direction and i to the negative y direction:

22. Edge Detection Using 1st Derivative – 2D Roberts Edge Detector

23. Edge Detection Using 1st Derivative – 2D Another approximation
Consider the arrangement of pixels about the pixel (i, j):
The partial derivatives can be computed by:
The constant c implies the emphasis given to pixels closer to the center of the mask.

24. Edge Detection Using 1st Derivative – 2D Prewitt Edge Detector
Setting c = 1, we get the Prewitt operator:

25. Edge Detection Using 1st Derivative – 2D Sobel Edge Detector
Setting c = 2, we get the Sobel operator:

26. Main Steps in Edge Detection
(i.e., sqrt is expensive!)

27. Isotropic property of gradient magnitude
The magnitude detects edges in all possible directions
magnitude

28. Example (using Prewitt)
(divisions by 2 are
for normalization only)

29. Some Practical Issues Noise suppression-localization tradeoff
Larger filters reduce noise, but worsen localization (i.e., it adds uncertainty to the location of the edge) and vice versa.
smaller filter
larger filter

30. Some Practical Issues (cont’d)
How should we choose the threshold?
gradient magnitude
low threshold
high threshold

31. Some Practical Issues (cont’d)
Edge thinning and linking are usually required.

32. Criteria for Optimal Edge Detection
(1) Good detection: the optimal detector must minimize the probability of false positives (detecting spurious edges caused by noise), as well as that of false negatives (missing real edges).
(2) Good localization: the edges detected must be as close as possible to the true edges.
(3) Single response constraint: the detector must return one point only for each true edge point; that is, minimize the number of local maxima around the true edge (created by noise).

33. The Canny edge detector
Canny has shown that the first derivative of the Gaussian closely approximates the operator that optimizes the product of signal-to-noise ratio and localization.
Canny’s analysis is based on "step-edges" corrupted by "additive Gaussian noise".

34. The Canny edge detector (cont’d)

35. The Canny edge detector (cont’d) Non-maxima suppression
Preserve local maxima of the gradient magnitude.
Suppress non-maxima along the direction of the gradient.
Thin edges!

36. The Canny edge detector (cont’d) Non-maxima suppression
(i,j)

37. The Canny edge detector (cont’d) Hysteresis thresholding/Edge Linking
The output of non-maxima suppression still contains local maxima created by noise.
Can we get rid of them just by using a single threshold?
If we set a low threshold, some noisy maxima will be accepted too.
If we set a high threshold, true maxima might be missed.
The value of true maxima will fluctuate above and below the threshold, fragmenting the edge.

38. The Canny edge detector (cont’d) Hysteresis thresholding/Edge Linking
gradient magnitude
low threshold
high threshold

39. The Canny edge detector (cont’d) Hysteresis thresholding/Edge Linking
A more effective scheme is to use two thresholds:
a low threshold tl
a high threshold th (usually, th = 2tl)
Note: large gaps are still difficult to bridge ...more powerful algorithms are needed!

40. Example – what is the effect of σ?

41. Example – what is the effect of σ?

42. Edge Detection Using 2nd Derivative – 2D Laplacian
mask:

43. Edge Detection Using 2nd Derivative – 2D Laplacian Example
detect zero-crossings

44. Edge Detection Using 2nd Derivative – 2D Properties of Laplacian
It is an isotropic operator.
It is cheaper to implement (i.e., one mask only).
It does not provide information about edge direction
It is more sensitive to noise (i.e., differentiates twice).

45. Edge Detection Using 2nd Derivative – 2D Laplacian of Gaussian (LoG)
To reduce the noise effect, the image is first smoothed with a low-pass filter.
In the case of the LoG, the low-pass filter is chosen to be a Gaussian.
It can be shown that:
σ controls smoothing

46. Edge Detection Using 2nd Derivative – 2D Laplacian of Gaussian (LoG) Example
convolution result
zero-crossings

47. Gradient vs LoG Comparison
Gradient works well when the image contains sharp intensity transitions and low noise.
Zero-crossings of LOG offer better localization, especially when the edges are not very sharp.
step edge
ramp edge
edge can be better
localized

48. Multiscale processing (scale space)
A serious practical problem with any edge detector is the matter of choosing the scale of smoothing (e.g.,the value of σ using a Gaussian).
For many applications, it is desirable to be able to process an image at multiple scales.
We determine which edges are most significant in terms of the range of scales over which they are observed to occur.

49. Multiscale processing (cont’d)

50. Multiscale processing (cont’d)