1. Chapter 4 Image Segmentation

2. Outline Introduction to segmentation Point,line and edge detection
Thresholding techniques
Watershed segmentation
Region based segmentation
Applications 2

3. Preview
Segmentation is to subdivide an image into its component regions or objects.
Segmentation should stop when the objects of interest in an application have been isolated. 3

4. Principal approaches
Segmentation algorithms generally are based on one of 2 basis properties of intensity values
discontinuity : to partition an image based on sharp changes in intensity (such as edges)
similarity : to partition an image into regions that are similar according to a set of predefined criteria. 4

5. Detection of Discontinuities
detect the three basic types of gray-level discontinuities
points , lines , edges
the common way is to run a mask through the image 5

6. Point Detection
a point has been detected at the location on which the mark is centered if
|R|  T
where
T is a nonnegative threshold
R is the sum of products of the coefficients with the gray levels contained in the region encompassed by the mark. 6

7. Point Detection
Note that the mark is the same as the mask of Laplacian Operation
The only differences that are considered of interest are those large enough (as determined by T) to be considered isolated points.
|R|  T 7

8. Example 8

9. Line Detection
Horizontal mask will result with max response when a line passed through the middle row of the mask with a constant background.
the similar idea is used with other masks.
note: the preferred direction of each mask is weighted with a larger coefficient (i.e.,2) than other possible directions. 9

10. Line Detection Apply every masks on the image
let R1, R2, R3, R4 denotes the response of the horizontal, +45 degree, vertical and -45 degree masks, respectively.
if, at a certain point in the image
|Ri| > |Rj|,
for all ji, that point is said to be more likely associated with a line in the direction of mask i. 10

11. Line Detection
Alternatively, if we are interested in detecting all lines in an image in the direction defined by a given mask, we simply run the mask through the image and threshold the absolute value of the result.
The points that are left are the strongest responses, which, for lines one pixel thick, correspond closest to the direction defined by the mask. 11

12. Example 12

13. Edge Detection we discussed approaches for implementing
first-order derivative (Gradient operator)
second-order derivative (Laplacian operator) 13

14. Ideal and Ramp Edges
because of optics, sampling, image acquisition imperfection 14

15. Thick edge
The slope of the ramp is inversely proportional to the degree of blurring in the edge.
We no longer have a thin (one pixel thick) path.
Instead, an edge point now is any point contained in the ramp, and an edge would then be a set of such points that are connected.
The thickness is determined by the length of the ramp.
The length is determined by the slope, which is in turn determined by the degree of blurring.
Blurred edges tend to be thick and sharp edges tend to be thin 15

16. First and Second derivatives
the signs of the derivatives would be reversed for an edge that transitions from light to dark 16

17. Second derivatives
produces 2 values for every edge in an image (an undesirable feature)
an imaginary straight line joining the extreme positive and negative values of the second derivative would cross zero near the midpoint of the edge. (zero-crossing property) 17

18. Noise in Images
First column: images and gray-level profiles of a ramp edge corrupted by random Gaussian noise of mean 0 and  = 0.0, 0.1, 1.0 and 10.0, respectively.
Second column: first-derivative images and gray-level profiles.
Third column : second-derivative images and gray-level profiles. 18

19. Noise in images(contd)
fairly little noise can have such a significant impact on the two key derivatives used for edge detection in images
image smoothing should be serious consideration prior to the use of derivatives in applications where noise is likely to be present. 19

20. Edge point to determine a point as an edge point
the transition in grey level associated with the point has to be significantly stronger than the background at that point.
use threshold to determine whether a value is “significant” or not.
the point’s two-dimensional first-order derivative must be greater than a specified threshold. 20

21. the magnitude becomes nonlinear
commonly approx.
Gradient Operator
first derivatives are implemented using the magnitude of the gradient. 21

22. Gradient Masks 22

23. Diagonal edges with Prewitt and Sobel masks23

24. Example 24

25. Example 25

26. Example 26

27. Laplacian
(linear operator)
Laplacian operator 27

28. Laplacian of Gaussian
Laplacian combined with smoothing to find edges via zero-crossing.
where r2 = x2+y2, and
 is the standard deviation 28

29. Mexican hat the coefficient must be sum to zero
positive central term surrounded by an adjacent negative region (a function of distance)
zero outer region
the coefficient must be sum to zero 29

30. Linear Operation second derivation is a linear operation
thus, 2f is the same as convolving the image with Gaussian smoothing function first and then computing the Laplacian of the result 30

31. Example a). Original image b). Sobel Gradient
c). Spatial Gaussian smoothing function
d). Laplacian mask
e). LoG
f). Threshold LoG
g). Zero crossing 31

32. Zero crossing & LoG Approximate the zero crossing from LoG image
to threshold the LoG image by setting all its positive values to white and all negative values to black.
the zero crossing occur between positive and negative values of the thresholded LoG. 32

33. Thresholding image with dark background and a light object
image with dark background and two light objects 33

34. Multilevel thresholding
a point (x,y) belongs to
to an object class if T1 < f(x,y)  T2
to another object class if f(x,y) > T2
to background if f(x,y)  T1
T depends on
only f(x,y) : only on gray-level values  Global threshold
both f(x,y) and p(x,y) : on gray-level values and its neighbors  Local threshold 34

35. The Role of Illumination
easily use global thresholding
object and background are separated
The Role of Illumination
f(x,y) = i(x,y) r(x,y)
a). computer generated reflectance function
b). histogram of reflectance function
c). computer generated illumination function (poor)
d). product of a). and c).
e). histogram of product image
difficult to segment 35

36. Basic Global Thresholding
generate binary image
use T midway between the max and min gray levels 36

37. Basic Global Thresholding
based on visual inspection of histogram
Select an initial estimate for T.
Segment the image using T. This will produce two groups of pixels: G1 consisting of all pixels with gray level values > T and G2 consisting of pixels with gray level values  T
Compute the average gray level values 1 and 2 for the pixels in regions G1 and G2
Compute a new threshold value
T = 0.5 (1 + 2)
Repeat steps 2 through 4 until the difference in T in successive iterations is smaller than a predefined parameter To. 37

38. Example: Heuristic method
note: the clear valley of the histogram and the effective of the segmentation between object and background
T0 = 0
3 iterations
with result T = 125 38

39. Basic Adaptive Thresholding
subdivide original image into small areas.
utilize a different threshold to segment each subimages.
since the threshold used for each pixel depends on the location of the pixel in terms of the subimages, this type of thresholding is adaptive. 39

40. Example : Adaptive Thresholding40

41. Further subdivision
a). Properly and improperly segmented subimages from previous example
b)-c). corresponding histograms
d). further subdivision of the improperly segmented subimage.
e). histogram of small subimage at top
f). result of adaptively segmenting d). 41

42. Segmentation by Morphological Watersheds

43. Introduction
Based on visualizing an image in 3D
imshow(I,[ ])
mesh(I)

44. Instead of working on an image itself, this technique is often applied on its gradient image.
In this case, each object is distinguished from the background by its up-lifted edges

45. Basic Definitions I: 2D gray level image DI: Domain of I
Path P of length l between p and q in I
A (l +1)-tuple of pixels (p0=p,p1,…,pl=q) such that pi,pi+1 are adjacent (4 adjacent, 8 adjacent, or m adjacent, see Section 2.5)
l(P): The length of a given path P
Minimum
A minimum M of I is a connected plateau of pixels from which it is impossible to reach a point of lower altitude without having to climb

46. Basic Definitions
Instead of working on an image itself, this technique is often applied on its gradient image.
Three types of points
Points belonging to a regional minimum
Catchment basin / watershed of a regional minimum
Points at which a drop of water will certainly fall to a single minimum
Divide lines / Watershed lines
Points at which a drop of water will be equally likely to fall to more than one minimum
Crest lines on the topographic surface
This technique is to identify all the third type of points for segmentation

47. Basic Steps Piercing holes in each regional minimum of I
The 3D topography is flooded from below gradually
When the rising water in distinct catchment basins is about to merge, a dam is built to prevent the merging

48. The dam boundaries correspond to the watershed lines to be extracted by a watershed segmentation algorithm
Eventually only constructed dams can be seen from above

49. Dam Construction Based on binary morphological dilation
At each step of the algorithm, the binary image in obtained in the following manner
Initially, the set of pixels with minimum gray level are 1, others 0.
In each subsequent step, we flood the 3D topography from below and the pixels covered by the rising water are 1s and others 0s. (See previous slides)

50. Notations M1, M2: Cn-1(M1), Cn-1(M2) C[n-1]
Sets of coordinates of points in the two regional minima
Cn-1(M1), Cn-1(M2)
Sets of coordinates of points in the catchment basins associated with M1 M2 at stage n-1 of flooding (catchment basins up to the flooding level)
C[n-1]
Union of Cn-1(M1), Cn-1(M2)

51. Dam Construction
At flooding step n-1, there are two connected components. At flooding step n, there is only one connected component
This indicates that the water between the two catchment basins has merged at flooding step n
Use “q” to denote the single connected component
Steps
Repeatedly dilate Cn-1(M1), Cn-1(M2) by the 3×3 structuring element shown, subject to the following condition
Constrained to q (center of the structuring element can not go beyond q during dilation

52. Dam Construction
The dam is constructed by the points on which the dilation would cause the sets being dilated to merge.
Resulting one-pixel thick connected path
Setting the gray level at each point in the resultant path to a value greater than the maximum gray value of the image. Usually max+1

53. Watershed Transform
Denote M1, M2, …, MR as the sets of the coordinates of the points in the regional minima of an (gradient) image g(x,y)
Denote C(Mi) as the coordinates of the points in the catchment basin associated with regional minimum Mi.
Denote the minimum and maximum gray levels of g(x,y) as min and max
Denote T[n] as the set of coordinates (s,t) for which g(s,t) < n
Flood the topography in integer flood increments from n=min+1 to n=max+1
At each flooding, the topography is viewed as a binary image

54. Watershed Transform C(n)
Denote Cn(Mi) as the set of coordinates of points in the catchment basin associated with minimum Mi at flooding stage n.
Cn(Mi)= C(Mi)  T[n]
Cn(Mi)=T[n]
Denote C[n] as the union of the flooded catchment basin portions at stage n:
Initialization
Let C[min+1]=T[min+1]
At each step n, assume C[n-1] has been constructed. The goal is to obtain C[n] from C[n-1]
C(n)

55. Watershed Transform
Denote Q[n] as the set of connected components in T[n].
For each qQ[n], there are three possibilities
q  C[n-1] is empty (q1)
A new minimum is encountered
q is incorporated into C[n-1] to form C[n]
q  C[n-1] contains one connected component of C[n-1] (q2)
q  C[n-1] contains more than one connected components of C[n-1] (q3)
A ridge separating two or more catchment basins has been encountered
A dam has to be built within q to prevent overflow between the catchment basins
Repeat the procedure until n=max+1

56. Example a: Original image b: Gradient image of image a
c: Watershed lines obtained from image b (oversegmentation)
 Each connected region contains one local minimum in the corresponding gradient image
d: Watershed lines obtained from smoothed image b

57. The Use of Markers
Internal markers are used to limit the number of regions by specifying the objects of interest
Like seeds in region growing method
Can be assigned manually or automatically
Regions without markers are allowed to be merged (no dam is to be built)
External markers those pixels we are confident to belong to the background
Watershed lines are typical external markers and they belong the same (background) region

58. Watershed Based Image Segmentation
Use internal markers to obtain watershed lines of the gradient of the image to be segmented.
Use the obtained watershed lines as external markers
Each region defined by the external markers contains a single internal marker and part of the background
The problem is reduced to partitioning each region into two parts: object (containing internal markers) and a single background (containing external markers)
Global thresholding, region growing, region splitting and merging, or watershed transform

59. Region-Based Segmentation - Region Growing
start with a set of “seed” points
growing by appending to each seed those neighbors that have similar properties such as specific ranges of gray level 59

60. Region Growing select all seed points with gray level 255 criteria:
the absolute gray-level difference between any pixel and the seed has to be less than 65
the pixel has to be 8-connected to at least one pixel in that region (if more, the regions are merged) 60

61. Region-based segmentation
Region growing: Groups pixels or sub-region into larger regions.
step1:
Start with a set of “seed” points and from these grow regions by appending to each seed those neighboring pixels that have properties similar to the seed.
step2:
Region splitting and merging 61

62. Region-based segmentation
Advantage:
With good connectivity
Disadvantage:
Initial seed-points:
different sets of initial seed-point cause different segmented result
Time-consuming problem 62

63. Region-based segmentation Unseeded region growing
no explicit seed selection is necessary, the seeds can be generated by the segmentation procedure automatically.
It is similar to SRG except the choice of seed point 63

64. Region-based segmentation USRG
Advantage:
easy to use
can readily incorporate high level knowledge of the image composition through region threshold
Disadvantage:
slow speed 64

65. Region-based segmentation fast scanning
Fast scanning Algorithm:
The fast scanning algorithm somewhat resembles unseeded region growing
the number of clusters of both two algorithm would not be decided before image passing through them. 65

66. Region-based segmentation fast scanning66

67. Region-based segmentation fast scanning
Last step:
merge small region to big region 67

68. Region-based segmentation fast scanning
Advantage:
The speed is very fast
The result of segmentation will be intact with good connectivity
Disadvantage:
The matching of physical object is not good
It can be improved by morphology and geometric mathematic 68