1. Image Segmentation – Detection of Discontinuities

2. Image Segmentation Outline Detection of gray level discontinuities
Point detection
Line detection
Edge detection
Gradient operators
LoG : Laplacian of Gaussian
Edge linking and boundary detection
Hough transform
Thresholding
Region-based segmentation
Segmentation by Morphological watersheds
The use of motion in segmentation

3. Revisit - Goals of image processing
Image Segmentation
Revisit - Goals of image processing
Image improvement – low level IP
Improvement of pictorial information for human interpretation (Improving the visual appearance of images to a human viewer )
Image analysis – high level IP
Processing of scene data for autonomous machine perception (Preparing images for measurement of the features and structures present )

4. Extracting information from an image
Image Segmentation
Image analysis – HLIP
Extracting information from an image
Step 1 : segment the image ---> objects or regions
Step 2 : describe and represent the segmented regions in a form suitable for computer processing
Step 3 : image recognition and interpretation

5. Image Segmentation
Image analysis – HLIP (cont’)

6. Heavily rely on one of two properties of intensity values:
Image Segmentation
What is segmentation?
Definition
Subdivides an image into its constituent regions or objects
从图像中提取出所需的语义对象
将图像划分成若干有一定涵义的区域
Heavily rely on one of two properties of intensity values:
Discontinuity ---- Partition based on abrupt changes in intensity, e.g. edges in an image
point / line / edge / corner detection
Similarity ---- Partition based on intensity similarity, e.g. thresholding
thresholding
region growing / splitting / merging

7. Image Segmentation Segmentation
We’ll discuss both approaches. Starting with the first one.

8. What Should Good Image segmentation be?
Region interiors
Simple
Without many small holes
Adjacent regions
Should have significantly different values
Boundaries
Not ragged
Spatially accurate
Achieving all these desired properties is difficult. There is no theory of image segmentation. Image segmentation techniques are basically ad hoc.
R.M. Haralick and L.G. Shapiro, Image Segmentation Techniques. Computer Vision, Graphics, and Image Processing, 1985, 29:

9. We want to extract 3 basic types of gray-level discontinuity:
Image Segmentation
Introduction
We want to extract 3 basic types of gray-level discontinuity:
Points
Lines
Edges
What have we learnt in previous lectures to help us in this process?
CONVOLUTION!
Grayscale image
Mask coefficient

10. Image Segmentation - 1
Point detection

11. Image Segmentation - 1
Line detection
Masks for lines of different directions:
Respond more strongly to lines of one pixel thick of the designated direction.
High or low pass filters?

12. Line detection (cont’)
Image Segmentation - 1
Line detection (cont’)
If interested in lines of any directions, run all 4 masks and select the highest response.
If interested only in lines of a specific direction (e.g. vertical), use only the mask associated with that direction.
Threshold the output.
The strongest responses for lines one pixel thick, and correspond closest to the direction defined by the mask.

13. Image Segmentation - 1
Line detection (cont’)

14. Far more practical than line detection.
Image Segmentation - 1
Edge Detection
Far more practical than line detection.
We’ll discuss approaches based on
1st-order digital derivative
2nd-order digital derivative

15. Ramp-like (in real life) edge
Image Segmentation - 1
What is an edge?
A set of connected pixels that lie on the boundary between two regions.
Local concept
Edge point
Any point could be an edge point
Ideal/step edge
Ramp-like (in real life) edge

16. Image Segmentation - 1
1st Derivative
Positive at the points of transition into and out of the ramp, moving from left to right along the profile
Constant for points in the ramp
Zero in areas of constant gray Level
Magnitude for presence of an edge at a point in an image (i.e. if a point is on a ramp)

17. Positive at the transition associated with the dark side of the edge
Image Segmentation - 1
2nd Derivative
Positive at the transition associated with the dark side of the edge
Negative at the transition associated with the bright side of the edge
Zero elsewhere
Producing 2 values for every edge in an image (an undesirable feature).
Center of a thick edge is located at the zero crossing
Zero crossing

18. Image Segmentation - 1
Edge detection (cont’)

19. Edge detection (cont’) ---- Effect of Noise
Image Segmentation - 1
Edge detection (cont’) ---- Effect of Noise
(a)
Corrupted by Random Gaussian noise of mean 0 and standard deviation of
(a) 0
(b) 0.1
(c) 1.0
(d) 10.0
Conclusion???
----Sensitivity of derivative to noise
(b)
(c)
(d)
grayscale
1st derivative
2nd derivative

20. Edge detection (cont’)
Image Segmentation - 1
Edge detection (cont’)
The difference between edge and boundary
Edge detection steps
Compute the local derivative
Magnitude of the 1st derivative can be used to detect the presence of an edge
The sign of the 2nd derivative can be used to determine whether an edge pixel lies on the dark or light side of an image
Zero crossing of the 2nd derivative is at the midpoint of a transition in gray level, which provides a powerful approach for locating the edge.
Edge: a “local” concept
Boundary: a more global idea

21. An edge element is associated with 2 components:
Image Segmentation - 1
Some Terminology
An edge element is associated with 2 components:
magnitude of the gradient, and
and edge direction , rotated with respect to the gradient direction by -90 deg.

22. Gradient operators (1st Derivative)
Image Segmentation - 1
Gradient operators (1st Derivative)
Use gradient for image differentiation
The gradient of an image f(x,y) at location (x,y) is defined as
Some properties about this gradient vector
It points in the direction of maximum rate of change of image at (x,y)
Magnitude
angle

23. Finite Gradient - Approximation
Image Segmentation - 1
Finite Gradient - Approximation
Central differences (not usually used because they neglect the impact of the pixel (x,y) itself)
h is a small integer, usually 1.
h should be chosen small enough to provide a good approximation to the derivative, but large enough to neglect unimportant changes in the image function.

24. Image Segmentation - 1
Edge operator

25. Better noise-suppression
Image Segmentation - 1
Sobel edge operator
Advantages : providing both differencing and a smooth effect and slightly superior noise reduction characteristics.
Better noise-suppression

26. Image Segmentation - 1
Edge detection example

27. Image Segmentation - 1
Edge detection example (cont’)

28. Image Segmentation - 1
Edge detection example (cont’)

29. 2nd Derivative: Laplacian Operator
Image Segmentation - 1
2nd Derivative: Laplacian Operator
Review: The Laplacian operator ( ) is a very popular operator approximating the second derivative which gives the gradient magnitude only.
We discussed this operator in spatial filtering
It is isotropic
4-neighborhood
8-neighborhood

30. Image Segmentation - 1 Issues with Laplacian Problems: Fixes:
Unacceptably sensitive to noise
Magnitude of Laplacian results in double edges
Does not provide gradient, so can’t detect edge direction
Fixes:
Smoothing
Using zero-crossing property for edge location
Not for gradient direction, but for establishing whether a pixel is on the dark or light side of and edge

31. Smoothing for Laplacian
Image Segmentation - 1
Smoothing for Laplacian
Our goal is to get a second derivative of a smoothed 2D function
We have seen that the Laplacian operator gives the second derivative, and is non-directional (isotropic).
Consider then the Laplacian of an image smoothed by a Gaussian.
This operator is abbreviated as LoG, from Laplacian of Gaussian:
The order of differentiation and convolution can be interchanged due to linearity of the operations:

32. Laplacian of Gaussian (LoG)
Image Segmentation - 1
Laplacian of Gaussian (LoG)
Let’s make the substitution where r measures distance from the origin.
Now we have a 1D Gaussian to deal with
Laplacian of Gaussian becomes
Normalize the sum of the mask elements to 0

33. Marr and hildreth’s approach
Image Segmentation - 1
Marr and hildreth’s approach
Smooth the image to reduce noise
Then calculate the 2nd derivative
Finally, find the zero-crossing
LoG (Laplacian of Gaussian, Mexican hat function)

34. Laplacian of Gaussian (LoG)
Image Segmentation - 1
Laplacian of Gaussian (LoG)
Because of its shape, the LoG operator is commonly called a Mexican hat.

35. Gradient operators – examples
Image Segmentation - 1
Gradient operators – examples
Zero-Crossing:
Advantages:
noise reduction capability;
edges are thinner.
Drawbacks:
edges form numerous closed loops (spaghetti effect);
computation complex.

36. Closed loops (spaghetti effect)
Image Segmentation - 1
Illustration
One simple method for approximating zero-crossing:
Setting all + values to white, - values to black.
Scanning the thresholded image and noting the transition between black and white.
Closed loops (spaghetti effect)
original
LoG
thresholded
zero crossing

37. Edge detection by gradient operations tends to work well when
Image Segmentation - 1
discussion
Edge detection by gradient operations tends to work well when
Images have sharp intensity transitions
Relative low noise
Zero-crossing approach work well when
Edges are blurry
High noise content
Provide reliable edge detection

38. Edge detection based on
Image Segmentation - 1
Summary
Point detection
Line detection
Edge detection based on
1st derivative
Provides gradient information
2nd derivative using zero-crossing
Indicates dark/bright side of an edge

39. Image Segmentation - 1
References
J. Canny, A computational approach for edge detection. IEEE Trans. on PAMI, 1986, 8(6): 679~698
J. Shen, An optimal linear operator for step edge detection. CVGIP: Graphical, Models and Image Processing, 1992, 54(2):