Image Segmentation – Detection of Discontinuities

Image Segmentation Outline Detection of gray level discontinuities Point detection Line detection Edge detection Gradient operators LoG : Laplacian of Gaussian Edge linking and boundary detection Hough transform Thresholding Region-based segmentation Segmentation by Morphological watersheds The use of motion in segmentation

Revisit - Goals of image processing Image Segmentation Revisit - Goals of image processing Image improvement – low level IP Improvement of pictorial information for human interpretation (Improving the visual appearance of images to a human viewer ) Image analysis – high level IP Processing of scene data for autonomous machine perception (Preparing images for measurement of the features and structures present )

Extracting information from an image Image Segmentation Image analysis – HLIP Extracting information from an image Step 1 : segment the image ---> objects or regions Step 2 : describe and represent the segmented regions in a form suitable for computer processing Step 3 : image recognition and interpretation

Image Segmentation Image analysis – HLIP (cont’)

Heavily rely on one of two properties of intensity values: Image Segmentation What is segmentation? Definition Subdivides an image into its constituent regions or objects 从图像中提取出所需的语义对象 将图像划分成若干有一定涵义的区域 Heavily rely on one of two properties of intensity values: Discontinuity ---- Partition based on abrupt changes in intensity, e.g. edges in an image point / line / edge / corner detection Similarity ---- Partition based on intensity similarity, e.g. thresholding thresholding region growing / splitting / merging

Image Segmentation Segmentation We’ll discuss both approaches. Starting with the first one.

What Should Good Image segmentation be? Region interiors Simple Without many small holes Adjacent regions Should have significantly different values Boundaries Not ragged Spatially accurate Achieving all these desired properties is difficult. There is no theory of image segmentation. Image segmentation techniques are basically ad hoc. R.M. Haralick and L.G. Shapiro, Image Segmentation Techniques. Computer Vision, Graphics, and Image Processing, 1985, 29: 100-132

We want to extract 3 basic types of gray-level discontinuity: Image Segmentation Introduction We want to extract 3 basic types of gray-level discontinuity: Points Lines Edges What have we learnt in previous lectures to help us in this process? CONVOLUTION! Grayscale image Mask coefficient

Image Segmentation - 1 Point detection

Image Segmentation - 1 Line detection Masks for lines of different directions: Respond more strongly to lines of one pixel thick of the designated direction. High or low pass filters?

Line detection (cont’) Image Segmentation - 1 Line detection (cont’) If interested in lines of any directions, run all 4 masks and select the highest response. If interested only in lines of a specific direction (e.g. vertical), use only the mask associated with that direction. Threshold the output. The strongest responses for lines one pixel thick, and correspond closest to the direction defined by the mask.

Image Segmentation - 1 Line detection (cont’)

Far more practical than line detection. Image Segmentation - 1 Edge Detection Far more practical than line detection. We’ll discuss approaches based on 1st-order digital derivative 2nd-order digital derivative

Ramp-like (in real life) edge Image Segmentation - 1 What is an edge? A set of connected pixels that lie on the boundary between two regions. Local concept Edge point Any point could be an edge point Ideal/step edge Ramp-like (in real life) edge

Image Segmentation - 1 1st Derivative Positive at the points of transition into and out of the ramp, moving from left to right along the profile Constant for points in the ramp Zero in areas of constant gray Level Magnitude for presence of an edge at a point in an image (i.e. if a point is on a ramp)

Positive at the transition associated with the dark side of the edge Image Segmentation - 1 2nd Derivative Positive at the transition associated with the dark side of the edge Negative at the transition associated with the bright side of the edge Zero elsewhere Producing 2 values for every edge in an image (an undesirable feature). Center of a thick edge is located at the zero crossing Zero crossing

Image Segmentation - 1 Edge detection (cont’)

Edge detection (cont’) ---- Effect of Noise Image Segmentation - 1 Edge detection (cont’) ---- Effect of Noise (a) Corrupted by Random Gaussian noise of mean 0 and standard deviation of (a) 0 (b) 0.1 (c) 1.0 (d) 10.0 Conclusion??? ----Sensitivity of derivative to noise (b) (c) (d) grayscale 1st derivative 2nd derivative

Edge detection (cont’) Image Segmentation - 1 Edge detection (cont’) The difference between edge and boundary Edge detection steps Compute the local derivative Magnitude of the 1st derivative can be used to detect the presence of an edge The sign of the 2nd derivative can be used to determine whether an edge pixel lies on the dark or light side of an image Zero crossing of the 2nd derivative is at the midpoint of a transition in gray level, which provides a powerful approach for locating the edge. Edge: a “local” concept Boundary: a more global idea

An edge element is associated with 2 components: Image Segmentation - 1 Some Terminology An edge element is associated with 2 components: magnitude of the gradient, and and edge direction , rotated with respect to the gradient direction by -90 deg.

Gradient operators (1st Derivative) Image Segmentation - 1 Gradient operators (1st Derivative) Use gradient for image differentiation The gradient of an image f(x,y) at location (x,y) is defined as Some properties about this gradient vector It points in the direction of maximum rate of change of image at (x,y) Magnitude angle

Finite Gradient - Approximation Image Segmentation - 1 Finite Gradient - Approximation Central differences (not usually used because they neglect the impact of the pixel (x,y) itself) h is a small integer, usually 1. h should be chosen small enough to provide a good approximation to the derivative, but large enough to neglect unimportant changes in the image function.

Image Segmentation - 1 Edge operator

Better noise-suppression Image Segmentation - 1 Sobel edge operator Advantages : providing both differencing and a smooth effect and slightly superior noise reduction characteristics. Better noise-suppression

Image Segmentation - 1 Edge detection example

Image Segmentation - 1 Edge detection example (cont’)

Image Segmentation - 1 Edge detection example (cont’)

2nd Derivative: Laplacian Operator Image Segmentation - 1 2nd Derivative: Laplacian Operator Review: The Laplacian operator ( ) is a very popular operator approximating the second derivative which gives the gradient magnitude only. We discussed this operator in spatial filtering It is isotropic 4-neighborhood 8-neighborhood

Image Segmentation - 1 Issues with Laplacian Problems: Fixes: Unacceptably sensitive to noise Magnitude of Laplacian results in double edges Does not provide gradient, so can’t detect edge direction Fixes: Smoothing Using zero-crossing property for edge location Not for gradient direction, but for establishing whether a pixel is on the dark or light side of and edge

Smoothing for Laplacian Image Segmentation - 1 Smoothing for Laplacian Our goal is to get a second derivative of a smoothed 2D function We have seen that the Laplacian operator gives the second derivative, and is non-directional (isotropic). Consider then the Laplacian of an image smoothed by a Gaussian. This operator is abbreviated as LoG, from Laplacian of Gaussian: The order of differentiation and convolution can be interchanged due to linearity of the operations:

Laplacian of Gaussian (LoG) Image Segmentation - 1 Laplacian of Gaussian (LoG) Let’s make the substitution where r measures distance from the origin. Now we have a 1D Gaussian to deal with Laplacian of Gaussian becomes Normalize the sum of the mask elements to 0

Marr and hildreth’s approach Image Segmentation - 1 Marr and hildreth’s approach Smooth the image to reduce noise Then calculate the 2nd derivative Finally, find the zero-crossing LoG (Laplacian of Gaussian, Mexican hat function)

Laplacian of Gaussian (LoG) Image Segmentation - 1 Laplacian of Gaussian (LoG) Because of its shape, the LoG operator is commonly called a Mexican hat.

Gradient operators – examples Image Segmentation - 1 Gradient operators – examples Zero-Crossing: Advantages: noise reduction capability; edges are thinner. Drawbacks: edges form numerous closed loops (spaghetti effect); computation complex.

Closed loops (spaghetti effect) Image Segmentation - 1 Illustration One simple method for approximating zero-crossing: Setting all + values to white, - values to black. Scanning the thresholded image and noting the transition between black and white. Closed loops (spaghetti effect) original LoG thresholded zero crossing

Edge detection by gradient operations tends to work well when Image Segmentation - 1 discussion Edge detection by gradient operations tends to work well when Images have sharp intensity transitions Relative low noise Zero-crossing approach work well when Edges are blurry High noise content Provide reliable edge detection

Edge detection based on Image Segmentation - 1 Summary Point detection Line detection Edge detection based on 1st derivative Provides gradient information 2nd derivative using zero-crossing Indicates dark/bright side of an edge

Image Segmentation - 1 References J. Canny, A computational approach for edge detection. IEEE Trans. on PAMI, 1986, 8(6): 679~698 J. Shen, An optimal linear operator for step edge detection. CVGIP: Graphical, Models and Image Processing, 1992, 54(2): 112-133