What is Image Segmentation? There are many definitions. Three common ones are Image Segmentation is the process of isolating objects of interest from the rest of the scene. (Castleman ) Image segmentation is the process of partitioning (分割) an image into non-intersecting region such that each region is homogeneous (相似的) and the union of no two adjacent regions is homogeneous. (Pal )

What is Image Segmentation? Image segmentation is to divide an image into parts that have a strong correlation with objects or areas of the real world contained in the image. (Watt ) In brief, segmentation is to subdivide an image into its constituent (组成的) regions or objects. Segmentation should stop when the objects of interest in an application have been isolated.

Image Processing Flow based on Image Segmentation Feature Extraction Classification Input Image Object Image Feature Vector Object Type

Why is it difficult? In general autonomous (自主的) segmentation is one of the most difficult tasks in image processing. It is difficult because of many reasons. Here are some typical obstacles: Non-uniform illumination No control of the environment Inadequate model of the object of interest Noise etc.

Principal approaches Segmentation methods can be divided into three groups according to the dominant features they employ. Segmentation based on global knowledge about an image; The knowledge is usually represented by a histogram of image features. Edge-based segmentations; Utilizing edge detection processes to find a closed boundary so that an inside and an outside can be defined. Region-based segmentations; This techniques proceed by dividing the image into regions that exhibit similar properties.

2 basis properties of intensity values Segmentation algorithms generally are based on one of 2 basis properties of intensity values Discontinuity to partition an image based on abrupt (突然的) changes in intensity (such as edges) Similarity to partition an image into regions that are similar according to a set of predefined criteria.

Detection of Discontinuities detect the three basic types of gray level discontinuities points , lines , edges the common way is to run a mask through the image

Contents Thresholding Point Detection Line Detection Edge-based Segmentation Region-based Segmentation

7.1 Thresholding Thresholding is a labeling operation on a gray scale image that distinguishes pixels of a higher intensity from pixels with a lower intensity value. The output of thresholding usually is a binary image. This technique is particularly useful for scenes which contain solid objects on a uniform contrasting background.

Classification of Thresholding Thresholding can be viewed as an operation that involves tests against a function T of the form: where p(x,y) denotes some local property of this point.

Classification of Thresholding When T depends on only f(x,y) : only on gray-level values 􀀾  Global thresholding both f(x,y) and p(x,y) : on gray-level values and its neighbors  Local thresholding x and y (in addition)  Dynamic thresholding

Basic Global Thresholding See “basic_global_thre.m” Original image Histogram Solution: use T midway between the max and min gray levels

Basic Global Thresholding Let light objects in dark background To extract the objects: Select a “T” that separates the objects from the background i.e. any (x,y) for which f(x,y)>T is an object point. A thresholded image:

Heuristic Global Thresholding Select an initial estimate for T. Segment the image using T. This will produce two groups of pixels: G1 consisting of all pixels with gray level values < T and G2 consisting of pixels with gray level values  T Compute the average gray level values μ1 and μ2 for the pixels in regions G1 and G2 Compute a new threshold value: T=0.5(μ1+μ2) Repeat steps 2 through 4 until the difference in T in successive iterations is smaller than a predefined parameter T0. see”h….m”

Basic Adaptive Thresholding subdivide original image into small areas. utilize a different threshold to segment each subimages. since the threshold used for each pixel depends on the location of the pixel in terms of the subimages, this type of thresholding is adaptive. See “Adaptive_threshold.m”

Multilevel Thresholding a point (x,y) belongs to an object class, if T1 < f(x,y) ≤ T2 another object class, if f(x,y) > T2 to background if f(x,y) ≤ T1

The histogram of an image containing two principal brightness regions can be considered an estimate of the brightness probability density function p(z). p(z) is the sum (or mixture) of two unimodal (单峰的) densities (one for light, one for dark regions).

Optimal Thresholding If the form of the densities is known or assumed, in terms of minimum error, determining an optimal threshold for segmenting the image is possible.

Optimal Thresholding Probability of erroneously

Optimal Thresholding Minimum error Differentiating ( 微分) E(T) with respect to T (using Leibniz’s rule) and equating the result to 0 find T which makes

Optimal Thresholding Minimum error Specially, if P1 = P2 then the optimum threshold is where the curve p1(z) and p2(z) intersect

Optimal Thresholding For example, Let PDF=Gaussian density : p1(z) and p2(z) where μ1 and σ12 are the mean and variance of the Gaussian density of one object μ2 and σ22 are the mean and variance of the Gaussian density of the other object

Optimal Thresholding Quadratic equation (二次方程)

Problems of Thresholding Original image Thresholded image

Problems of Thresholding Exact threshold segmentation Threshold too low Threshold too high

7.2 Point Detection a point has been detected at the location on which the mark is centered if |R|≥T where T is a nonnegative threshold R is the sum of products of the coefficients with the gray levels contained in the region encompassed by the mark

7.2 Point Detection Note that the mark is the same as the mask of Laplacian Operation (in chapter 3) The only differences that are considered interest are those large enough (as determined by T) to be considered isolated points.

Example

7.3 Line Detection Horizontal mask will result with max response when a line passed through the middle row of the mask with a constant background. the similar idea is used with other masks. Note: the preferred direction of each mask is weighted with a larger coefficient (i.e.,2) than other possible directions.

Idea 1 of Line Detection Apply every masks on the image let R1, R2, R3, R4 denotes the response of the horizontal, +45 degree, vertical and -45 degree masks, respectively. if, at a certain point in the image |Ri|>|Rj|, for all j≠i, that point is said to be more likely associated with a line in the direction of mask i.

Idea 2 of Line Detection Alternatively, if we are interested in detecting all lines in an image in the direction defined by a given mask, we simply run the mask through the image and threshold the absolute value of the result. After thresholding, the points that are left are the strongest responses, which, for lines one pixel thick, correspond closest to the direction defined by the mask.

Example

7.4 Edge-based Segmentation Edge-based segmentations rely on edges found in an image by edge detecting operators these edges mark image locations of discontinuities in gray level Edge detection is the most common approach for detecting meaningful discontinuities in gray level There are a large group of methods based on information about edges in the image

What is edge? Edge is where change occurs Change is measured by derivative in 1D Biggest change, derivative has maximum magnitude Or 2nd derivative is zero. we discuss approaches for implementing first-order derivative (Gradient operator) second-order derivative (Laplacian operator) we have introduced both derivatives in chapter 3 Here, we will talk only about their properties for edge detection.

What is edge? In other words, an edge is a set of connected pixels that lie on the boundary between two regions with relatively distinct gray-level properties. Note: edge vs. boundary an edge is a “local” concept. whereas a region boundary, owing to the way it is defined, is a more global idea.

Ideal and Ramp (斜坡) Edges because of optics, sampling, image acquisition imperfection

Thick and Thin Edge The slope of the ramp is inversely proportional to the degree of blurring in the edge. Namely, we no longer have a thin (one pixel thick) path. Instead, an edge point now is any point contained in the ramp, and an edge would then be a set of such points that are connected. The thickness is determined by the length of the ramp. The length is determined by the slope, which is in turn determined by the degree of blurring. Blurred edges tend to be thick and sharp edges tend to be thin.

First and Second derivatives (导数) Gray-level profile First derivate the signs of the derivatives would be reversed for an edge that transitions from light to dark Second derivate

Second derivatives an undesirable feature zero-crossing property produces 2 values for every edge in an image zero-crossing property an imaginary straight line joining the extreme positive and negative values of the second derivative would cross zero near the midpoint of the edge. quite useful for locating the centers of thick edges.

Basic idea of edge detection A profile is defined perpendicularly to the edge direction and the results are interpreted. The magnitude of the first derivative is used to detect an edge (if a point is on a ramp) The sign of the second derivative can determine whether an edge pixel is on the dark or light side of an edge.

Review of First Derivate Gradient Operator: simplest approximation, 22 Roberts cross-gradientoperators, 22 Sobel operators, 33

Edge direction and strength Let α(x,y) represents the direction angle of the vector f at (x,y) α(x,y)=tan-1(Gy/Gx) The direction of an edge at (x,y) perpendicular (垂直) to the direction of the gradient vector at that point The edge strength is given by the gradient magnitude

Gradient Masks

Diagonal edges with Prewitt and Sobel masks

Review of Second Derivate Laplacian Operator Laplacian Mask

Example of edge detection See Matlab Help--demo--toolbox--image processing--analysis…--Edge detection Note: The Laplacian is seldom used in practice, because: unacceptably sensitive to noise (as second-order derivative) produces double edges unable to detect edge direction

Canny edge detector The most powerful edge-detection method It differs from the other edge-detection methods in that it uses two different thresholds (to detect strong and weak edges), and includes the weak edges in the output only if they are connected to strong edges. This method is therefore less likely than the others to be "fooled" by noise, and more likely to detect true weak edges.

Laplacian of Gaussian Laplacian combined with smoothing to find edges via zero-crossing.  determines the degree of blurring that occurs.

Laplacian of Gaussian (Mexican hat) The coefficient must sum to zero

Edge Detection and Segmentation Image resulting from edge detection cannot be used as a segmentation result. Supplementary processing steps must follow to combine edges into edge chains that correspond better with borders (boundary) in the image.

7.5 Region-based Segmentation Goal: find regions that are “homogeneous” by some criterion Segmentation is a process that partitions R into n subregions R1, R2, …, Rn such that: P(Ri) is a logical predicate property defined over the points in set Ri For example, P(Ri) = TRUE if all pixel in Ri have the same gray level

Two methods of Region Segmentation Region Growing Region Splitting Region growing is the opposite of the split and merge approach.

Region Growing The objective of segmentation is to partition an image into regions A region is a connected component with some uniformity (say gray-levels or texture) In region growing, we start with a set of “seed” points, growing by appending to each seed’s neighbor pixels, if they have similar properties, such as specific ranges of gray level, and ‘8-connected neighbor’ Need: initialization, similarity criterion

Steps of Region Growing Start by choosing an arbitrary seed pixel and compare it with neighbor pixels. When seed and neighbor pixels are similar and 8-connected neighbor, region is grown from the seed pixel by adding neighbor pixels. When the growth of one region stops, choose another seed pixel which does not yet belong to any region and start again.

Region growing An initial set of small areas are iteratively merged according to similarity constraints.

Example: defective welds (焊缝) detecting Seed points X ray of weld (the horizontal dark region) containing several cracks and porosities (孔, the bright, white streaks running horizontally through the image) We need initial seed points to grow into regions On looking at the histogram and image, cracks are bright Select all pixels having value of 255 as seeds

Result of region growing and boundaries of defects Criterion: There is a valley at around 190 in the histogram. A pixel should have a value>190 to be considered as a part of region to the seed point. The pixel should be a 8-connected neighbor to at least one pixel in that region Result of region growing and boundaries of defects

Region Splitting The opposite approach to region merging. A top-down approach and starts with the assumption that the entire image is homogeneous. If this is not true, the image is split into four sub images This splitting procedure is repeated recursively(递归的) until the image is split into homogeneous regions. Need: homogeneity criterion, split rule

Region Splitting Disadvantage they create regions that may be adjacent and homogeneous, but not merged.

Region Splitting and Merging Procedure: Split image into 4 disjointed quadrants, for any region Ri, P(Ri)=FALSE Merge any adjacent regions Rj and Rk, for which P(Rj∪Rk)=TRUE Stop when no further splitting or merging is possible.

Region Splitting and Merging Quadtree (四叉树)

Result of Splitting and Merging Example Original image Thresholded image Result of Splitting and Merging P(Ri) = TRUE if at least 80% of the pixels in Ri have the property |zj-mi|≤2σi where zj is the gray level of the jth pixel in Ri mi is the mean gray level of that region σi is the standard deviation of the gray levels in Ri