Image Segmentation Image segmentation (segmentace obrazu) division or separation of the image into segments (connected regions) of similar properties (attributes). Haralick and Shapiro ”The regions should be uniform and homogenous with respect to some characteristic such as intensity value or texture. Region interiors should be simple and without many small holes. Adjacent regions should have significantly different values with respect to the characteristic on which they are uniform. Boundaries of each segment should be simple, not ragged, and must be spatially accurate.”

Approaches to Image Segmentation Thresholding Global thresholding Local (adaptive) thresholding Edge detection Curve fitting Heuristic edge linking Hough transform Region-based methods Region growing Region splitting Split and merge Texture segmentation Amplitude approach Edge-based approach Clustering methods Cluster analysis Template matching Correlation with pattern Advanced methods Mathematical morphology tools Watershed algorithm LCS algorithm

Thresholding Thresholding (prahování) the pixels are split into regions based on their intensity: the pixels with the intensity higher than a predefined threshold value belong to a different region than the pixels with intensity lower than the threshold Global thresholding = thresholding the threshold is equal for all pixels in the image (independent of their position in the image) Local (adaptive) thresholding the threshold depends on the position of the pixel in the image Determination of the threshold Manual: the threshold is set by user using a slider with preview Automatic: using intensity histogram analysis

Bilevel Thresholding Bilevel thresholding (dvouúrovňové prahování) Idea: If the image is simple (objects of similar intensity on a uniform background, e.g. typed text or cells of the same type), the histogram contains two peaks with a valley between them. The threshold is taken at the valley (at the local minimum between the two maxima). For more precise determination of the minimum, the valley can be fitted by parabola. If the histogram does not contain two peaks, sometimes it helps to compute the histogram only of those pixels, where the magnitude of Laplacian is large.

Multilevel Thresholding Multilevel thresholding (víceúrovňové prahování) Idea: If the image contains several types of objects on a uniform background and each type of object has got a specific intensity range, (e.g. a mixture of cells of several types), the histogram contains several peaks with valleys between them. Several thresholds are taken, one threshold per each valley (at the local minimum between the two neighboring maxima). For more precise determination of the minimum, the valley can be fitted by parabola.

Thresholding of Color Images Color images can be segmented in the following ways: The thresholding is performed in each color channel (RGB) separately. The thresholding is performed not according to the intensity but according to hue, saturation or other characteristics. The user (or computer) selects specific ranges of colors for each object type.

Local (Adaptive) Thresholding Motivation a) Uneven background intensity b) Uniform background intensity but uneven object intensities which do not form individual peaks in the histogram (the peaks of indivi- dual object types overlap in the histogram forming one wider peak)

Local (Adaptive) Thresholding Solutions 1) The image is divided into regular subregions (e.g. squares) and the threshold is set individually for each region using the analysis of the histogram computed from this particular region. Not very good results at the edges between regions. 2) The same as 1) but thresholds for individual pixels are determined using interpolation of the threshold value among the regions. Good in case a) but not in case b). 3) First, global thresholding is performed to separate background from objects. Then, for each object, local intensity histogram is computed and threshold determined. All objects are re-segmented with individual thresholds Good in case b) but problems arise when two types of objects (of different intensities) touch each other and are treated as one region after the initial global thresholding.

Local Thresholding Example Original grayscale image High global threshold Low global threshold Individual threshold for each object

Edge Detection Segmentation based on edge detection Problem Determination of object boundaries from a gradient image (1st derivative of the original image) or Laplacian image (2nd derivative of the original image), which are usually obtained using Sobel or Laplacian filters, respectively. Problem Boundaries in gradient image are not continuous! Solution 1: Curve fitting Assumption: Objects in the image are simple, the edge map does not contain branches. Iterative end-point line fitting Other polygon fittings Polynomial fittings Bezier polynomial or spline fittings

Edge Detection Solution 2: Heuristic edge linking Assumption: objects in the image are not too complicated. Boundary points are linked with lines using a certain heuristic algorithm, then multiple connections and branches are deleted and finally boundary fragments are connected with ”bridges”. Solution 3: Hough transform Hough transform is a transformation of a line in Cartesian coordinates to a point in polar coordinates according to the equation: where r is the normal distance of the line from the origin and q is the angle of the normal with respect to the x axis.

Hough Transform Basic facts about Hough transform A line is transformed to a point. A family of lines passing through a common point is transformed to a connected set of points (a curve). Three families of lines passing through 3 colinear points are transformed to 3 curves that intersect at a single point corresponding to a line in Cartesian coordinates passing through these three points. In discrete Hough transform of a binary image, each non-zero data point in the original image is transformed to a curve in the polar domain, which is quantized into cells (pixels). If the curve passes the given cell in polar coordinates, the cell is incremented by one count. In discrete Hough transform of a gray-scale image, the cell is incremented by a value corresponding to the gradient magnitude (i.e. strong edges have bigger influence than weak edges).

Hough Transform Hough transform edge linking Each cell in the Hough image whose magnitude is sufficiently large defines a straight line that passes through the original image. If this line is overlaid with the image edge map (gradient image), it should cover the missing links of straight line edge segments. The missing links must be filled using a certain heuristic method (like in Solution 2). Recommended web page on Hough transform http://cs-alb-pc3.massey.ac.nz/notes/59318/l11.html

Generalized Hough Transform Hough transform for arbitrary shapes (functions) Consider an arbitrary curve represented by equation , where a is the vector of curve parameters. Algorithm (curve detection using the HT): Quantize parameter space within the limits of parameters a. The dimensionality n of the parameter space is given by the number of parameters of the vector a. Form an n-dimensional accumulator array A(a) with structure matching the quantization of parameter space; set all elements to zero. For each image point (x1, x2) in the appropriately thresholded gradient image, increase all accumulator cells A(a) if f(x,a) = 0, i.e. A(a) = A(a) + A for all a inside the limits used in step (1) Local maxima in the A-array correspond to realization of curve f(x,a) that are present in the original image.

Generalized Hough Transform Circular HT (CHT): Besides lines, the most important is the detection of circles in images. In this case, the parameters in Hough space are the circle center (x,y) and radius r if it is not known a priori. Algorithm for CHT: The analytic expression is The dimensionality of accumulator structure data is 3. For each pixel (x,y) whose edge magnitude exceeds a given threshold, all accumulator cells corresponding to potential circle centers (a,b) are incremented. The accumulator cell A(a,b,r) is incremented if the point (a,b) is at distance r from point (x,y), and this condition is valid for all triplets (a,b,r) satisfying circle equation. If some potential center (a,b) of a circle of radius r is frequently found in the parameter space, it is highly probable that a circle with radius r and center (a,b) really exists in the processed data.

Generalized Hough Transform useful web page: http://markschulze.net/java/hough/ example: source: Sonka, Hlavac, Image Processing, Analysis and Machine Vision