Chapter 10 – Image Segmentation DIGITAL IMAGE PROCESSING Chapter 10 – Image Segmentation Instructor: Harikanth Pasinibilli mr.harikanth@gmail.com harikanth@gvpcew.ac.in

Road map of chapter 10 Segmentation Using Morphological watersheds 10.1 10.1 10.2 10.2 10.3 10.3 10.4 10.4 10.5 10.5 10.6 10.6 Segmentation Using Morphological watersheds Fundamentals Region-Based Segmentation Thresholding Image Smoothing Using Frequency Domain Filters The Use of Motion in Segmentation Point, Line and Edge Detection 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation (P. Harikanth)

Thresholding (P. Harikanth)

Thresholding Foundation Foundation Basic Global Thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Foundation Foundation Basic Global Thresholding Optimum Global Thresholding Using Otsu’s Method Using Image Smoothing to improve Global Thresholding Using Edges to improve Global thresholding Multiple Thresholds Variable Thresholding Multivariable Thresholding (P. Harikanth)

Thresholding Foundation image with dark background and a light object image with dark background and two light objects 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation (P. Harikanth)

Thresholding a point (x,y) belongs to T depends on Foundation- Multilevel thresholding a point (x,y) belongs to to an object class if T1 < f(x,y)  T2 to another object class if f(x,y) > T2 to background if f(x,y)  T1 T depends on only f(x,y) : only on gray-level values  Global threshold both f(x,y) and p(x,y) : on gray-level values and its neighbors  Local threshold 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation (P. Harikanth)

Thresholding f(x,y) = i(x,y) r(x,y) Foundation-The Role of Illumination easily use global thresholding object and background are separated f(x,y) = i(x,y) r(x,y) a). computer generated reflectance function b). histogram of reflectance function c). computer generated illumination function (poor) d). product of a). and c). e). histogram of product image difficult to segment (P. Harikanth)

Thresholding Foundation Basic Global Thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Foundation Basic Global Thresholding Basic Global Thresholding Optimum Global Thresholding Using Otsu’s Method Using Image Smoothing to improve Global Thresholding Using Edges to improve Global thresholding Multiple Thresholds Variable Thresholding Multivariable Thresholding (P. Harikanth)

Thresholding Basic Global Thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation based on visual inspection of histogram Select an initial estimate for T. Segment the image using T. This will produce two groups of pixels: G1 consisting of all pixels with gray level values > T and G2 consisting of pixels with gray level values  T Compute the average gray level values 1 and 2 for the pixels in regions G1 and G2 Compute a new threshold value T = 0.5 (1 + 2) Repeat steps 2 through 4 until the difference in T in successive iterations is smaller than a predefined parameter To. (P. Harikanth)

Thresholding Basic Global Thresholding-Example: Heuristic method note: the clear valley of the histogram and the effective of the segmentation between object and background 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation T0 = 0 3 iterations with result T = 125 (P. Harikanth)

Thresholding Foundation Basic Global Thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Basic Global Thresholding Optimum Global Thresholding Using Otsu’s Method Optimum Global Thresholding Using Otsu’s Method Using Image Smoothing to improve Global Thresholding Using Edges to improve Global thresholding Multiple Thresholds Variable Thresholding Multivariable Thresholding (P. Harikanth)

Thresholding Otsu’s Method Assumptions Optimum Global Thresholding Using Otsu’s Method Otsu’s Method 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Assumptions It does not depend on modeling the probability density functions. It does assume a bimodal histogram distribution (P. Harikanth)

Thresholding Otsu’s Method Optimum Global Thresholding Using Otsu’s Method Otsu’s Method 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Segmentation is based on “region homogeneity”. Region homogeneity can be measured using variance (i.e., regions with high homogeneity will have low variance). Otsu’s method selects the threshold by minimizing the within-class variance. (P. Harikanth)

Thresholding Optimum Global Thresholding Using Otsu’s Method T Otsu’s Method (cont’d) Mean andVariance 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Consider an image with L gray levels and its normalized histogram P(i) is the normalized frequency of i. Assuming that we have set the threshold at T, the normalized fraction of pixels that will be classified as background and object will be: T object background (P. Harikanth)

Thresholding Optimum Global Thresholding Using Otsu’s Method 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation The mean gray-level value of the background and the object pixels will be: The mean gray-level value over the whole image (“grand” mean) is: (P. Harikanth)

Thresholding Optimum Global Thresholding Using Otsu’s Method 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation The variance of the background and the object pixels will be: The variance of the whole image is: (P. Harikanth)

17 Thresholding Optimum Global Thresholding Using Otsu’s Method Otsu’s Method (cont’d) Within-class and between-class variance 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation It can be shown that the variance of the whole image can be written as follows: within-class variance should be minimized! should be maximized! between-class variance (P. Harikanth)

Thresholding Optimum Global Thresholding Using Otsu’s Method 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Otsu’s Method (cont’d) Determining the threshold Since the total variance does not depend on T, the T that minimizes will also maximize Let us rewrite as follows: Find the T value that maximizes where (P. Harikanth)

Thresholding Optimum Global Thresholding Using Otsu’s Method Otsu’s Method (cont’d) Determining the threshold 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Start from the beginning of the histogram and test each gray- level value for the possibility of being the threshold T that maximizes (P. Harikanth)

Thresholding Drawbacks of the Otsu’s method Optimum Global Thresholding Using Otsu’s Method Otsu’s Method (cont’d) 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Drawbacks of the Otsu’s method The method assumes that the histogram of the image is bimodal (i.e., two classes). The method breaks down when the two classes are very unequal (i.e., the classes have very different sizes) In this case, may have two maxima. The correct maximum is not necessary the global one. The method does not work well with variable illumination. (P. Harikanth)

Thresholding Optimum Global Thresholding Using Otsu’s Method Otsu’s Method (cont’d) 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation (P. Harikanth)

Thresholding Foundation Basic Global Thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Foundation Basic Global Thresholding Optimum Global Thresholding Using Otsu’s Method Using Image Smoothing to improve Global Thresholding Using Image Smoothing to improve Global Thresholding Using Edges to improve Global thresholding Multiple Thresholds Variable Thresholding Multivariable Thresholding (P. Harikanth)

Thresholding Using Image Smoothing to improve Global Thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation (P. Harikanth)

Thresholding Foundation Basic Global Thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Basic Global Thresholding Optimum Global Thresholding Using Otsu’s Method Using Image Smoothing to improve Global Thresholding Using Edges to improve Global thresholding Using Edges to improve Global thresholding Multiple Thresholds Variable Thresholding Multivariable Thresholding (P. Harikanth)

Thresholding Using Edges to improve Global thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation (P. Harikanth)

Thresholding Using Edges to improve Global thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation (P. Harikanth)

Using Edges to improve Global thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation (P. Harikanth)

Using Edges to improve Global thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation (P. Harikanth)

Thresholding Foundation Basic Global Thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Basic Global Thresholding Optimum Global Thresholding Using Otsu’s Method Using Image Smoothing to improve Global Thresholding Using Edges to improve Global thresholding Multiple Thresholds Multiple Thresholds Variable Thresholding Multivariable Thresholding (P. Harikanth)

Thresholding Multiple Thresholds Otsu’s method can be extended to a 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Otsu’s method can be extended to a multiple thresholding method between-class variance can be reformulated as (P. Harikanth)

Thresholding Multiple Thresholds The K classes are separated by K-1 thresholds and these optimal thresholds can be solved by maximizing 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation For example (two thresholds) (P. Harikanth)

Thresholding Multiple Thresholds The following relationships hold: 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation The optimum thresholds can be found by : The image is then segmented by (P. Harikanth)

Thresholding Multiple Thresholds 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation (P. Harikanth)

Thresholding Foundation Basic Global Thresholding 10.1- Fundamentals 10.2- Point, Line and Edge Detection 10.3- Thresholding 10.4- Region-Based Segmentation 10.5 - Segmentation Using Morphological watersheds 10.6- The Use of Motion in Segmentation Basic Global Thresholding Optimum Global Thresholding Using Otsu’s Method Using Image Smoothing to improve Global Thresholding Using Edges to improve Global thresholding Multiple Thresholds Variable Thresholding Variable Thresholding Multivariable Thresholding (P. Harikanth)