Chap. 9: Image Segmentation Jen-Chang Liu, 2005

Motivation Segmentation subdivides an image into its constituent regions or objects Example: 生物細胞在影像序列中的追蹤

Basics of image segmentation Input: image => Output: attributes Segmentation of nontrivial image is one of the most difficult tasks in image processing Ex. Separate the sky, clouds, and mountains

Two approaches Based on intensity values discontinuity Partition an image based on abrupt changes, such as edges Based on intensity similarity in regions Partition an image into regions that are similar according to some criteria

Outline Point, line detection Edge detection Thresholding Use of motion in segmentation Pixel discontinuity Region similarity

Detection of discontinuity Three basic types of gray-level discontinuity Points, lines, and edges Method: run a mask through the image Input: 3x3 region z 1, z 2, …, z 9 Output: z 5 -> R z1z1 z2z2 z3z3 z4z4 z5z5 z6z6 z7z7 z8z8 z9z9 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9

Point detection Point detection mask 1.Mask operation over whole image 2.Threshold: |R|  T 渦輪葉片 X 光 thresholding point mask

Problem with previous point detection Assume single-pixel discontinuity Bigger region of point? Assume simple background: homogeneous background Noisy environment ? Selection of threshold T

Line detection 1. Line detection mask 2. Which orientation? Thresholding: |R i |  |R j |  T Ex. A thin line: Hori. R 1 =8 45 o R 2 =2 -45 o R 4 =2 Veri. R 3 =2

Example: single-line detection Input: binary imageabsolute value after – 45 o mask |R|  T 1 Detect line that are one pixel thick 2 Isolated “ line points ”

Outline Point, line detection Edge detection Boundary with arbitrary shape Thresholding Use of motion in segmentation

Definition of edges? Intuition of edge: set of connected pixels that line one the boundary between two regions Model of digital edge

Two factors affects acquired edges Blur 模糊化 Sampling rate, illumination condition zoom Noise 雜訊

Method of edge detection: derivatives Edge detection method Magnitude of 1 st derivatives  T 1. Sign determines Dark or light region 2. Zero-crossing

Methods: 1 st derivatives 1st derivatives 2-D gradient Magnitude = Direction =

Example: Sobel filter Original: 1200x1600 Ver. Hor. abs +

Effects of noise Unwanted Detailed Structure !

1st 2nd Noisy edges Fail to detect

Reduce noise: Smoothing Smooth with 5x5 average filter Ver. Hor. abs +

Review 2nd derivative f(x) f(x-1) 1st derivative 2nd derivative f(x)-f(x-1) f(x+1)-2f(x)+f(x-1)

Method: 2nd derivative Edge detection method Magnitude of 1 st derivatives  T 1. Sign determines Dark or light region 2. Zero-crossing 2nd derivative mask Laplacian

Example: 2nd derivative originalSobel Gaussian Smoothing Laplacian (2nd derivative) LoG Threshold(LoG)Zero-crossing of LoG

Outline Point, line detection Edge detection Thresholding Use of motion in segmentation

Foundation of thresholding Idea: object and background pixels have gray levels grouped into two dominant modes Original imagehistogram

Foundation of thresholding Input f(x,y), given threshold T

Issues of thresholding Selection of threshold T ? Complex environment – illumination Multiple thresholds – more than one object Global threshold Local threshold Thresholding as a multi-variable function: g(x,y) = T[ f(x,y), x, y, p(x,y) ] Adaptive: Depend on position Local: local property func.

1. Automatic selection of T 1. Select an initial T Average gray level Mean of max. and min. gray level 2. Segment the image using T T G1G1 G2G2 3. Calculate mean of G 1 and G 2   4. New threshold: T 2 = 0.5(   +   ) T2T2 5. Repeat steps 2~4 until difference in successive T is small

Example: automatically select T fingerprint Initial: gray level mean 3 iterations T = 125.4

2. Effects of illumination Recall: f(x,y)=i(x,y) r(x,y) Illumination source scene reflection illumination:reflectance:

Example: illumination Original image histogram Illumination source x histogram

Example: bad histogram * The gray levels of the object is mixed with background

Matlab code I = imread( ‘ PIC00074.JPG ’, ‘ jpg ’ ); I = rgb2gray(I); I = double( reshape(I, prod(size(I)), 1) ); X=0:255; Y = hist(I, X); bar(Y ’ );

Why illumination is hard to handle? f(x,y)=i(x,y) r(x,y) => z(x,y) = ln f(x,y) = ln i(x,y) + ln r(x,y) Suppose they are independent random variables Histogram (distribution) convolution

3. Multiple thresholds Multiple objects or bad illumination Thresholds:

Result of thresholding 4 gray levels

4. Motivation for adaptive thresholding histogram A single Global threshold

Adaptive local thresholding Subdivide image into blocks Q: Improperly segmented subimages !

Iterative subdivision histogram subdivision

Outline Point, line detection Edge detection Thresholding Use of motion in segmentation

Example:

Motion as a clue to extract object Spatial technique Reference image f(x,y,1) next image f(x,y,2) time index Thresholded difference image 1 if |d(x,y)| > T 0 otherwise

Image difference + thresholding difference >T>T

Use more than one images in time: eliminate noise Reference image R(x,y) Image f(x,y,2) Accumulative difference image counter + 1, Image f(x,y,3) … counter a. if d(x,y) > T positive ADI b. if d(x,y) < -T negative ADI c. if |d(x,y)| > T absolute ADI d(x,y)=R(x,y)-f(x,y,t)

Example: Absolute ADI Positive ADI Negative ADI * Object shape * Location in ref. image

Accumulative difference image(ADI) 13 frames, threshold = 10 Absolute ADIPositive ADI Negative ADI Fail to find object shape ? => Success when object intensity > background