Presentation is loading. Please wait.

Presentation is loading. Please wait.

Lecture 6 Image Segmentation

Similar presentations

Presentation on theme: "Lecture 6 Image Segmentation"— Presentation transcript:

1 Lecture 6 Image Segmentation
Slides by: David A. Forsyth Clark F. Olson Steven M. Seitz Linda G. Shapiro

2 Segmentation and grouping
Motivation: obtain a compact representation for an image (or set of edge pixels or motion sequence, etc.) Should support application. Much work has been done, but broad theory is absent at present.

3 Image segmentation Collecting edge pixels into curves, lines, etc. is one form of segmentation. Another form of segmentation is grouping (or clustering) image pixels into coherent regions. Collect together pixels that “belong together” Segments are usually disjoint Segments are often connected Union of segments should cover image

4 Segmentation How do we decide what pixels belong together?

5 Segmentation cues What are properties of pixels (or other objects) that “belong together?” Some criteria that tend to cause tokens to be grouped.

6 Segmentation cues More such

7 Segmentation as thresholding
The simplest way to segment a (grey-level) image is to apply a threshold at some brightness value. How do we know what value to use for thresholding?

8 Thresholding A histogram can give us information about the distribution of pixels. Bimodal distributions are best for this technique. Modes are not always easy to determine.

9 Where do we threshold this distribution?
Thresholding Most images are not straightforward to threshold. Where do we threshold this distribution?

10 Thresholding application
Thresholding works best in controlled situations. For example, it is sometimes useful in medical imaging.

11 Segmentation as clustering
Goal: cluster together pixels, that belong together. Agglomerative clustering: start with every pixel as separate cluster combine closest two clusters repeat Divisive clustering: start with every pixel in one cluster split cluster along best boundary Pixel-Pixel distances: color texture proximity etc. Cluster-Cluster distances single-link clustering - minimum distance complete-link clustering - maximum distance average-link clustering - average distance

12 Segmentation as clustering
Overall methodology for determining the location and composition of clusters is not straightforward. If we know the cluster locations, how should we assign points to clusters? If we know the assignment of points to clusters, how should we assign locations to clusters?

13 K-means clustering Must choose a fixed number of clusters.
Choose cluster centers and point-cluster allocations to minimize error. Can’t do this by search, because there are too many possible allocations. Algorithm fix cluster centers; allocate points to closest cluster fix allocation; compute best cluster centers Note the distance doesn’t need to be just proximity. be careful about scaling

14 K-means clustering How many clusters should you use?
K-means clustering will always converge after some number of iterations. However, it won’t always be the global minimum of the error measure:

15 K-means clustering Image Clusters on intensity Clusters on color I gave each pixel the mean intensity or mean color of its cluster --- this is basically just vector quantizing the image intensities/colors. Notice that there is no requirement that clusters be spatially localized and they’re not. K-means clustering using intensity alone and color alone (5 clusters, no location information included in the distance)

16 Divisive clustering In divisive clustering, an image is represented by a graph whose nodes are pixels or small groups of pixels. The goal is to partition the vertices into disjoint sets so that the similarity within each set is high and across different sets is low.

17 Minimal cuts Let G = (V,E) be a graph. Each edge (u,v) has a weight cu,v that represents the similarity between u and v. Graph G can be broken into 2 disjoint graphs with node sets A and B by removing edges that connect these sets. Let One way to segment G is to find the minimal cut. - Fast algorithms for this exist.

18 Normalized cuts Using minimal cuts favors small sets of pixels to be divided from a larger group. “Normalized cuts” can improve this. cuts are weighted using the “volume” of the set volume: all weights that are connected to nodes in the set

19 Normalized cuts Example of divisive clustering using color, texture, and location. Figure from “Image and video segmentation: the normalized cut framework”, by Shi and Malik, copyright IEEE, 1998 This is figure caption there explains it all.

20 Segmentation as contour optimization
Another approach to segmentation optimizes a contour such that it lies along a strong boundary in the image.

21 Active contours These methods start with a particular contour and iteratively improve it until a final state is reached. Several names: Active contours Snakes Intelligent Scissors Example from: E. N. Mortensen and W. A. Barrett, Intelligent Scissors for Image Composition, in ACM Computer Graphics (SIGGRAPH `95), pp , 1995

22 Intelligent scissors The goal is to find path a between two pixels that stays on image edges. Each pixel is given a cost depending on how “edgy” it is: Edges have low cost Non-edge have high cost Now must find lowest cost path.

23 Intelligent scissors Can be formulated as a graph algorithm:
Each pixel in the image is a node There is a cost between each pair of adjacent pixels (rather than a cost for each pixel) Contour is path between two pixels that minimizes the cost between each pair of adjacent pixel in the contour How do we determine the “shortest” path in the graph? Can determine all shortest paths for one starting pixel in less than 1 second for reasonable image sizes.

24 Expectation maximization
The EM (expectation maximization) algorithm is common in computer vision and is a popular variation of this method. Clusters are represented by Gaussian distributions E step: assign points to clusters - Assignment is “soft” using probability M step: determine mean, standard deviation of Gaussian

25 Segmentation with EM 3 clusters 4 clusters
This is figure story in the caption 3 clusters 4 clusters Figure from “Color and Texture Based Image Segmentation Using EM and Its Application to Content Based Image Retrieval”,S.J. Belongie et al., Proc. Int. Conf. Computer Vision, 1998.

Download ppt "Lecture 6 Image Segmentation"

Similar presentations

Ads by Google