Presentation on theme: "Lecture 6 Image Segmentation"— Presentation transcript:
1 Lecture 6 Image Segmentation Slides by:David A. ForsythClark F. OlsonSteven M. SeitzLinda G. Shapiro
2 Segmentation and grouping Motivation: obtain a compact representation for an image (or set of edge pixels or motion sequence, etc.)Should support application.Much work has been done, but broad theory is absent at present.
3 Image segmentationCollecting edge pixels into curves, lines, etc. is one form of segmentation.Another form of segmentation is grouping (or clustering) image pixels into coherent regions.Collect together pixels that “belong together”Segments are usually disjointSegments are often connectedUnion of segments should cover image
4 SegmentationHow do we decide what pixels belong together?
5 Segmentation cuesWhat are properties of pixels (or other objects) that “belong together?”Some criteria that tend to cause tokens to be grouped.
7 Segmentation as thresholding The simplest way to segment a (grey-level) image is to apply a threshold at some brightness value.How do we know what value to use for thresholding?
8 ThresholdingA histogram can give us information about the distribution of pixels.Bimodal distributions are best for this technique.Modes are not always easy to determine.
9 Where do we threshold this distribution? ThresholdingMost images are not straightforward to threshold.Where do we threshold this distribution?
10 Thresholding application Thresholding works best in controlled situations. For example, it is sometimes useful in medical imaging.
11 Segmentation as clustering Goal: cluster together pixels, that belong together.Agglomerative clustering:start with every pixel as separate clustercombine closest two clustersrepeatDivisive clustering:start with every pixel in one clustersplit cluster along best boundaryPixel-Pixel distances:colortextureproximityetc.Cluster-Cluster distancessingle-link clustering- minimum distancecomplete-link clustering- maximum distanceaverage-link clustering- average distance
12 Segmentation as clustering Overall methodology for determining the location and composition of clusters is not straightforward.If we know the cluster locations, how should we assign points to clusters?If we know the assignment of points to clusters, how should we assign locations to clusters?
13 K-means clustering Must choose a fixed number of clusters. Choose cluster centers and point-cluster allocations to minimize error.Can’t do this by search, because there are too many possible allocations.Algorithmfix cluster centers; allocate points to closest clusterfix allocation; compute best cluster centersNote the distance doesn’t need to be just proximity.be careful about scaling
14 K-means clustering How many clusters should you use? K-means clustering will always converge after some number of iterations.However, it won’t always be the global minimum of the error measure:
15 K-means clusteringImageClusters on intensityClusters on colorI gave each pixel the mean intensity or mean color of its cluster --- this is basically just vector quantizing the image intensities/colors. Notice that there is no requirement that clusters be spatially localized and they’re not.K-means clustering using intensity alone and color alone(5 clusters, no location information included in the distance)
16 Divisive clusteringIn divisive clustering, an image is represented by a graph whose nodes are pixels or small groups of pixels.The goal is to partition the vertices into disjoint sets so that the similarity within each set is high andacross different sets is low.
17 Minimal cutsLet G = (V,E) be a graph. Each edge (u,v) has a weight cu,v that represents the similarity between u and v.Graph G can be broken into 2 disjoint graphs with node sets A and B by removing edges that connect these sets.LetOne way to segment G is to find the minimal cut.- Fast algorithms for this exist.
18 Normalized cutsUsing minimal cuts favors small sets of pixels to be divided from a larger group.“Normalized cuts” can improve this.cuts are weighted using the “volume” of the setvolume: all weights that are connected to nodes in the set
19 Normalized cutsExample of divisive clustering using color, texture, and location.Figure from “Image and video segmentation: the normalized cut framework”,by Shi and Malik, copyright IEEE, 1998This is figure caption there explains it all.
20 Segmentation as contour optimization Another approach to segmentation optimizes a contour such that it lies along a strong boundary in the image.
21 Active contoursThese methods start with a particular contour and iteratively improve it until a final state is reached.Several names:Active contoursSnakesIntelligent ScissorsExample from: E. N. Mortensen and W. A. Barrett, Intelligent Scissors for Image Composition, in ACM Computer Graphics (SIGGRAPH `95), pp , 1995
22 Intelligent scissorsThe goal is to find path a between two pixels that stays on image edges.Each pixel is given a cost depending on how “edgy” it is:Edges have low costNon-edge have high costNow must find lowest cost path.
23 Intelligent scissors Can be formulated as a graph algorithm: Each pixel in the image is a nodeThere is a cost between each pair of adjacent pixels (rather than a cost for each pixel)Contour is path between two pixels that minimizes the cost between each pair of adjacent pixel in the contourHow do we determine the “shortest” path in the graph?Can determine all shortest paths for one starting pixel in less than 1 second for reasonable image sizes.
24 Expectation maximization The EM (expectation maximization) algorithm is common in computer vision and is a popular variation of this method.Clusters are represented by Gaussian distributionsE step: assign points to clusters- Assignment is “soft” using probabilityM step: determine mean, standard deviation of Gaussian
25 Segmentation with EM 3 clusters 4 clusters This is figure story in the caption3 clusters4 clustersFigure from “Color and Texture Based Image Segmentation Using EM and Its Application to Content Based Image Retrieval”,S.J. Belongie et al., Proc. Int. Conf. Computer Vision, 1998.