1. May 2003 SUT Color image segmentation – an innovative approach Amin Fazel May 2003 Sharif University of Technology Course Presentation base on a paper by Tie Qi Chen, Yi Lu

2. May 2003 SUT Page 1 of 36 Machine Vision Course Presentation Image segmentation Definition Definition –Process of partitioning image pixel based on selected image features –Pixel of same region spatially connected and have similar image feature –Level of subdivision depends on the problem –Segmentation should stop when the objects of interest have been isolated

3. May 2003 SUT Page 2 of 36 Machine Vision Course Presentation Image segmentation Applications Applications –Image analysis –Machine vision –Target acquisition –Object recognition –…

4. May 2003 SUT Page 3 of 36 Machine Vision Course Presentation Color image segmentation Definition Definition –Here selected segmentation feature is color –This process group pixels into same region that Spatially connected Have similar color feature

5. May 2003 SUT Page 4 of 36 Machine Vision Course Presentation Unsupervised color image segmentation Definition Definition –If below knowledge is not available Number of regions present in the image Type of region present in the image

6. May 2003 SUT Page 5 of 36 Machine Vision Course Presentation color Choosing a suitable color space Common color spaces Common color spaces –RGB, HSI, YIQ, CMY Benefits of L*u*v* color space Benefits of L*u*v* color space 1. covers the whole of the visible gamut of colors 2. the difference between two colors can be measured by their Euclidean distance 3. additive mixture of two colors lies on the line joining them. 4. decreases the chance that any given step in color value will be noticeable on a display

7. May 2003 SUT Page 6 of 36 Machine Vision Course Presentation color Choosing a suitable color space L*u*v* parameters L*u*v* parameters –L* is intensity (lightness) : 0 to 100 –u* is redness-greenness : -127 to 128 –v* is yellowness-blueness : -127 to 128 Converting formula Converting formula –by CIE standard formula RGB and L*u*v* color space

8. May 2003 SUT Page 7 of 36 Machine Vision Course Presentation Color histogram Definition Definition –Three dimensional (3D) discrete feature space –Provide the color distribution of the image –Is obtained by discretizing the colors in the color space counting the number of times each discrete color occurs in the image

9. May 2003 SUT Page 8 of 36 Machine Vision Course Presentation Color histogram Example : Example :

10. May 2003 SUT Page 9 of 36 Machine Vision Course Presentation Overview of the system The system consists of two stages The system consists of two stages 1.Fuzzy clustering algorithm to generate clusters of similar colors Using a color histogram of an image The output of the fuzzy clustering algorithm –Set of non-overlapping color clusters, CL1 Each cluster in CL1 contain similar color All colors in the same cluster are assigned with the same color label

11. May 2003 SUT Page 10 of 36 Machine Vision Course Presentation Overview of the system 2.Region segmentation algorithm agglomerates the initial clusters based on Spatial connection & Color distance between the adjacent regions The second set of clusters, CL2, is obtained by labeling image pixels with the corresponding color clusters in CL1 Therefore, |CL2| >> |CL1| In this stage merges the selected adjacent regions

12. May 2003 SUT Page 11 of 36 Machine Vision Course Presentation A color image segmentation system Stage 1: color color segmentation Fuzzy clustering in color histogram domain color color Image ComputeHistogram In a color space Map initial clusters to image domain CL3:a set of color color region Mergingneighboringclusters CL1 a color histogramCL2 Stage 2: Region segmentation

13. May 2003 SUT Page 12 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm Applying fuzzy logic to color clustering Applying fuzzy logic to color clustering –Consider a cluster of similar colors as a fuzzy set –Represent the likeliness of a color pixel belonging to a fuzzy set by a fuzzy membership function

14. May 2003 SUT Page 13 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm Two critical issues involved in a fuzzy clustering algorithm Two critical issues involved in a fuzzy clustering algorithm –Generating fuzzy membership function –Defining a color distance function between two color clusters and a distance function between a color and a color cluster

15. May 2003 SUT Page 14 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm Fuzzy membership function Fuzzy membership function –Let be the set of possible colors in the image –Use Gaussian function to define the probability of a color C belonging to a color cluster –P is the center of the cluster –R is the radius of the cluster –||-|| denote the Euclidean distance between a color and a cluster

16. May 2003 SUT Page 15 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm …Fuzzy membership function …Fuzzy membership function –The probability of a color belonging to the k-th cluster and not belonging to any other cluster –M is the number of clusters – is used as a fuzzy membership function for color k in the color space

17. May 2003 SUT Page 16 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm Fuzzy membership function Fuzzy membership function

18. May 2003 SUT Page 17 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm Important characteristics of membership function Important characteristics of membership function –Belief value decrease when distance between a color C and a color cluster P increase –Suppresses the belief value of a color to a cluster when it is close to the other clusters Prevent two clusters moving towards each other during the optimization process –The belief value of a color belonging to a cluster is always greater than zero

19. May 2003 SUT Page 18 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm Measure of goodness of fit Measure of goodness of fit –Express how well a given n-cluster description matches a given set of data –Objective function (mean square error) The colors near the border of each cluster give large contribution to the mean square error

20. May 2003 SUT Page 19 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm Algorithm Algorithm First cluster is generated by finding such that – becomes the initial center of cluster 1, i.e. The center of the first cluster is optimized through the following iteration until

21. May 2003 SUT Page 20 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm … Algorithm … Algorithm For M > 1, the initial center of cluster, is set to such that –Function V is the probability of a color not belonging to any existing cluster This new cluster is optimized using the previous iterative procedure The algorithm stops generating a new cluster M when

22. May 2003 SUT Page 21 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm –Effect of objective function used in the cluster generation

23. May 2003 SUT Page 22 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm The only parameters that need to be evaluated are The only parameters that need to be evaluated are –R, the cluster radius –, the distance between the cluster generated in the previous iteration and the current iteration –, the stop threshold of cluster generation Parameters and have less effect on the clustering result The most critical parameter is R

24. May 2003 SUT Page 23 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm More on Cluster Radius More on Cluster Radius –Determines how much the clusters can overlap with each other in the histogram domain –This parameter is provided by the user –For image that have coarse feature a large R is recommended –A smaller R is a good choice for image with fine detailed color feature

25. May 2003 SUT Page 24 of 36 Machine Vision Course Presentation A fuzzy clustering algorithm Effects of the parameter Effects of the parameter R R=64 R=32 R=16 R=8

26. May 2003 SUT Page 25 of 36 Machine Vision Course Presentation Image segmentation in image domain At this stage system map the clusters in CL1 to the image domain to obtain CL2 Each cluster in CL2 contains pixels that are –Spatially connected –Belong to the same color cluster in CL1 –Region segmentation uses following parameters The color distance among neighboring clusters in the spatial domain Cluster size The maximum number of clusters in CL3

27. May 2003 SUT Page 26 of 36 Machine Vision Course Presentation Image segmentation in image domain Investigation of clustering merging methods Investigation of clustering merging methods These methods use a common parameter, max_cls, to control the max number of clusters 1.Attempts to merge the adjacent clusters that are similar in colors –This algorithm use a control parameter to denote color difference threshold

28. May 2003 SUT Page 27 of 36 Machine Vision Course Presentation Image segmentation in image domain 2.Considers the size of clusters as the only selection criterion –It selects the smallest clusters and merges the clusters with one of its neighbors to witch It has the smallest color distance 3.Considers the color distance as the more important criterion in cluster merging –This algorithm consists of three passes of merging

29. May 2003 SUT Page 28 of 36 Machine Vision Course Presentation Image segmentation in image domain 3.…continue I.The algorithm repeatedly merges the smallest clusters with their neighbors that have the closet color distance II.The algorithm selects a pair of two adjacent clusters that has the smallest color distance within the entire image to merge III.The algorithm repeatedly merges the smallest cluster with its closest neighbors in color distance

30. May 2003 SUT Page 29 of 36 Machine Vision Course Presentation Image segmentation in image domain Comparison of clustering result generated by tree different spatial merging method Comparison of clustering result generated by tree different spatial merging method a)An egg nebula image b)Clusters generated by the fuzzy clustering algorithm c)Clustering result by method 1 d)Clustering result by method 2 e)Clustering result by method 3

31. May 2003 SUT Page 30 of 36 Machine Vision Course Presentation Image segmentation in image domain computing the color distance between two neighboring clusters and computing the color distance between two neighboring clusters A and B 1.The first function is based on the color difference of the border pixels of clusters A and B. – –where

32. May 2003 SUT Page 31 of 36 Machine Vision Course Presentation Image segmentation in image domain –and –Where and are the minimum and maximum coordinates of all pixels in A that have direct neighbors in B –Similarly,

33. May 2003 SUT Page 32 of 36 Machine Vision Course Presentation Image segmentation in image domain Illustration of border points between region and Illustration of border points between region A and B –Border (A, B) contain the yellow points within the red bounding box –Border (B, A) contain the blue points within the red bounding box

34. May 2003 SUT Page 33 of 36 Machine Vision Course Presentation Image segmentation in image domain 2.The second color distance function is based on the central color of a cluster defined as: – – where p is a pixel € A – |A| is the size of A – C(p) is the 3-D color vector of pixel p in L*u*v space the color distance of two clusters is measured using the Euclidean distance between their central color vector

35. May 2003 SUT Page 34 of 36 Machine Vision Course Presentation Results of implementation Clustering result Clustering result a)The original image b) 4 clusters generated by the fuzzy clustering algorithm c)4 clusters generated by the segmentation algorithm in image domain

36. May 2003 SUT Page 35 of 36 Machine Vision Course Presentation Results of implementation An example of applying the image segmentation system to a car image An example of applying the image segmentation system to a car image a)The original image b)The image 12 color clusters generated by the fuzzy clustering algorithm and 598 spatial clusters in the image domain c)The segmentation result

37. May 2003 SUT Page 36 of 36 Machine Vision Course Presentation Results of implementation Image segmentation result on two face images Image segmentation result on two face images Setting the cluster radius parameter to R=8, 16, 32 and 64

38. May 2003 SUT Color image segmentation – an innovative approach Course Presentation Based on a paper by Tie Qi Chen, Yi Lu Thanks For Your Attention The End