1. Clustering & image segmentation Goal::Identify groups of pixels that go together Segmentation

2. Clustering & image segmentation In segmentation we group together similar-looking pixels The result of the segmentation is the separation of an image into coherent ‘objects’

3. Clustering & image segmentation In segmentation we group together similar-looking pixels The result of the segmentation is the separation of an image into coherent ‘objects’

4. Clustering & image segmentation How? Numerous methods…

6. Basic segmentation General idea:: cluster by features: Color Intensity Location Texture Etc…

7. Clustering & image segmentation Segmentation by clustering:: Basic example 1. Define features (e.g color) Input space

8. Clustering & image segmentation 1. Define features (e.g color) 2. Transform observations (e.g pixels) into feature space Feature spaceInput space Segmentation by clustering:: Basic example

9. Clustering & image segmentation 1. Define features (e.g color) 2. Transform observations (e.g pixels) into feature space Feature spaceInput space 3. Cluster features in feature space Segmentation by clustering:: Basic example

10. Clustering & image segmentation Segmentation = quantization Color (only) based segmentation) Clusters don’t have to be spatially coherent Input space Color segmentation

11. Clustering & image segmentation E.G. Clustering based on {RGB,XY} enforces more spatial coherence w/ spatial features Input space Color+spatial segmentation

13. The fundamental drawback of histogram-based region detection is that histograms provide no spatial information (only the distribution of gray levels) Region growing Region-growing approaches exploit the important fact that pixels which are close together have similar gray values

14. Region growing Region growing in a brief… 1.Choose the seed pixels (1 for every segment) 2.Check the neighboring pixels and add them to the region if they are similar to the seed 3.Repeat step 2 for each of the newly added pixels; stop if no more pixels can be added

15. Region growing Region growing:: under the hood Choosing a seed pixel: Preferably provided by the user. A good seed can be drawn from the peak of the object histogram Minimum area thresholding: No region will be smaller than this threshold in the segmented image

16. Region growing Similarity threshold: If a pixel and a region (or region A and region B) are considered similar enough a union is made, Otherwise a new region is formed High threshold value – easy for new pixels to get accepted to the region Low threshold value – hard for new pixels to get accepted to the region Region growing:: under the hood

17. Region growing Similarity threshold: If a pixel and a region (or region A and region B) are considered similar enough a union is made, Otherwise a new region is formed At each iteration, and for each region Compute arithmetic mean M and standard deviation Region growing:: under the hood Similarity check example::

18. Region growing Similarity threshold: If a pixel and a region (or region A and region B) are considered similar enough a union is made, Otherwise a new region is formed If the regions adhere the similarity (homogeneity) condition than we can unite them Region growing:: under the hood Similarity check example::

19. Region growing Similarity threshold: If a pixel and a region (or region A and region B) are considered similar enough a union is made, Otherwise a new region is formed Other homogeneity criteria (with more features) can be considered… average intensityvariancecolortexture etc… Motion shape size Region growing:: under the hood

20. Region growing Region growing:: Demo

22. Applications and Algorithms in CV Tutorial 4: Clustering & image Segmentation Split & merge Split & Split and merge Split and merge is the opposite of Region growing Split & merge (top bottom) Region growing (bottom up)

23. 1. Split starts from the assumption that the entire image is homogeneous 2. If this is not true (by the homogeneity criterion), the image is split into four sub images 3. This splitting procedure is repeated recursively until we split the image into homogeneous regions Split in a brief… Split & Split and merge

24. Applications and Algorithms in CV Tutorial 4: Clustering & image Segmentation Split & merge Since the procedure is recursive, it produces an image representation that can be described by a tree whose nodes have four sons each *Such a tree is called a ‘Quadtree’. Therefore split is also known as ‘quadtree segmentation’ Split & Split and merge

25. Created regions are adjacent and homogenous Split advantage: Over splitting since no merge is performed Split dis-advantage: Improvement:: Split & merge Split & Split and merge

26. 1. Split (& merge) starts from the assumption that the entire image is homogeneous 2. If this is not true (by the homogeneity criterion), the image is split into four sub images 3. This split is repeated until no further splitting is possible Split & merge in a brief… 4. Merging phase: If 2 adjacent regions are homogenous, they are merged Split & Split and merge 5. Repeat step 4 until no further merging is possible

27. Split vs. Split & merge original Split onlySplit & merge Split & Split and merge

28. Region growing Split & merge:: Demo