1. Computer Vision Lecture 12: Image Segmentation II
Demo Website
I highly recommend taking a look at this website:
It has nice interactive demonstrations of the Fourier transform, the Hough transform, edge detection, and many other useful operations.
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

2. Computer Vision Lecture 12: Image Segmentation II
Region Detection
There are two basic – and often complementary – approaches to segmenting an image into individual objects or parts of objects: region-based segmentation and boundary estimation.
Region-based segmentation is based on region detection, which we will discuss in this lecture.
Boundary estimation is based on edge detection, which we already discussed earlier.
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

3. Computer Vision Lecture 12: Image Segmentation II
Region Detection
We have already seen the simplest kind of region detection.
It is the labeling of connected components in binary images.
Of course, in general, region detection is not that simple.
Successful region detection through component labeling requires that we can determine an intensity threshold in such a way that all objects consist of 1-pixels and do not touch each other.
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

4. Computer Vision Lecture 12: Image Segmentation II
Region Detection
We will develop methods that can do a better job at finding regions in real-world images.
In our discussion we will first address the question of how to segment an image into regions.
Afterwards, we will look at different ways to represent the regions that we detected.
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

5. Computer Vision Lecture 12: Image Segmentation II
Region Detection
How shall we define regions?
The basic idea is that within the same region the intensity, texture, or other features do not change abruptly.
Between adjacent regions we do find such a change in at least one feature.
Let us now formalize the idea of partitioning an image into a set of regions.
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

6. Computer Vision Lecture 12: Image Segmentation II
Region Detection
A partition S divides an image I into a set of n regions Ri. Regions are sets of connected pixels meeting three requirements:
The union of regions includes all pixels in the image,
𝑖=1 𝑛 𝑅 𝑖 =𝐼
Each region Ri is homogeneous, i.e., satisfies a homogeneity predicate P so that P(Ri) = True.
The union of two adjacent regions Ri and Rj never satisfies the homogeneity predicate, i.e., P(Ri  Rj) = False.
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

7. Computer Vision Lecture 12: Image Segmentation II
Region Detection
The homogeneity predicate could be defined as, for example, the maximum difference in intensity values between two pixels being no greater than some threshold .
Usually, however, the predicate will be more complex and include other features such as texture.
Also, the parameters of the predicate such as  may be adapted to the properties of the image.
Let us take a look at the split-and-merge algorithm of image segmentation.
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

8. The Split-and-Merge Algorithm
First, we perform splitting:
At the start of the algorithm, the entire image is considered as the candidate region.
If the candidate region does not meet the homogeneity criterion, we split it into four smaller candidate regions.
This is repeated until there are no candidate regions to be split anymore.
Then, we perform merging:
Check all pairs of neighboring regions and merge them if it does not violate the homogeneity criterion.
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

9. The Split-and-Merge Algorithm1 2 3 4 9 8 6 5
Sample image to be segmented with  = 1
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

10. The Split-and-Merge Algorithm1 2 3 4 9 8 6 5
First split
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

11. The Split-and-Merge Algorithm1 2 3 4 9 8 6 5
Second split
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

12. The Split-and-Merge Algorithm1 2 3 4 9 8 6 5
Third split
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

13. The Split-and-Merge Algorithm1 2 3 4 9 8 6 5
Merge
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

14. The Split-and-Merge Algorithm1 2 3 4 9 8 6 5
Final result
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

15. Computer Vision Lecture 12: Image Segmentation II
A Better Criterion
The split-and-merge algorithm is a straightforward way of finding a segmentation of an image that provides homogeneity within regions and non-homogeneity of neighboring regions.
In practice, it is not a good idea to use a maximum intensity difference as the criterion.
A single outlier pixel (black or white) could decide about splitting or merging large regions.
Instead, we should use the standard deviation  of pixel intensities within a region.
Our criterion could be that   max.
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

16. Examples Input image (512×512 pixels) Image after splitting (max= 40)
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

17. Examples Input image (512×512 pixels)
Image after split/merge (max= 40)
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

18. Examples Input image (512×512 pixels)
Image after split/merge (max= 10)
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

19. Image Segmentation by Cluster Analysis
We could also think of regions in images as clusters of pixels that are close to each other spatially and in terms of their intensity (or other features).
There are numerous clustering algorithms such as k-means or hierarchical techniques.
For image segmentation, one such approach, mean shift clustering, has proven to be particularly powerful.
Its basic idea is to, starting at each data point, follow the density gradient to a local density maximum.
Those points being associated with the same maximum are assigned to the same cluster.
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

20. Computer Vision Lecture 12: Image Segmentation II
Mean Shift Clustering
Algorithm:
Place point q on of one of the data points (pixels).
Find the center of gravity c (i.e., the mean) of all data points within a radius r from q.
Move q to point c.
Repeat steps 2 and 3 until the motion is smaller than one pixel per iteration.
Repeat steps 1 to 4 for each data point.
Assign all data points for which q ends up at the same point (within radius r) to the same cluster.
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

21. Computer Vision Lecture 12: Image Segmentation II
Mean Shift Clustering
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

22. Computer Vision Lecture 12: Image Segmentation II
Mean Shift Clustering
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

23. Computer Vision Lecture 12: Image Segmentation II
Mean Shift Clustering
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

24. Computer Vision Lecture 12: Image Segmentation II
Mean Shift Clustering
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

25. Computer Vision Lecture 12: Image Segmentation II
Mean Shift Clustering
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

26. Computer Vision Lecture 12: Image Segmentation II
Mean Shift Clustering
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II

27. Computer Vision Lecture 12: Image Segmentation II
Mean Shift Clustering
March 27, 2018
Computer Vision Lecture 12: Image Segmentation II