Download presentation

Presentation is loading. Please wait.

Published byJulian Clem Modified about 1 year ago

1
MASTERS THESIS By: Rahul Suresh COMMITTEE MEMBERS Dr.Stan Birchfield Dr.Adam Hoover Dr.Brian Dean

2
Introduction Related work Background theory: ◦ Image as a graph ◦ Kruskals’ Minimum Spanning Tree ◦ MST based segmentation Our algorithm Results Conclusion and future work

3
Dividing an image to disjoint regions such that similar pixels are grouped together Image Courtesy: [3]

4
Image Segmentation involves division of image I into K regions: R 1, R 2, R 3, … R K such that: Every pixel must be assigned to a region Regions must be disjoint Size: 1 pixel to the entire image itself

5
Pixels within a region share certain characteristics that is not found with pixels in another region. f is a function that returns TRUE if the region under consideration is homogenous

6
Biomedical applications ◦ Used as a preprocessing step to identify anatomical regions for medical diagnosis/analysis. Brain Tissue MRI Segmentation [1] CT Jaw segmentation [2]

7
Object recognition systems: ◦ Lower level features such as color and texture are used to segment the image ◦ Only relevant segments (subset of pixels) are fed to the object recognition system. Saves computational cost, especially for large scale recognition systems

8
As a preprocessing step in face and iris recognition Face segmentationIris Segmentation

9
Astronomy: Preprocessing step before further analysis Segmentation of Nebula [4]

10
Which segmentation is “correct”? (Manual segmentations from BSDS)

11
“Correctness”- Are similar pixels grouped together and dissimilar pixels grouped seperately? Granularity- Extent of resolution of segmentation ◦ Consider example in the previous image

12
There is ambiguity in defining “good”/ “optimal” segmentation. An image can have multiple segmentations. “correct” ◦ Make evaluation /benchmarking of segmentation algorithm hard

13
Related work Background theory: ◦ Image as a graph ◦ Kruskals’ Minimum Spanning Tree ◦ MST based segmentation Our algorithm Results Conclusion and future work

14
Related work Background theory: ◦ Image as a graph ◦ Kruskals’ Minimum Spanning Tree ◦ MST based segmentation Our algorithm Results Conclusion and future work

15
Some of the popular image segmentation approaches are: Split and Merge approaches Mean Shift and k-means Spectral theory and normalized cuts Minimum spanning tree

16
Split and Merge approaches ◦ Iteratively split If evidence of a boundary exists ◦ Iteratively merge Based on similarity Quad-tree used Image Courtesy: [5]

17
Mean shift and k-means are related. Mean-shift: ◦ Represent each pixel as a vector [color, texture, space] ◦ Define a window around every point. 1.Update the point to the mean of all the points within the window. 2.Repeat until convergence.

18
K-means: ◦ Represent each pixel as vector [color, texture, space] ◦ Choose K initial cluster centers 1.Assign every pixel to its closet cluster center. 2.Recompute the means of all the clusters 3.Repeat 1-2 until convergence. Difference between K means and mean-shift: ◦ In K-means, K has to be known beforehand ◦ K-means sensitive to initial choice of cluster centers

19
Represent image as a graph. Using graph cuts, partitions the image into regions. In Spectral theory and normalized cuts, Eigenvalues/vectors of the Laplacian matrix is used to determine the cut

20
Use Minimum Spanning Tree to segment image. ◦ Proposed by Felzenszwalb & Huttenlocher in ◦ Uses a variant of Kruskals MST to segment images ◦ Very efficient- O(NlogN) time Discussed in detail in the next section

21
Related work Background theory: ◦ Image as a graph ◦ Kruskals’ Minimum Spanning Tree ◦ MST based segmentation Our algorithm Results Conclusion and future work

22
Graph G=(V,E) is an abstract data type containing a set of vertices V and edges E. Useful operations using a graph: ◦ See if path exists between any 2 vertices ◦ Find connected components ◦ Check for cycles ◦ Find the shortest point between any 2 vertices ◦ Compute minimum spanning ◦ Graph partition based on cuts Graph algorithms are useful in image processing

23
Image graph: ◦ Pixels/group of pixels form vertices. ◦ Vertices connected to form edges ◦ Edge weight represents dissimilarity between vertices Types of image graph: ◦ Image grid ◦ Complete graph ◦ Nearest neighbor graph

24
Image grid: ◦ Edges: every vertex (pixel) is connected with its 4 (or 8) x-y neighbors. ◦ No of edges m= O(N) [Graph operations are quick] ◦ Fails to capture global properties

25
Complete graph: ◦ Edges: Connect every vertex (pixel) with every other vertex ◦ No of edges m= O(N 2 ) ◦ Captures global properties ◦ Graph operations are very expensive

26
Nearest neighbor graph: ◦ Compromise between grid (fails to capture global properties) and complete graph (too many edges). ◦ Represent every vertex as a combination of color and x-y features. [e.g. (R, G, B, x, y)] ◦ Find the K=O(1) neighbors for each pixel using Approximate nearest neighbor (ANN) ◦ Edges: Connect every pixel to K nearest neighbors

27
Related work Background theory: ◦ Image as a graph ◦ Kruskal’s Minimum Spanning Tree ◦ MST based segmentation Our algorithm Results Conclusion and future work

28
Tree is a graph which is: ◦ Connected ◦ Has no cycles Spanning tree: contains all the vertices of graph G ◦ A graph can have multiple spanning trees Minimum spanning tree is a spanning tree which has the least sum of weights of edges

29
Sorting: O(mlog(m)) time FindSet and Merge: O(mα(N)) time [very slow growing] OVERALL TIME: O(m log(m))

30
Related work Background theory: ◦ Image as a graph ◦ Kruskal’s Minimum Spanning Tree ◦ MST based segmentation Our algorithm Results Conclusion and future work

31
Use Minimum Spanning Tree to segment image. In Kruskal’s MST algorithm, ◦ Edges are sorted in ascending order of weights ◦ Edges are added in order to the spanning tree as long as a cycle is not formed. ◦ All vertices added to ONE spanning tree If Kruskal’s is applied directly to image segmentation: ◦ We will end up with ONE segment (entire image)

32
Variant of Kruskal’s used in image segmentation. 1. Create an image grid graph. 2. Sort edges in the increasing order of weights 3. For every edge e i in E, 1.If FindSet(u i ) ≠ FindSet(v i ) AND IsSimilar(u i, v i )=TRUE Merge(FindSet(u i ),FindSet(v i ) ) Instead of one MST, we end up with a forest of K trees ◦ Each tree represents a region

33
We add an edge e i connecting regions R u and R v to a tree only if : D(R u R v ): edge weight connecting vertices u and v Int(R i ): maximum edge weight in region R i WE MERGE IF THE EDGE WEIGHT IS LOWER THAN THE MAXIMUM EDGE WEIGHT IN EITHER REGIONS!!

34
Drawback 1: LEAK Felzenszwalb and Huttenlocher 2004

35
Drawback 2: SENSITIVITY TO PARAMETER k ◦ Notice how granularity changes by varying k k is arbitrary k is affected by the size of the image

36
Related work Background theory: ◦ Image as a graph ◦ Kruskal’s Minimum Spanning Tree ◦ MST based segmentation Our algorithm Results Conclusion and future work

37
Objective Constructing image grid Sort edges in ascending order For every edge ◦ If Merge criterion is satisfied Merge

38
Improve upon the drawbacks of MST ALGORITHM: ◦ Addressing Leak: Represent regions as a Gaussian distribution. Use Bidirectional Mahalanobis distance to compare Gaussians. ◦ Overcome sensitivity to parameter k: Propose parameter τ that is independent of image size Works well for Provide a mathematical intuition for it. Propose an approximation that enables real-time implementation.

39
D(u,v) u v Int(R u )Int(R v ) Check if D(u,v) < Int(Ru) && D(u,v) < Int(Rv) Leak can happen

40
D(u,v) u v Represent each region as a Gaussian Check if the Gaussians are similar: Mahalanobis distance is less than 2.5

41
Objective Constructing image grid Sort edges in ascending order For every edge ◦ If Merge criterion is satisfied Merge

42
Initialize Vertices: ◦ Every pixel is mapped to a vertex ◦ Information about vertex v i is stored at the ‘i’th entry of the disjoint set data structure D. The ‘i’th entry in D contains following information: ◦ Root node ◦ Zeroth, first and second order moments ◦ List of all the edges connected to vertex v i

43
Initialize Edges: ◦ Between neighboring pixels in x-y space ◦ Number of edges m= O(N) ◦ Use List to maintain edges Edge weight: ◦ Euclidean distance between pixels to begin with ◦ Mahalanobis distance between Gaussians as region grows Note that Euclidean distance is a special instance of Mahalanobis distance

44
Objective Constructing image grid Sort edges in ascending order For every edge ◦ If Merge criterion is satisfied Merge

45

46
Objective Constructing image grid Sort edges in ascending order For every edge ◦ If Merge criterion is satisfied Merge

47
While adding edge e i to the MST, regions R u and R v are merged if the following criterion is satisfied: Forces small regions to merge Bidirectional Mahalanobis Around 2.5 is a good threshold

48
Objective Constructing image grid Sort edges in ascending order For every edge ◦ If Merge criterion is satisfied Merge

49
Merging regions R u and R v ◦ Update information at the root node of the disjoint set data- structure (Similar to MST) ◦ Updating information about root node, zeroth, first and second order moment is easy However, after merging R u and R v ◦ All edges connected to either R u or R v have to be updated w.r.t. (R u ∪ R v ) ◦ The edges have to be re-sorted. ◦ The above operations will slow down the overall running time to O(N 2 ).

50
To speed up weight update that needs to be performed after every iteration, ◦ For every region in the DSDS, we store the pointers to all the edges connected to it. ◦ When 2 regions are merged, we merge their neighbor lists also ◦ Assuming that the number of neighbors for every region is constant, every iteration of merging neighbors can also be accomplished in O(1) time

51
Re-sorting the edges after every merge is an expensive operation. ◦ We use skip lists to maintain edges. ◦ Skip list is a data structure that helps maintain sorted items. Every insert, delete and search operation takes O(logN) amortized time. Although the asymptotic running time is O(NplogN), it is still slower than MST

52
Do not update weights or re-sort edges after every iteration. This runs in O(NlogN) time ◦ Speed comparable to MST Our experiments show that the approximated algorithm still improves upon the drawback of MST: ◦ Leak ◦ Sensitivity to parameter k

53
Related work Background theory: ◦ Image as a graph ◦ Kruskals’ Minimum Spanning Tree ◦ MST based segmentation Our algorithm Results Conclusion and future work

54
Tested the algorithm on: ◦ Synthetic images ◦ Berkeley Segmentation Dataset Compared its performance with MST based segmentation algorithm

55
Our algorithm overcomes leak! Gradient rampMST merges the entire image into 1 region Our algorithm has 2 stable regions

56
Effect of parameter tau on granularity MST based Segmentation Our algorithm k is arbitrary! Varies for different image sizes τ represents the distance between Gaussians! Best results when 2< τ<2.5

57

58
We ran the segmentation exhaustively for multiple values of τ Studied the effect of τ on the number of regions formed Notice that curve flattens for τ in the range ◦ Represents “stable” regions ◦ Segmentation unaffected by parameter change Mona Lisa Man Notice the flat regions cf. Yu 2007

59
We ran the algorithm exhaustively on Berkeley Segmentation dataset. Our algorithm produced more “correct” segmentations than MST segmentations. Segmentation was sharper in our algorithm. Some specific example illustrated in the subsequent slides

60
Notice the Man on the Hill

61
Notice the face of the man kneeling down!

62
Notice how a small leak has merged grass with part of bison’s body

63

64
Related work Background theory: ◦ Image as a graph ◦ Kruskal’s Minimum Spanning Tree ◦ MST based segmentation Our algorithm Results Conclusion and future work

65
Proposed a new segmentation algorithm that improves upon the drawbacks of MST: ◦ Leak Represent regions as Gaussians Use bidirectional Mahalanobis distance to compare regions ◦ Sensitivity to parameter k τ = 2.5 works well for all images, represents normalized distance between Gaussian distributions Shown experimentally to be “stable”

66
In the worst case scenerio, ◦ naïve version of our algorithm runs in O(N 2 ) time. ◦ Using skip list improves to O(NlogN) but still not as fast as MST An approximated algorithm is proposed: ◦ Runs in O(NlogN) time and speed comparable to MST ◦ Still overcomes the two drawbacks of MST based segmentation

67
Pre-processing: Homographic filtering Efficiency: Speed up the original version of the algorithm using more sophisticated priority queues. Benchmarking: Mathematical study of the accuracy of our segmentation algorithm on BSDS dataset

68
[1] [2] [3] [4] [5] Stan Birchfield. Image Segmentation. Lecture Notes

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google