Presentation is loading. Please wait.

Presentation is loading. Please wait.

MASTERS THESIS By: Rahul Suresh COMMITTEE MEMBERS Dr.Stan Birchfield Dr.Adam Hoover Dr.Brian Dean.

Similar presentations


Presentation on theme: "MASTERS THESIS By: Rahul Suresh COMMITTEE MEMBERS Dr.Stan Birchfield Dr.Adam Hoover Dr.Brian Dean."— Presentation transcript:

1 MASTERS THESIS By: Rahul Suresh COMMITTEE MEMBERS Dr.Stan Birchfield Dr.Adam Hoover Dr.Brian Dean

2  Introduction  Related work  Background theory: ◦ Image as a graph ◦ Kruskals’ Minimum Spanning Tree ◦ MST based segmentation  Our algorithm  Results  Conclusion and future work

3  Dividing an image to disjoint regions such that similar pixels are grouped together Image Courtesy: [3]

4  Image Segmentation involves division of image I into K regions: R 1, R 2, R 3, … R K such that: Every pixel must be assigned to a region Regions must be disjoint Size: 1 pixel to the entire image itself

5  Pixels within a region share certain characteristics that is not found with pixels in another region. f is a function that returns TRUE if the region under consideration is homogenous

6  Biomedical applications ◦ Used as a preprocessing step to identify anatomical regions for medical diagnosis/analysis. Brain Tissue MRI Segmentation [1] CT Jaw segmentation [2]

7  Object recognition systems: ◦ Lower level features such as color and texture are used to segment the image ◦ Only relevant segments (subset of pixels) are fed to the object recognition system.  Saves computational cost, especially for large scale recognition systems

8  As a preprocessing step in face and iris recognition Face segmentationIris Segmentation

9  Astronomy: Preprocessing step before further analysis Segmentation of Nebula [4]

10 Which segmentation is “correct”? (Manual segmentations from BSDS)

11  “Correctness”- Are similar pixels grouped together and dissimilar pixels grouped seperately?  Granularity- Extent of resolution of segmentation ◦ Consider example in the previous image

12  There is ambiguity in defining “good”/ “optimal” segmentation.  An image can have multiple segmentations. “correct” ◦ Make evaluation /benchmarking of segmentation algorithm hard

13  Related work  Background theory: ◦ Image as a graph ◦ Kruskals’ Minimum Spanning Tree ◦ MST based segmentation  Our algorithm  Results  Conclusion and future work

14  Related work  Background theory: ◦ Image as a graph ◦ Kruskals’ Minimum Spanning Tree ◦ MST based segmentation  Our algorithm  Results  Conclusion and future work

15 Some of the popular image segmentation approaches are:  Split and Merge approaches  Mean Shift and k-means  Spectral theory and normalized cuts  Minimum spanning tree

16  Split and Merge approaches ◦ Iteratively split  If evidence of a boundary exists ◦ Iteratively merge  Based on similarity  Quad-tree used Image Courtesy: [5]

17  Mean shift and k-means are related.  Mean-shift: ◦ Represent each pixel as a vector [color, texture, space] ◦ Define a window around every point. 1.Update the point to the mean of all the points within the window. 2.Repeat until convergence.

18  K-means: ◦ Represent each pixel as vector [color, texture, space] ◦ Choose K initial cluster centers 1.Assign every pixel to its closet cluster center. 2.Recompute the means of all the clusters 3.Repeat 1-2 until convergence.  Difference between K means and mean-shift: ◦ In K-means, K has to be known beforehand ◦ K-means sensitive to initial choice of cluster centers

19  Represent image as a graph.  Using graph cuts, partitions the image into regions.  In Spectral theory and normalized cuts,  Eigenvalues/vectors of the Laplacian matrix is used to determine the cut

20  Use Minimum Spanning Tree to segment image. ◦ Proposed by Felzenszwalb & Huttenlocher in ◦ Uses a variant of Kruskals MST to segment images ◦ Very efficient- O(NlogN) time  Discussed in detail in the next section

21  Related work  Background theory: ◦ Image as a graph ◦ Kruskals’ Minimum Spanning Tree ◦ MST based segmentation  Our algorithm  Results  Conclusion and future work

22  Graph G=(V,E) is an abstract data type containing a set of vertices V and edges E.  Useful operations using a graph: ◦ See if path exists between any 2 vertices ◦ Find connected components ◦ Check for cycles ◦ Find the shortest point between any 2 vertices ◦ Compute minimum spanning ◦ Graph partition based on cuts  Graph algorithms are useful in image processing

23  Image graph: ◦ Pixels/group of pixels form vertices. ◦ Vertices connected to form edges ◦ Edge weight represents dissimilarity between vertices  Types of image graph: ◦ Image grid ◦ Complete graph ◦ Nearest neighbor graph

24  Image grid: ◦ Edges: every vertex (pixel) is connected with its 4 (or 8) x-y neighbors. ◦ No of edges m= O(N) [Graph operations are quick] ◦ Fails to capture global properties

25  Complete graph: ◦ Edges: Connect every vertex (pixel) with every other vertex ◦ No of edges m= O(N 2 ) ◦ Captures global properties ◦ Graph operations are very expensive

26  Nearest neighbor graph: ◦ Compromise between grid (fails to capture global properties) and complete graph (too many edges). ◦ Represent every vertex as a combination of color and x-y features. [e.g. (R, G, B, x, y)] ◦ Find the K=O(1) neighbors for each pixel using Approximate nearest neighbor (ANN) ◦ Edges: Connect every pixel to K nearest neighbors

27  Related work  Background theory: ◦ Image as a graph ◦ Kruskal’s Minimum Spanning Tree ◦ MST based segmentation  Our algorithm  Results  Conclusion and future work

28  Tree is a graph which is: ◦ Connected ◦ Has no cycles  Spanning tree: contains all the vertices of graph G ◦ A graph can have multiple spanning trees  Minimum spanning tree is a spanning tree which has the least sum of weights of edges

29 Sorting: O(mlog(m)) time FindSet and Merge: O(mα(N)) time [very slow growing] OVERALL TIME: O(m log(m))

30  Related work  Background theory: ◦ Image as a graph ◦ Kruskal’s Minimum Spanning Tree ◦ MST based segmentation  Our algorithm  Results  Conclusion and future work

31  Use Minimum Spanning Tree to segment image.  In Kruskal’s MST algorithm, ◦ Edges are sorted in ascending order of weights ◦ Edges are added in order to the spanning tree as long as a cycle is not formed. ◦ All vertices added to ONE spanning tree  If Kruskal’s is applied directly to image segmentation: ◦ We will end up with ONE segment (entire image)

32  Variant of Kruskal’s used in image segmentation. 1. Create an image grid graph. 2. Sort edges in the increasing order of weights 3. For every edge e i in E, 1.If FindSet(u i ) ≠ FindSet(v i ) AND IsSimilar(u i, v i )=TRUE Merge(FindSet(u i ),FindSet(v i ) )  Instead of one MST, we end up with a forest of K trees ◦ Each tree represents a region

33  We add an edge e i connecting regions R u and R v to a tree only if : D(R u R v ): edge weight connecting vertices u and v Int(R i ): maximum edge weight in region R i WE MERGE IF THE EDGE WEIGHT IS LOWER THAN THE MAXIMUM EDGE WEIGHT IN EITHER REGIONS!!

34  Drawback 1: LEAK Felzenszwalb and Huttenlocher 2004

35  Drawback 2: SENSITIVITY TO PARAMETER k ◦ Notice how granularity changes by varying k k is arbitrary k is affected by the size of the image

36  Related work  Background theory: ◦ Image as a graph ◦ Kruskal’s Minimum Spanning Tree ◦ MST based segmentation  Our algorithm  Results  Conclusion and future work

37  Objective  Constructing image grid  Sort edges in ascending order  For every edge ◦ If Merge criterion is satisfied  Merge

38  Improve upon the drawbacks of MST ALGORITHM: ◦ Addressing Leak:  Represent regions as a Gaussian distribution.  Use Bidirectional Mahalanobis distance to compare Gaussians. ◦ Overcome sensitivity to parameter k:  Propose parameter τ that is  independent of image size  Works well for  Provide a mathematical intuition for it.  Propose an approximation that enables real-time implementation.

39 D(u,v) u v Int(R u )Int(R v ) Check if D(u,v) < Int(Ru) && D(u,v) < Int(Rv) Leak can happen

40 D(u,v) u v  Represent each region as a Gaussian  Check if the Gaussians are similar:  Mahalanobis distance is less than 2.5

41  Objective  Constructing image grid  Sort edges in ascending order  For every edge ◦ If Merge criterion is satisfied  Merge

42  Initialize Vertices: ◦ Every pixel is mapped to a vertex ◦ Information about vertex v i is stored at the ‘i’th entry of the disjoint set data structure D.  The ‘i’th entry in D contains following information: ◦ Root node ◦ Zeroth, first and second order moments ◦ List of all the edges connected to vertex v i

43  Initialize Edges: ◦ Between neighboring pixels in x-y space ◦ Number of edges m= O(N) ◦ Use List to maintain edges  Edge weight: ◦ Euclidean distance between pixels to begin with ◦ Mahalanobis distance between Gaussians as region grows  Note that Euclidean distance is a special instance of Mahalanobis distance

44  Objective  Constructing image grid  Sort edges in ascending order  For every edge ◦ If Merge criterion is satisfied  Merge

45

46  Objective  Constructing image grid  Sort edges in ascending order  For every edge ◦ If Merge criterion is satisfied  Merge

47  While adding edge e i to the MST, regions R u and R v are merged if the following criterion is satisfied: Forces small regions to merge Bidirectional Mahalanobis Around 2.5 is a good threshold

48  Objective  Constructing image grid  Sort edges in ascending order  For every edge ◦ If Merge criterion is satisfied  Merge

49  Merging regions R u and R v ◦ Update information at the root node of the disjoint set data- structure (Similar to MST) ◦ Updating information about root node, zeroth, first and second order moment is easy  However, after merging R u and R v ◦ All edges connected to either R u or R v have to be updated w.r.t. (R u ∪ R v ) ◦ The edges have to be re-sorted. ◦ The above operations will slow down the overall running time to O(N 2 ).

50  To speed up weight update that needs to be performed after every iteration, ◦ For every region in the DSDS, we store the pointers to all the edges connected to it. ◦ When 2 regions are merged, we merge their neighbor lists also ◦ Assuming that the number of neighbors for every region is constant, every iteration of merging neighbors can also be accomplished in O(1) time

51  Re-sorting the edges after every merge is an expensive operation. ◦ We use skip lists to maintain edges. ◦ Skip list is a data structure that helps maintain sorted items.  Every insert, delete and search operation takes O(logN) amortized time.  Although the asymptotic running time is O(NplogN), it is still slower than MST

52  Do not update weights or re-sort edges after every iteration.  This runs in O(NlogN) time ◦ Speed comparable to MST  Our experiments show that the approximated algorithm still improves upon the drawback of MST: ◦ Leak ◦ Sensitivity to parameter k

53  Related work  Background theory: ◦ Image as a graph ◦ Kruskals’ Minimum Spanning Tree ◦ MST based segmentation  Our algorithm  Results  Conclusion and future work

54  Tested the algorithm on: ◦ Synthetic images ◦ Berkeley Segmentation Dataset  Compared its performance with MST based segmentation algorithm

55  Our algorithm overcomes leak! Gradient rampMST merges the entire image into 1 region Our algorithm has 2 stable regions

56  Effect of parameter tau on granularity MST based Segmentation Our algorithm k is arbitrary! Varies for different image sizes τ represents the distance between Gaussians! Best results when 2< τ<2.5

57

58  We ran the segmentation exhaustively for multiple values of τ  Studied the effect of τ on the number of regions formed  Notice that curve flattens for τ in the range ◦ Represents “stable” regions ◦ Segmentation unaffected by parameter change Mona Lisa Man Notice the flat regions cf. Yu 2007

59  We ran the algorithm exhaustively on Berkeley Segmentation dataset.  Our algorithm produced more “correct” segmentations than MST segmentations.  Segmentation was sharper in our algorithm.  Some specific example illustrated in the subsequent slides

60 Notice the Man on the Hill

61 Notice the face of the man kneeling down!

62 Notice how a small leak has merged grass with part of bison’s body

63

64  Related work  Background theory: ◦ Image as a graph ◦ Kruskal’s Minimum Spanning Tree ◦ MST based segmentation  Our algorithm  Results  Conclusion and future work

65  Proposed a new segmentation algorithm that improves upon the drawbacks of MST: ◦ Leak  Represent regions as Gaussians  Use bidirectional Mahalanobis distance to compare regions ◦ Sensitivity to parameter k  τ = 2.5 works well for all images, represents normalized distance between Gaussian distributions  Shown experimentally to be “stable”

66  In the worst case scenerio, ◦ naïve version of our algorithm runs in O(N 2 ) time. ◦ Using skip list improves to O(NlogN) but still not as fast as MST  An approximated algorithm is proposed: ◦ Runs in O(NlogN) time and speed comparable to MST ◦ Still overcomes the two drawbacks of MST based segmentation

67  Pre-processing: Homographic filtering  Efficiency: Speed up the original version of the algorithm using more sophisticated priority queues.  Benchmarking: Mathematical study of the accuracy of our segmentation algorithm on BSDS dataset

68 [1] [2] [3] [4] [5] Stan Birchfield. Image Segmentation. Lecture Notes


Download ppt "MASTERS THESIS By: Rahul Suresh COMMITTEE MEMBERS Dr.Stan Birchfield Dr.Adam Hoover Dr.Brian Dean."

Similar presentations


Ads by Google