1. Globally Optimal Grouping for Symmetric Closed Boundaries with Stahl/ Wang Method Vida Movahedi September 2007

2. 2/31Introduction Grouping seeks to identify some perceptually salient structure in noisy images Minimize a pre-defined grouping cost (function) that negatively measures the perceptual saliency of the resulting structure based on some psychological vision rules, such as the Gestalt laws

3. 3/31 Combine Boundary & Region info Two boundary properties: –proximity and closure One region property: –enclosed region area Grouping cost: –Ratio between the total gap length along the boundary and the area enclosed by the boundary

4. 4/31 Problem Formulation Step 1: edge detection, construct a set of line segments Step 2: gap-filling (line) segments Step 3: Find a boundary that minimizes the grouping cost

5. 5/31 Grouping Cost Solid and dashed edges Graph G can be called a solid-dashed (SD) graph Alternate cycle (alternating between solid and dashed) Construct a pair of edges e + and e - for each line segment (e + left to right direction)

6. 6/31 Grouping Cost (Cont.) Define two edge-weight functions w 1 (e) = 0 if solid edge = length of the line segment if dashed w 2 (e) = signed area associated to the corresponding line segment

7. 7/31 Grouping Cost (Cont.) Cost function= W 1 /W 2 Looking for a C that minimizes above cost Corresponds to a boundary B, that minimizes

8. 8/31 Symmetry as a cue Many structures show (bilateral) symmetry Grouping for symmetric boundaries is a challenging problem: a)Boundary symmetry is not a simple local measure b)Need a unified grouping cost to flexibly integrate different grouping cues (proximity, etc) c)The cost should avoid undesirable explicit or implicit biases, such as bias toward shorter boundaries

9. 9/31 Changing the grouping tokens Previously the grouping tokens were line segments, detected (solid) segments vs. gap- filling (dashed) segments n detected segments, 2n endpoints, ideally n(2n- 2) gap filling segments Encoding symmetry into an individual line segment?! Symmetry can be encoded to a pair of segments  Symmetric Trapezoids

10. 10/31 Symmetric Trapezoids 1)Find the angle-bisector line l 2)Find the projections of segments to l called the axis segment 3)Map this axis segment back to segments, resulting in a (symmetric) trapezoid If no overlaps, no symmetric trapezoid constructed pair every two detected segments, or pair a gap-filling segment with a detected segment

11. 11/31 Gap-filling quadrilaterals constructed by connecting a parallel side of one trapezoid and a parallel side of another trapezoid May not be symmetric, its axis segment constructed simply by connecting the endpoints of the axis segments of two neighboring symmetric trapezoid Two endpoints for a trapezoid  four different gap- filling quadrilaterals

12. 12/31 Grouping Cost Function : The total gap along the boundary, reflecting the preference of a boundary with good proximity : A measure related to the collinearity of the boundary’s axis, reflecting the preference of a boundary with good symmetry : The region area enclosed by the boundary, sets a preference to produce larger rounder structures

13. 13/31 Graph Modeling Model the trapezoids and quadrilaterals using an undirected graph A pair of solid edges e T + and e T - for each trapezoid T A pair of dashed edges e G + and e G - for each quadrilateral G mirror edges: an abstraction of the axis segment

14. 14/31 Graph Modeling (Cont.) Consider only the quadrilaterals that lead to a non-intersected boundary (not P 2 P 6 P 3 P 7 ) Special case where the constructed gap-filling quadrilateral contains a self intersection (P 2 P 3 and P 6 P 7 not intersecting) e + for counterclockwise, e - for clockwise

15. 15/31 Graph Modeling (Cont.)

16. 16/31 Edge Weight Functions For each e, two weight functions w 1 (e): measuring gap and symmetry –w 1 (e)= 0 if corresponding trapezoid constructed from detected segments –w 1 (e)= length of gap-filling segment if constructed from detected and gap-filling –If e is dashed, w 1 (e)= total gap length + collinearity of the axis

17. 17/31 Auxiliary Edges Four auxiliary edges W 1 (e) = |P 1 P 12 | D + |P 6 P 7 | D

18. 18/31 Optimal Boundary w 2 (e)= signed area of the T or G corresponding to e, w 2 (e)=0 for auxiliary edges Note w 1 (e + )= w 1 (e - ) and w 2 (e + )= -w 2 (e - )>0 Mirror cycles: Cycle ratio:

19. 19/31 Optimal Boundary (Cont.) Proven that C does not contain more than one auxiliary edge Use an available graph algorithm to find an alternate cycle C with minimum cycle ratio e.g. the minimum-ratio-alternate-cycle algorithm introduced in a previous paper finds the optimal cycle in polynomial time

20. 20/31 Consider quadrilateral P 2 P 3 P 6 P 7 Contribution to | B | D =|P 2 P 3 | D +|P 6 P 7 | D ≠|P 2 P 3 |+|P 6 P 7 | (a) (b) Gap-Length Measure

21. 21/31Implementation n detected segments For every endpoint, consider K shortest gap-filling segments (K=5)  O(n) gap-filling segments Constructing trapezoids (a) Pair every two detected segments (b) Pair every detected with every gap filling  O(n 2 ) trapezoids Constructing quadrilaterals Pair every two trapezoids  O(n 4 ) ?!

22. 22/31Strategies 3 strategies to reduce this number (reduce the number of dashed edges in the constructed graph): (1) Consider the quadrilateral that has the shortest axis segment out of four choices, do not consider if the total gap length introduced is >D 1 (D 1 =30 pixels) (2) Avoid constructing a quadrilateral to connect two trapezoids that share a same portion of a detected segment (3) Avoid constructing quadrilaterals that lead to an axis with low collinearity or |sin (angles)| > D 2 (D 2 =0.5) To reduce the number of auxiliary edges: –Only consider the axis-segment end-points around which the gap length is less than a given threshold <D 3 (D 3 =20)

23. 23/31 Experiments- Synthetic Data A pair of synthetic boundaries (one desired symmetric, another non-symmetric) Introducing gaps along boundary and adding noise segments in image Comparing with RC (authors’ method without symmetry) and EZ (Elder & Zucker method ‘96)

24. 24/31 Performance evaluation Performance using a region coincidence measure with ground truth –R: region enclosed by the desired ground-truth boundary –R’: region enclosed by the detected boundary –Region coincidence measure: How good is this measure?

25. 25/31 Synthetic Data- Results

26. 26/31 Experiments- Real Images SRCRCEZ

27. 27/31 Other experiments Detecting multiple boundaries –Remove all the trapezoids along the detected boundary and then repeat –Problem: detecting the same boundary again especially when boundary has multiple symmetry axes Effects of changing –Related to image size Effects of changing other thresholds –No effect on results, just on running time Special cases: –Two disjoint closed boundaries that are symmetric –Two disjoint closed boundaries that form a ring

28. 28/31Extensions Use boundary continuity, or smoothness –Not so good when the actual boundary is NOT smooth Use region’s intensity homogeneity

29. 29/31 Running Time Worst-case time complexity: = O(|V| 3/4.|E|) = O(n 5.5 ) if n detected segments Actual running times, not so bad, because of introduced strategies

30. 30/31Summary A new grouping algorithm for detecting closed boundaries that show good bilateral symmetry Combining boundary and region information Combining local and global information

31. 31/31References J.S. Stahl and S. Wang, “Edge Grouping Combining Boundary and Region Information”, accepted for publication in IEEE Trans. on Image Processing, 2007. J.S. Stahl and S. Wang, “Globally Optimal Grouping for Symmetric Closed Boundaries by Combining Boundary and Region Information”, accepted for publication in IEEE Trans. on Pattern Analysis and Machine Intelligence, 2007.