1. Computer Vision Group University of California Berkeley 1 Learning Scale-Invariant Contour Completion Xiaofeng Ren, Charless Fowlkes and Jitendra Malik

2. Computer Vision Group University of California Berkeley 2 Abstract We present a model of curvilinear grouping using piecewise linear representations of contours and a conditional random field to capture continuity and the frequency of different junction types. Potential completions are generated by building a constrained Delaunay triangulation (CDT) over the set of contours found by a local edge detector. Maximum likelihood parameters for the model are learned from human labeled groundtruth. Using held out test data, we measure how the model, by incorporating continuity structure, improves boundary detection over the local edge detector. We also compare performance with a baseline local classifier that operates on pairs of edgels. Both algorithms consistently dominate the low-level boundary detector at all thresholds. To our knowledge, this is the first time that curvilinear continuity has been shown quantitatively useful for a large variety of natural images. Better boundary detection has immediate application in the problem of object detection and recognition.

3. Computer Vision Group University of California Berkeley 3 Boundary Detection Edge detection: 20 years after Canny Pb (Probability of Boundary): learning to combine brightness, color and texture contrasts There is psychophysical evidence that we might have been approaching the limit of local edge detection

4. Computer Vision Group University of California Berkeley 4 Curvilinear Continuity Boundaries are smooth in nature A number of associated phenomena –Good continuation –Visual completion –Illusory contours Well studied in human vision –Wertheimer, Kanizsa, von der Heydt, Kellman, Field, Geisler, … Extensively explored in computer vision –Shashua, Zucker, Mumford, Williams, Jacobs, Elder, Jermyn, Wang, … Is the net effect of completion positive? Or negative? Lack of quantitative evaluation Boundaries are smooth in nature A number of associated phenomena –Good continuation –Visual completion –Illusory contours Well studied in human vision –Wertheimer, Kanizsa, von der Heydt, Kellman, Field, Geisler, … Extensively explored in computer vision –Shashua, Zucker, Mumford, Williams, Jacobs, Elder, Jermyn, Wang, … Is the net effect of completion positive? Or negative? Lack of quantitative evaluation

5. Computer Vision Group University of California Berkeley 5 Scale Invariance Sources of scale invariance arbitrary viewing distancehierarchy of parts Power laws in natural images –Lots of findings, e.g. in power spectra or wavelet coefficients (Ruderman, Mumford, Simoncelli, …) –Also in boundary contours [Ren and Malik 02] How to incorporate scale-invariance?

6. Computer Vision Group University of California Berkeley 6 A Scale-Invariant Representation Piecewise linear approximation of low-level contours –recursive splitting based on angle Constrained Delaunay Triangulation –a variant of the standard Delaunay Triangulation –maximizes the minimum angle (avoids skinny triangles)

7. Computer Vision Group University of California Berkeley 7 The CDT Graph scale-invariant fast to compute <1000 edges completes gaps little loss of structure

8. Computer Vision Group University of California Berkeley 8 No Loss of Structure Use P human the soft groundtruth label defined on CDT graphs: precision close to 100% Pb averaged over CDT edges: no worse than the orignal Pb Increase in asymptotic recall rate: completion of gradientless contours

9. Computer Vision Group University of California Berkeley 9 CDT vs. K-Neighbor Completion An alternative scheme for completion: connect to k- nearest neighbor vertices, subject to visibility CDT achieves higher asymptotic recall rates

10. Computer Vision Group University of California Berkeley 10 Inference on the CDT Graph Xe Local inference: Xe Global inference:

11. Computer Vision Group University of California Berkeley 11 Baseline Local Model “Bi-gram” model:  contrast + continuity  binary classification (0,0) vs (1,1) logistic classifier “Tri-gram” model: 11 22 LL  Pb L = Xe

12. Computer Vision Group University of California Berkeley 12 Global Model w/ Conditional Random Fields Graphical model with expoential potential functions edge potentials exp(  i ) junction potentials exp(  j ) Inference with loopy belief propagation Maximum likelihood learning (convex) with gradient descent converges < 10 iterations

13. Computer Vision Group University of California Berkeley 13 Junctions and Continuity Junction types (deg g,deg c ): deg g =1,deg c =0deg g =0,deg c =2deg g =1,deg c =2 Continuity term for degree-2 junctions deg g +deg c =2  deg g =0,deg c =0

14. Computer Vision Group University of California Berkeley 14 Continuity improves boundary detection in both low-recall and high-recall ranges Global inference helps; mostly in low-recall/high-precision Roughly speaking, CRF>Local>CDT only>Pb

15. Computer Vision Group University of California Berkeley 15

16. Computer Vision Group University of California Berkeley 16

17. Computer Vision Group University of California Berkeley 17 ImagePbLocalGlobal

18. Computer Vision Group University of California Berkeley 18 ImagePbLocalGlobal

19. Computer Vision Group University of California Berkeley 19 ImagePbLocalGlobal

20. Computer Vision Group University of California Berkeley 20 Conclusion Constrained Delaunay Triangulation is a scale-invariant discretization of images with little loss of structure; Moving from 100,000 pixels to <1000 edges, CDT achieves great statistical and computational efficiency; Curvilinear Continuity improves boundary detection; –the local model of continuity is simple yet very effective –global inference of continuity further improves performance –Conditional Random Fields w/ loopy belief propagation works well on CDT graphs Curvilinear Continuity improves boundary detection; –the local model of continuity is simple yet very effective –global inference of continuity further improves performance –Conditional Random Fields w/ loopy belief propagation works well on CDT graphs Mid-level vision is useful.

21. Computer Vision Group University of California Berkeley 21 Thank You

22. Computer Vision Group University of California Berkeley 22