1 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Rock, paper, and scissors Joint extrinsic and intrinsic similarity of non-rigid shapes Alex Bronstein, Michael Bronstein, Ron Kimmel Department of Computer Science Technion – Israel Institute of Technology

2 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Extrinsic vs intrinsic similarity Intrinsic similarity Are the shapes congruent? Do the shapes have the same metric structure? Extrinsic similarity Rock, paper, and scissors: is the hand similar to a rock? Is it similar to another posture of a hand? The answer depends on the definition of similarity.

3 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Extrinsic similarity Can be expressed as a distance between two shapes and Find a rigid motion bringing the shapes into best alignment Misalignment is quantified using the Hausdorff distance or some of its variants Computed using ICP algorithms

4 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Extrinsic similarity – limitations Extrinsically similarExtrinsically dissimilar Suitable for nearly rigid shapes Unsuitable for non-rigid shapes

5 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Intrinsic similarity Compare the intrinsic geometries of two shapes Intrinsic geometry is expressed in terms of geodesic distances Geodesic distances are computed using Dijkstra’s shortest path algorithm or fast marching Euclidean distance Geodesic distance

6 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Intrinsic similarity – canonical forms Embed intrinsic geometries of and into a common metric space Minimum-distortion embeddings and computed using multidimensional scaling (MDS) algorithms Compare the images and as rigid shapes A. Elad, R. Kimmel, CVPR 2001

7 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Intrinsic similarity – GMDS Find the minimum distortion embedding of one shape into the other The minimum distortion is the measure of intrinsic dissimilarity Computed using the generalized MDS BBK, PNAS 2006

8 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Intrinsic similarity – limitations Intrinsically dissimilar Intrinsically similar Suitable for near-isometric shape deformations Unsuitable for deformations modifying shape topology

9 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Extrinsically dissimilar Intrinsically similar Extrinsically similar Intrinsically dissimilar Extrinsically dissimilar Intrinsically dissimilar THIS IS THE SAME SHAPE! Desired result:

10 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Joint extrinsic and intrinsic similarity Combine intrinsic and extrinsic similarities into a single criterion Find a deformation of whose intrinsic geometry is similar to and extrinsic geometry is more similar to defines the relative importance of intrinsic and extrinsic criteria is a collection of optimal tradeoffs between intrinsic and extrinsic criteria Can be formalized using the notion of Pareto optimality

11 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Intrinsic similarity Extrinsic similarity

12 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Computation of joint similarity Hybridization of ICP and GMDS in L 2 formulation for robustness Fix correspondence between and for intrinsic similarity where is precomputed and are computed at each iteration Closest-point distance for extrinsic similarity where are the closest points to in More details in the paper

13 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Numerical example – dataset = topology change Data: tosca.cs.technion.ac.il

14 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Numerical example – tradeoff curves Dissimilar Similar

15 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Numerical example – intrinsic similarity = topology-preserving no topology changes

16 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Numerical example – intrinsic similarity = topology change= topology-preserving

17 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Numerical example – extrinsic similarity = topology change= topology-preserving

18 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Numerical example – joint similarity = topology change= topology-preserving

19 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Numerical example – ROC curves

20 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Numerical example – shape morphing Stronger intrinsic similarity (smaller λ) Stronger extrinsic similarity (larger λ)

21 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Conclusion Extrinsic similarity is insensitive to topology changes, but sensitive to non-rigid deformations Intrinsic similarity is insensitive to nearly-isometric non-rigid deformations, but sensitive to topology changes Joint similarity is insensitive to both non-rigid deformations and topology changes Can be used to produce near-isometric morphs

22 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes References A. M. Bronstein, M. M. Bronstein, A. M. Bruckstein, R. Kimmel, Analysis of two-dimensional non-rigid shapes, IJCV, to appear. A. M. Bronstein, M. M. Bronstein, R. Kimmel, Rock, Paper, and Scissors: extrinsic vs. intrinsic similarity of non-rigid shapes, Proc. ICCV, (2007). I. Eckstein, J. P. Pons, Y. Tong, C. C. J. Kuo, and M. Desbrun, Generalized surface flows for mesh processing, Proc. SGP, (2007). M. Kilian, N. J. Mitra, and H. Pottmann, Geometric modeling in shape space, Proc. SIGGRAPH, vol. 26, (2007). A. M. Bronstein, M. M. Bronstein, A. M. Bruckstein, R. Kimmel, Paretian similarity for partial comparison of non-rigid objects, Proc. SSVM, pp , A. M. Bronstein, M. M. Bronstein, R. Kimmel, Calculus of non-rigid surfaces for geometry and texture manipulation, IEEE TVCG, Vol. 13/5, pp , (2007). A. M. Bronstein, M. M. Bronstein, R. Kimmel, Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching, PNAS, Vol. 103/5, pp , (2006).

23 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes References F. Mémoli and G. Sapiro, A theoretical and computational framework for isometry invariant recognition of point cloud data, Foundations of Computational Mathematics 5 (2005), N. J. Mitra, N. Gelfand, H. Pottmann, and L. Guibas, Registration of point cloud data from a geometric optimization perspective, Proc. SGP, (2004), pp A. Elad, R. Kimmel, On bending invariant signatures for surfaces, Trans. PAMI 25 (2003), no. 10, P. J. Besl and N. D. McKay, A method for registration of 3D shapes, Trans. PAMI 14 (1992), Y. Chen and G. Medioni, Object modeling by registration of multiple range images, Proc. Conf. Robotics and Automation, (1991). E. L. Schwartz, A. Shaw, and E. Wolfson, A numerical solution to the generalized mapmaker's problem: flattening nonconvex polyhedral surfaces, Trans. PAMI 11 (1989),

24 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Shameless advertisement COMING SOON… Published by Springer Verlag To appear in early 2008 Approximately 320 pages Over 50 illustrations Color figures Book website tosca.cs.technion.ac.il