1 Numerical geometry of non-rigid shapes Lecture I – Introduction Numerical geometry of shapes Lecture I – Introduction non-rigid Michael Bronstein

2 Numerical geometry of non-rigid shapes Lecture I – Introduction Welcome to non-rigid world

3 Numerical geometry of non-rigid shapes Lecture I – Introduction Non-rigid shapes everywhere Articulated shapes Volumetric medical data Computer graphics models Two-dimensional shapes

4 Numerical geometry of non-rigid shapes Lecture I – Introduction Auguste Rodin Non-rigid shapes in art

5 Numerical geometry of non-rigid shapes Lecture I – Introduction Rock Paper Scissors じゃんけんぽん Jan-ken-pon (Rock-paper-scissors )

6 Numerical geometry of non-rigid shapes Lecture I – Introduction Hands Rock Paper Scissors じゃんけんぽん

7 Numerical geometry of non-rigid shapes Lecture I – Introduction Invariant similarity SIMILARITY TRANSFORMATION

8 Numerical geometry of non-rigid shapes Lecture I – Introduction Deformation-invariant similarity Define a class of deformations Find properties of the shape which are invariant under the class of deformations and discriminative (uniquely describe the shape) Define a shape distance based on these properties

9 Numerical geometry of non-rigid shapes Lecture I – Introduction Rigid Elastic TopologicalInelastic Invariance

10 Numerical geometry of non-rigid shapes Lecture I – Introduction Invariant correspondence CORRESPONDENCE TRANSFORMATION

11 Numerical geometry of non-rigid shapes Lecture I – Introduction Analysis and synthesis Elephant image: courtesy M. Kilian and H. Pottmann SYNTHESISANALYSIS

12 Numerical geometry of non-rigid shapes Lecture I – Introduction Landscape “HORSE” Image processingGeometry processing Pattern recognition Computer vision Computer graphics 2D world3D world

13 Numerical geometry of non-rigid shapes Lecture I – Introduction In a nutshell Analysis and synthesis of non-rigid shapes Archetype problems: shape similarity and correspondence Metric geometry as a common denominator Tools from geometry, algebra, optimization, numerical analysis, statistics, and multidimensional data analysis Practical numerical methods Applications in computer vision, pattern recognition, computer graphics, and geometry processing

14 Numerical geometry of non-rigid shapes Lecture I – Introduction Additional reading Excerpts from the book On paper Online tosca.cs.technion.ac.il/book Problems Solutions Lecture slides Software Links Tutorials Data Springer, October 2008

15 Numerical geometry of non-rigid shapes Lecture I – Introduction Raffaello Santi, School of Athens, Vatican

16 Numerical geometry of non-rigid shapes Lecture I – Introduction Metric model Shape = metric space, where is a metric Shape similarity = similarity of metric spaces

17 Numerical geometry of non-rigid shapes Lecture I – Introduction Isometries Two metric spaces and are equivalent if there exists a distance-preserving map (isometry) satisfying Self-isometries of form an isometry group Such and are called isometric, denoted

18 Numerical geometry of non-rigid shapes Lecture I – Introduction Euclidean metric Shape is a subset of the Euclidean embedding space Restricted Euclidean metric for all

19 Numerical geometry of non-rigid shapes Lecture I – Introduction Euclidean isometries Isometry group in the Euclidean space consists of rigid motions Two shapes differing by a Euclidean isometry are congruent RotationTranslationReflection

20 Numerical geometry of non-rigid shapes Lecture I – Introduction Geodesic metric Given a path on, define its length The length can be induced by the Euclidean metric Geodesic (intrinsic) metric Geodesic = minimum-length path Technical condition: is a smooth submanifold of

21 Numerical geometry of non-rigid shapes Lecture I – Introduction Riemannian view Define a Euclidean tangent space at every point Define an inner product (Riemannian metric) on the tangent space Measure the length of a curve using the Riemannian metric Bernhard Riemann ( )

22 Numerical geometry of non-rigid shapes Lecture I – Introduction Nash embedding theorem John Forbes Nash Embedding theorem (1956): Any smooth Riemannian manifold can be realized as an embedded surface in Euclidean space of sufficiently high yet finite dimension Technical conditions: Manifold is For -dimensional manifold, embedding space dimension is Practically: intrinsic and extrinsic views are equivalent! Nash, 1956

23 Numerical geometry of non-rigid shapes Lecture I – Introduction Uniqueness of the embedding Nash theorem guarantees existence but not uniqueness of embedding Embedding is clearly defined up to a congruence (Euclidean isometry) IN OTHER WORDS: Do isometric yet incongruent shapes exist? Are there cases of non-trivial non-uniqueness? Riemannian manifold Embedded surface

24 Numerical geometry of non-rigid shapes Lecture I – Introduction Bending Shapes with incongruent isometries are called bendable Plane is the simplest example of a bendable surface Shapes that do not have incongruent isometries are called rigid Extrinsic geometry of a rigid shape is fully determined by the intrinsic one

25 Numerical geometry of non-rigid shapes Lecture I – Introduction Rigidity conjecture Leonhard Euler ( ) In practical applications shapes are represented as polyhedra (triangular meshes), so… If the faces of a polyhedron were made of metal plates and the polyhedron edges were replaced by hinges, the polyhedron would be rigid. Do non-rigid shapes really exist?

26 Numerical geometry of non-rigid shapes Lecture I – Introduction Rigidity conjecture timeline Euler’s Rigidity Conjecture: every polyhedron is rigid Cauchy: every convex polyhedron is rigid Connelly finally disproves Euler’s conjecture Cohn-Vossen: all surfaces with positive Gaussian curvature are rigid Gluck: almost all simply connected surfaces are rigid

27 Numerical geometry of non-rigid shapes Lecture I – Introduction Connelly sphere Isocahedron Rigid polyhedron Connelly sphere Non-rigid polyhedron Connelly, 1978

28 Numerical geometry of non-rigid shapes Lecture I – Introduction “Almost rigidity” Most of the shapes (especially, polyhedra) are rigid This may give the impression that the world is more rigid than non-rigid This is true if isometry is considered in the strict sense: if exists such that Many objects have some elasticity and therefore can bend almost isometrically No known results about “almost rigidity” of shapes

29 Numerical geometry of non-rigid shapes Lecture I – Introduction Rock-paper-scissors again INTRINSICALLY SIMILAR EXTRINSICALLY SIMILAR Invariant to inelastic deformations Invariant to rigid motions

30 Numerical geometry of non-rigid shapes Lecture I – Introduction Extrinsic vs. intrinsic similarity INTRINSIC SIMILARITY isometry w.r.t. geodesic metric EXTRINSIC SIMILARITY isometry w.r.t. Euclidean metric

31 Numerical geometry of non-rigid shapes Lecture I – Introduction Extrinsic vs. intrinsic similarity RIGID MOTION EXTRINSIC SIMILARITY = CONGRUENCE For rigid shapes, intrinsic similarity = extrinsic similarity (since all the isometries are congruences)

32 Numerical geometry of non-rigid shapes Lecture I – Introduction Extrinsic similarity Given two shapes and, find the degree of their incongruence Compare and as subsets of the Euclidean space Invariance to Euclidean isometry where Euclidean isometries = rotation, translation, (reflection): is a rotation matrix, is a translation vector

33 Numerical geometry of non-rigid shapes Lecture I – Introduction Given two shapes and, find the best rigid motion bringing as close as possible to : is some shape-to-shape distance Minimum = extrinsic dissimilarity of and Minimizer = best rigid alignment between and ICP is a family of algorithms differing in The choice of the shape-to-shape distance The choice of the numerical minimization algorithm Iterative closest point (ICP) algorithms

34 Numerical geometry of non-rigid shapes Lecture I – Introduction Shape-to-shape distance Hausdorff distance: distance between subsets of a metric space where, Non-symmetric version of Hausdorff distance where is closest-point correspondence

35 Numerical geometry of non-rigid shapes Lecture I – Introduction Iterative closest point algorithm Initialize Find the closest point correspondence Minimize the misalignment between corresponding points Update Iterate until convergence… Chen & Medioni, 1991; Besl & McKay, 1992

36 Numerical geometry of non-rigid shapes Lecture I – Introduction Iterative closest point algorithm Closest point correspondenceOptimal alignment

37 Numerical geometry of non-rigid shapes Lecture I – Introduction And now, intrinsic similarity… INTRINSIC SIMILARITYEXTRINSIC SIMILARITY Part of the same metric spaceTwo different metric spaces SOLUTION: Find a representation of and in a common metric space

38 Numerical geometry of non-rigid shapes Lecture I – Introduction Canonical forms Isometric embedding Elad & Kimmel, 2003

39 Numerical geometry of non-rigid shapes Lecture I – Introduction Canonical form distance Compute canonical forms EXTRINSIC SIMILARITY OF CANONICAL FORMS INTRINSIC SIMILARITY = INTRINSIC SIMILARITY Elad & Kimmel, 2003

40 Numerical geometry of non-rigid shapes Lecture I – Introduction Examples of canonical forms Elad & Kimmel, 2003

41 Numerical geometry of non-rigid shapes Lecture I – Introduction Expression-invariant face recognition Images: Leonid Larionov

42 Numerical geometry of non-rigid shapes Lecture I – Introduction Is geometry sensitive to expressions? x x’ y y’ Euclidean distances

43 Numerical geometry of non-rigid shapes Lecture I – Introduction Is geometry sensitive to expressions? x x’ y y’ Geodesic distances

44 Numerical geometry of non-rigid shapes Lecture I – Introduction Extrinsic vs. intrinsic Distance distortion distribution Extrinsic geometry sensitive to expressions Intrinsic geometry insensitive to expressions Bronstein, Bronstein & Kimmel, 2003

45 Numerical geometry of non-rigid shapes Lecture I – Introduction Isometric model of expressions Expressions are approximately inelastic deformations of the facial surface Identity = intrinsic geometry Expression = extrinsic geometry Bronstein, Bronstein & Kimmel, 2003

46 Numerical geometry of non-rigid shapes Lecture I – Introduction Canonical forms of faces Bronstein, Bronstein & Kimmel, 2005

47 Numerical geometry of non-rigid shapes Lecture I – Introduction Telling identical twins apart Extrinsic similarity Intrinsic similarity MichaelAlex Bronstein, Bronstein & Kimmel, 2005

48 Numerical geometry of non-rigid shapes Lecture I – Introduction Telling identical twins apart MichaelAlex

49 Numerical geometry of non-rigid shapes Lecture I – Introduction

50 Numerical geometry of non-rigid shapes Lecture I – Introduction Summary Shape = metric space Shape similarity = distance between metric spaces Invariance = isometry Definition of the metric determines the class of transformations to which the similarity is invariant Extrinsic similarity = congruence (Euclidean metric) computed using ICP Intrinsic similarity = congruence of canonical forms obtained by isometric embedding

51 Numerical geometry of non-rigid shapes Lecture I – Introduction References Metric geometry Burago, Burago, Ivanov, A course on metric geometry, AMS (2001) Rigidity S. E. Cohn-Vossen, Nonrigid closed surfaces, Annals of Math. (1929) R. Connelly, The rigidity of polyhedral surfaces, Math. Magazine (1979) Iterative closest point algorithms Y. Chen and G. Medioni, Object modeling by registration of multiple range images, Proc. Robotics and Automation (1991) P. J. Besl and N. D. McKay, A method for registration of 3D shapes, Trans. PAMI (1992)

52 Numerical geometry of non-rigid shapes Lecture I – Introduction References S. Rusinkiewicz and M. Levoy, Efficient variants of the ICP algorithm, Proc. 3D Digital Imaging and Modeling (2001) N. Gelfand, N. J. Mitra, L. Guibas, and H. Pottmann, Robust global registration, Proc. SGP (2005) H. Li and R. Hartley, The 3D-3D registration problem revisited, Proc. ICCV (2007) N. J. Mitra, N. Gelfand, H. Pottmann, and L. Guibas, Registration of point cloud data from a geometric optimization perspective, Proc. SGP (2004) Canonical forms A. Elad and R. Kimmel, On bending invariant signatures for surfaces, Trans. PAMI (2003)

53 Numerical geometry of non-rigid shapes Lecture I – Introduction References Face recognition A. M. Bronstein, M. M. Bronstein, R. Kimmel, Expression-invariant 3D face recognition, Proc. AVBPA (2003) A. M. Bronstein, M. M. Bronstein, R. Kimmel, Three-dimensional face recognition, IJCV (2005) A. M. Bronstein, M. M. Bronstein, R. Kimmel, Expression-invariant representation of faces, Trans. Image Processing (2007)