Numerical geometry of non-rigid shapes Non-rigid similarity Alexander Bronstein, Michael Bronstein, Ron Kimmel © 2007 All rights reserved. Web: tosca.technion.ac.il

Visualization of shape space Abstract space of deformable shapes (point = shape) A distance measuring intrinsic similarity of shapes Equivalence relation: if they are isometric Similar (small d) Dissimilar (large d) Visualization of shape space

Intrinsic similarity properties Non-negativity: Symmetry: Triangle inequality: Similarity: if then and are -isometric if and are -isometric, then iff is a metric on the quotient space Consistency to sampling: if is a finite -covering of , then Efficiency: can be efficiently approximated numerically A. M. Bronstein et al., PNAS, 2006

Canonical forms distance Embed and into a given common metric space by minimum-distortion embeddings and . Compare the canonical forms as rigid objects A. Elad, R. Kimmel, CVPR 2001

Canonical forms distance Canonical form is an approximate representation of intrinsic geometry (unavoidable embedding error) satisfies the metric axioms only approximately Approximately consistent to sampling Efficient computation using MDS A. Elad, R. Kimmel, CVPR 2001

Gromov-Hausdorff distance Include the embedding space into the optimization problem where and are isometric embeddings Satisfies the metric axioms with Consistent to sampling: if is an -covering of , then Computationally intractable M. Gromov, 1981

Gromov-Hausdorff distance If , then there exist and such that . distance preservation bijectivity

Gromov-Hausdorff distance Given two shapes measure how far they are from being isometric . distance preservation bijectivity

Gromov-Hausdorff distance Given two shapes measure how far they are from being isometric . distance preservation bijectivity

Gromov-Hausdorff distance Given two shapes measure how far they are from being isometric . distance preservation bijectivity

Gromov-Hausdorff distance Equivalent definition of Gromov-Hausdorff distance in terms of metric distortions (for compact surfaces): where:

Computing the Gromov-Hausdorff distance Mémoli & Sapiro (2005) Replace with a simpler expression Probabilistic bound on the error Combinatorial problem F. Mémoli, G. Sapiro, Foundations Comp. Math, 2005

Computing the Gromov-Hausdorff distance BBK (2006) Generalized MDS problem Continuous optimization Deterministic approximation (exact up to numerical accuracy / local convergence) A. M. Bronstein et al., PNAS, 2007

Gromov-Hausdorff distance via GMDS Sampling: , Optimization over images and Two coupled GMDS problems A. M. Bronstein et al., PNAS, 2007

Gromov-Hausdorff distance via GMDS (cont) Equivalent formulation as a constrained problem using an artificial variable A. M. Bronstein et al., PNAS, 2007

Gromov-Hausdorff vs. canonical forms Two stages: embedding and comparison Embedding error is a problem degrading accuracy Many points (~1000) are required for accurate comparison Computational core: MDS One stage: generalized embedding Embedding error is the measure of similarity Few points (~10) are required to compute accurate distortion Computational core: GMDS

Example: 3D objects BBK, SIAM J. Sci. Comp, 2006

Canonical forms distance Gromov-Hausdorff distance Example: 3D objects Canonical forms distance (MDS, 500 points) Gromov-Hausdorff distance (GMDS, 50 points) BBK, SIAM J. Sci. Comp, 2006

How to compare a centaur to a horse? Partial similarity How to compare a centaur to a horse? Example: Jacobs et al.

Horse is not similar to man Partial similarity Horse is similar to centaur Man is similar to centaur Horse is not similar to man Partial similarity is an intransitive relation Non-metric (no triangle inequality) Weaker than full similarity (shapes may be partially but not fully similar)

Human vision example Recognition of objects according to partial information Certain parts have more importance in recognition A significant part is usually sufficient to recognize the entire object ?

Recognition by parts Divide the shapes into meaningful parts and Compare each part separately using full similarity criterion Merge the partial similarities

What are the parts of a shoe? What is a part? Problem: how to divide the shapes into parts? What are the parts of a shoe? Solution: consider all parts Optimize over the sets and of all the possible parts of shapes and : Technically, and are -algebras

Partiality Problem: are all parts equally important? Just having common parts is insufficient, parts must be significant Solution: define partiality measuring how large the selected parts are w.r.t. entire shapes (larger parts = smaller partiality) Illustration: Herluf Bidstrup

Goal: find the largest most similar common part Partial similarity recipe Secret sauce ingredients Sets of all parts Full similarity criterion (e.g. Gromov-Hausdorff distance) Partiality e.g. where are the measures of area Goal: find the largest most similar common part A. M. Bronstein et al., SSVM, 2007

Multicriterion optimization Minimize the vector objective function over Competing criteria – impossible to minimize and simultaneously ATTAINABLE CRITERIA UTOPIA A. M. Bronstein et al., SSVM, 2007

Minimum of scalar function Scalar versus vector optimality Minimum of scalar function Pareto optimum Pareto optimum: a point at which no criterion can be improved without compromising the other V. Pareto, 1901

Pareto distance Pareto distance: set of all Pareto optima (Pareto frontier), acting as a set-valued criterion of partial dissimilarity Only partial order relation exists between set-valued distances: not always possible to compare Infinite possibilities to convert Pareto distance into a scalar-valued one One possibility: select a point on the Pareto frontier closest to the utopia point, A. M. Bronstein et al., SSVM, 2007

Scalar- versus set-valued distances Large Gromov-Hausdorff distance Small partial dissimilarity Large Gromov-Hausdorff distance Large partial dissimilarity A. M. Bronstein et al., SSVM, 2007

Fuzzy approximation Problem: Optimization over subsets is an NP-hard problem ( possible parts) Solution: fuzzy approximation A part can be represented by the binary function Relax the problem: define membership function, which can obtain continuous values, A. M. Bronstein et al., SSVM, 2007

Fuzzy approximation Crisp part Fuzzy part A. M. Bronstein et al., SSVM, 2007

Fuzzy approximation Discrete membership functions Discrete measures Fuzzy partiality Fuzzy Gromov-Hausdorff distance A. M. Bronstein et al., SSVM, 2007

Alternating minimization Alternating minimization over and Fix , optimize over Fix , optimize over

Example: mythological creatures A. M. Bronstein et al., IJCV

Gromov-Hausdorff distance Partial dissimilarity Example: mythological creatures Gromov-Hausdorff distance Partial dissimilarity A. M. Bronstein et al., SSVM, 2007

Conclusions so far Axiomatic construction of isometry-invariant distances on the space of non-rigid shapes Gromov-Hausdorff computation using GMDS Pareto formalism for partial similarity of shapes