1 Michael Bronstein Heat diffusion descriptors deformable Michael Bronstein Weizmann Institute of Science, 4 November 2010 Institute of Computational Science Universita della Svizzera Italiana Lugano, Switzerland shapesfor Heat diffusion descriptors

2 Michael Bronstein Heat diffusion descriptors Dan Raviv Technion Ron Kimmel Technion Maks Ovsjanikov Stanford Leo Guibas Stanford Iasonas Kokkinos ECP Paris Alex Bronstein TAU

3 Michael Bronstein Heat diffusion descriptors The next challenge TextVisual dataGeometric data

4 Michael Bronstein Heat diffusion descriptors Shape retrieval today

5 Michael Bronstein Heat diffusion descriptors Bags of words Notre Dame de Paris is a Gothic cathedral in the fourth quarter of Paris, France. It was the first Gothic architecture cathedral, and its construction spanned the Gothic period. construction architecture Italy France cathedral church basilica Paris Rome Gothic Roman St. Peter’s basilica is the largest church in world, located in Rome, Italy. As a work of architecture, it is regarded as the best building of its age in Italy. Notre Dame de Paris is a Gothic cathedral in the fourth quarter of Paris, France. It was the first Gothic architecture cathedral, and its construction spanned the Gothic period. St. Peter’s basilica is the largest church in world, located in Rome, Italy. As a work of architecture, it is regarded as the best building of its age in Italy.

6 Michael Bronstein Heat diffusion descriptors Outline Feature descriptor Geometric words Bag of geometric words Geometric expressions Spatially-sensitive bag of words “ ” “ ” Volumetric descriptors Scale invariance

7 Michael Bronstein Heat diffusion descriptors Curvature RigidBendingTopology Integral volume 1 Scale Spin image 2 Shape context 3 Representation HKS 4 SI-HKS 5 vHKS 6 1 Gelfand et al. 2005; 2 Johnson, Hebert 1999; 3 Belongie et al. 2002; 4 Sun et al Shape descriptors Any Volume/Mesh Any Volume/Mesh 5 B, Kokkinos 2010; 6 Raviv, BBK 2010

8 Michael Bronstein Heat diffusion descriptors Diffusion geometry Amount of heat transferred from point x to point y in time t Heat equation where - positive semidefinite Laplace-Beltrami operator - heat distribution Fundamental solution (heat kernel, ) – heat equation solution for initial conditions Spectral expression

9 Michael Bronstein Heat diffusion descriptors Sun, Ovsjanikov, Guibas, 2009 Heat kernel interpretation Geometric interpretation: “multiscale Gaussian curvature” Probabilistic interpretation: the probability of a random walk to remain at point x after time t.

10 Michael Bronstein Heat diffusion descriptors Sun, Ovsjanikov, Guibas, 2009 Heat kernel signature Multiscale descriptor Time (scale) ■Intrinsic, hence deformation-invariant ■Provably informative ■Efficiently computable on different shape representations ■Multiscale

11 Michael Bronstein Heat diffusion descriptors Ovsjanikov, BB, Guibas, 2009 BB. Ovsjanikov, Guibas 2010 Shape Geometric vocabulary Bag of geometric words

12 Michael Bronstein Heat diffusion descriptors Index in geometric vocabulary 164 Ovsjanikov, BB, Guibas, 2009 BB. Ovsjanikov, Guibas 2010 Bags of geometric words

13 Michael Bronstein Heat diffusion descriptors B et al SHREC 2010: Robust shape retrieval benchmark Transformation Query set Database (>1K shapes)

14 Michael Bronstein Heat diffusion descriptors B et al QueryToldo et al. 2009Shape

15 Michael Bronstein Heat diffusion descriptors Bags of words using HKS descriptor, vocabulary of size 48 Shape Bags of words using spin image descriptor Performance results Toldo et al B et al Performance criterion: mean average precision (mAP) in %

16 Michael Bronstein Heat diffusion descriptors Scale invariance Original shapeScaled by Not scale invariant!

17 Michael Bronstein Heat diffusion descriptors Scale-invariant HKS B, Kokkinos CVPR 2010 Log scale-space Scaling = shift and multiplicative constant log + d/d  Undo scaling Fourier transform magnitude Undo shift    =2k  /T

18 Michael Bronstein Heat diffusion descriptors Scale invariant HKS B, Kokkinos 2010 HKSSI-HKS

19 Michael Bronstein Heat diffusion descriptors B, Kokkinos 2010 QueryHKSSI-HKS

20 Michael Bronstein Heat diffusion descriptors B, Kokkinos 2010 HKS, vocabulary of size 48SI-HKS, vocabulary of size 48 HKS vs SI-HKS Performance criterion: mean average precision (mAP) in %

21 Michael Bronstein Heat diffusion descriptors Expressions In math science, matrix decomposition is a factorization of a matrix into some canonical form. Each type of decomposition is used in a particular problem. In biological science, decomposition is the process of organisms to break down into simpler form of matter. Usually, decomposition occurs after death. Matrix is a science fiction movie released in Matrix refers to a simulated reality created by machines in order to subdue the human population. matrix decomposition matrix factorization science fiction canonical form In math science, matrix decomposition is a factorization of a matrix into some canonical form. Each type of decomposition is used in a particular problem. In biological science, decomposition is the process of organisms to break down into simpler form of matter. Usually, decomposition occurs after death. Matrix is a science fiction movie released in Matrix refers to a simulated reality created by machines in order to subdue the human population. matrix decomposition is a the of in to by science form In math science, matrix decomposition is a factorization of a matrix into some canonical form. Each type of decomposition is used in a particular problem. Matrix is a science fiction movie released in Matrix refers to a simulated reality created by machines in order to subdue the human population. Ovsjanikov, BB & Guibas 2009

22 Michael Bronstein Heat diffusion descriptors Expressions In math science, matrix decomposition is a factorization of a matrix into some canonical form. Each type of decomposition is used in a particular problem. matrix decomposition is a the of in to by science form In particular matrix used type a some science, decomposition form a factorization of is canonical. matrix math decomposition is in a Each problem. into of matrix decomposition matrix factorization science fiction canonical form Ovsjanikov, BB & Guibas 2009

23 Michael Bronstein Heat diffusion descriptors Geometric expressions Ovsjanikov, BB & Guibas 2009 “Yellow Yellow”Yellow No total order between points (only “far” and “near”) Geometric expression = a pair of spatially close geometric words

24 Michael Bronstein Heat diffusion descriptors Ovsjanikov, BB & Guibas 2009 Spatially-sensitive bags of words

25 Michael Bronstein Heat diffusion descriptors B et al HKS, vocabulary of size 48Spatially-sensitive HKS, vocabulary of size 8x8 HKS vs SI-HKS Performance criterion: mean average precision (mAP) in %

26 Michael Bronstein Heat diffusion descriptors Is our shape model good? Raviv, BBK 2010 Boundary ∂X Interior X

27 Michael Bronstein Heat diffusion descriptors Is our shape model good? Camel illustration from Sumner et al. Raviv, BBK 2010 Volume isometryBoundary isometry Preserves geodesic distances on the boundary surface Preserves geodesic distances inside the volume

28 Michael Bronstein Heat diffusion descriptors where Diffusion equation Raviv, BBK 2010 Volumetric diffusionBoundary diffusion - Laplace-Beltrami operator - Euclidean Laplacian - normal to boundary surface

29 Michael Bronstein Heat diffusion descriptors where Heat kernels Raviv, BBK 2010 Volumetric heat kernelBoundary heat kernel Geometric interpretation “Multiscale Gaussian curvature”

30 Michael Bronstein Heat diffusion descriptors Heat kernel signatures Raviv, BBK 2010 vHKSHKS Boundary+volume isometry Boundary isometry

31 Michael Bronstein Heat diffusion descriptors Raviv, BBK 2010 vHKSHKS

32 Michael Bronstein Heat diffusion descriptors HKS, vocabulary of size 48vHKS, vocabulary of size 48 HKS vs vHKS Raviv, BBK 2010 Performance criterion: mean average precision (mAP) in %

33 Michael Bronstein Heat diffusion descriptors Summary Feature descriptor Geometric words Bag of geometric words Geometric expressions Spatially-sensitive bag of words “ ” “ ” Volumetric descriptors Scale invariance

34 Michael Bronstein Heat diffusion descriptors Thank you