Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding GrUVi Lab, School of Computing Science Simon Fraser University, Burnaby, BC Canada Varun Jain and Hao Zhang

Problem Overview

Shape Retrieval Applications Computer aided design Game design Shape recognition Face recognition Database Output User Interface Outline Problem Overview Retrieval Problem Methods Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements … Query …

Shape Retrieval How? Using Correspondence Efficiency?? Using Global Descriptors For 3D shapes Fourier Descriptors Light Field Spherical Harmonics Skeletal Graph Matching Outline Problem Overview Retrieval Problem Methods Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements Non-rigid transformations?? StretchingStretching Articulation (bending)Articulation (bending)

Shape Retrieval Our Method Normalize non-rigid transformations Construct affinity matrix Spectral embedding Use global shape descriptors Light Field Descriptor (LFD) Spherical Harmonics Descriptor (SHD) Eigenvalues?? Outline Problem Overview Retrieval Problem Methods Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

Shape Retrieval Advantages of Our Method Handles shape articulation (best performance for articulated shapes). Flexibility of affinity matrices Robustness of affinity matrices Outline Problem Overview Retrieval Problem Methods Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

Spectral Embeddings

Affinity matrix Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements > > : 4 ¢¢¢ n A n £ n = n 8 > > > > > > > > < > > > > > > z}|{ a 11 a 12 ¢¢¢ a 1 n a 21 ¢¢¢ a i 1 a i 2 ¢¢¢ a i n a n 1 a n 2 a nn a ij = e geo d ( i ; j ) 2 2 ¾ 2

Spectral Embeddings Eigenvalue decomposition: Scaled eigenvectors: Jain, V., Zhang, H.: Robust 3D Shape Correspondence in the Spectral Domain. Proc. Shape Modeling International Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements A = E¤E T k-dimensional spectral embedding coordinates of i th point of P

Spectral Embeddings Examples of 3D embeddings Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ U 3 = n 8 > > > > > > > > > > < > > > > > > > > > > : ~ u 2 ~ u 3 ~ u u i 2 u i 3 u i

EigenValue Descriptor (EVD): Use deviation in projected data as descriptor: Our shape descriptor: ¾ i = ³ k u i k 2 n ´ 1 2 = p ¸ i r P i ¸ i Spectral Embeddings Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements f ¾ 1 ; ¾ 2 ;:::; ¾ k g

Why use eigenvalues?? Spectral Embeddings Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements A = a 11 a 12 ¢¢¢ a 1 n a 21 ¢¢¢ a i 1 a i 2 ¢¢¢ a i n a n 1 a n 2 ¢¢¢ a nn B = b 11 b 12 ¢¢¢ b 1 n b 21 ¢¢¢ b i 1 b i 2 ¢¢¢ b i n b n 1 b n 2 ¢¢¢ b nn R = AA T RE = E¤ R = A 2

Spectral Embeddings Problems: Geodesic distance computation Efficiency of geodesic distance computation & eigendecomposition: Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements O ( n 2 l ogn ) + O ( k n 2 )

Spectral Embeddings Geodesics using Structural Graph: Add edges to make mesh connected Geodesic distance ≈ Shortest graph distance Problem: Unwanted (topology modifying) edges! Solution: Add shortest possible edges. Choice of graph to take edges from: p-nearest neighbor (may not return connected graph) p-edge connected [Yang 2004] Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements

Spectral Embeddings Efficiency with Nyström approximation Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements E l £ l = ~ u 1 ¢¢¢ ~ u l ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ E n £ 3 = n8> > > > > > < > > > > > > : ~ u 2 ~ u 3 ~ u ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯¯¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ex t rapola t i on O ( l n ) max-min su b samp l i ng O ( l n l ogn ) A l £ l = l 8 > > < > > : n z}|{ a 11 a 12 ¢¢¢ a 1 n a 21 a 22 ¢¢¢ a 2 n a l 1 a l 2 ¢¢¢ a l n eigen d ecompos i t i on O ( l 3 )

Shape Descriptor

Global Shape Descriptors Light Field Descriptor (LFD) Spherical Harmonics Descriptor (SHD) Our Similarity Measure (EVD): Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements S i m C os t ( P ; Q ) = 1 2 k X i = 1 £ p ¸ i ¡ p ° i ¤ 2 p ¸ i + p ° i

Results

Experimental Database McGill 3D Articulated Shapes Database Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

Results Precision-Recall plot for McGill database Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

Results McGill articulated shape database Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

Non-robustness of geodesic distances Non-robustness to outliers Limitations & Future Work Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

McGill 3D Shape Benchmark. Phil Shilane (LFD & SHD implementations). Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

Thank You! for you attention