1. Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding GrUVi Lab, School of Computing Science Simon Fraser University, Burnaby, BC Canada Varun Jain and Hao Zhang {vjain,haoz}@cs.sfu.ca

2. Problem Overview

3. Shape Retrieval Applications Computer aided design Game design Shape recognition Face recognition Database Output User Interface Outline Problem Overview Retrieval Problem Methods Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements … Query …

4. Shape Retrieval How? Using Correspondence Efficiency?? Using Global Descriptors For 3D shapes Fourier Descriptors Light Field Spherical Harmonics Skeletal Graph Matching Outline Problem Overview Retrieval Problem Methods Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements Non-rigid transformations?? StretchingStretching Articulation (bending)Articulation (bending)

5. Shape Retrieval Our Method Normalize non-rigid transformations Construct affinity matrix Spectral embedding Use global shape descriptors Light Field Descriptor (LFD) Spherical Harmonics Descriptor (SHD) Eigenvalues?? Outline Problem Overview Retrieval Problem Methods Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

6. Shape Retrieval Advantages of Our Method Handles shape articulation (best performance for articulated shapes). Flexibility of affinity matrices Robustness of affinity matrices Outline Problem Overview Retrieval Problem Methods Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

7. Spectral Embeddings

8. Affinity matrix Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements > > : 4 ¢¢¢ n A n £ n = n 8 > > > > > > > > < > > > > > > 2 6 6 6 6 6 6 6 6 z}|{ a 11 a 12 ¢¢¢ a 1 n a 21 ¢¢¢......... a i 1 a i 2 ¢¢¢ a i n......... a n 1 a n 2 a nn 3 7 7 7 7 7 7 7 7 5 a ij = e geo d ( i ; j ) 2 2 ¾ 2

9. Spectral Embeddings Eigenvalue decomposition: Scaled eigenvectors: Jain, V., Zhang, H.: Robust 3D Shape Correspondence in the Spectral Domain. Proc. Shape Modeling International 2006. Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements A = E¤E T k-dimensional spectral embedding coordinates of i th point of P

10. Spectral Embeddings Examples of 3D embeddings Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ U 3 = n 8 > > > > > > > > > > < > > > > > > > > > > : 2 6 6 6 6 6 6 6 6 6 6 4 ~ u 2 ~ u 3 ~ u 4......... u i 2 u i 3 u i 4......... 3 7 7 7 7 7 7 7 7 7 7 5

11. EigenValue Descriptor (EVD): Use deviation in projected data as descriptor: Our shape descriptor: ¾ i = ³ k u i k 2 n ´ 1 2 = p ¸ i r P i ¸ i Spectral Embeddings Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements f ¾ 1 ; ¾ 2 ;:::; ¾ k g

12. Why use eigenvalues?? Spectral Embeddings Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements A = 2 6 6 6 6 6 6 6 6 4 a 11 a 12 ¢¢¢ a 1 n a 21 ¢¢¢......... a i 1 a i 2 ¢¢¢ a i n......... a n 1 a n 2 ¢¢¢ a nn 3 7 7 7 7 7 7 7 7 5 B =26 6 6 6 6 6 6 6 4 b 11 b 12 ¢¢¢ b 1 n b 21 ¢¢¢......... b i 1 b i 2 ¢¢¢ b i n......... b n 1 b n 2 ¢¢¢ b nn37 7 7 7 7 7 7 7 5 R = AA T RE = E¤ R = A 2

13. Spectral Embeddings Problems: Geodesic distance computation Efficiency of geodesic distance computation & eigendecomposition: Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements O ( n 2 l ogn ) + O ( k n 2 )

14. Spectral Embeddings Geodesics using Structural Graph: Add edges to make mesh connected Geodesic distance ≈ Shortest graph distance Problem: Unwanted (topology modifying) edges! Solution: Add shortest possible edges. Choice of graph to take edges from: p-nearest neighbor (may not return connected graph) p-edge connected [Yang 2004] Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements

15. Spectral Embeddings Efficiency with Nyström approximation Outline Problem Overview Spectral Embeddings Basics Problems Solutions Shape Descriptors Results Future Work Acknowledgements E l £ l =26 6 4...... ~ u 1 ¢¢¢ ~ u l......37 7 5 ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ E n £ 3 = n8> > > > > > < > > > > > > : 2 6 6 6 6 6 6 4......... ~ u 2 ~ u 3 ~ u 4......... 3 7 7 7 7 7 7 5 ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯¯¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ex t rapola t i on O ( l n ) max-min su b samp l i ng O ( l n l ogn ) A l £ l = l 8 > > < > > : 2 6 6 4 n z}|{ a 11 a 12 ¢¢¢ a 1 n a 21 a 22 ¢¢¢ a 2 n......... a l 1 a l 2 ¢¢¢ a l n 3 7 7 5 eigen d ecompos i t i on O ( l 3 )

16. Shape Descriptor

17. Global Shape Descriptors Light Field Descriptor (LFD) Spherical Harmonics Descriptor (SHD) Our Similarity Measure (EVD): Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements S i m C os t ( P ; Q ) = 1 2 k X i = 1 £ p ¸ i ¡ p ° i ¤ 2 p ¸ i + p ° i

18. Results

19. Experimental Database McGill 3D Articulated Shapes Database http://www.cim.mcgill.ca/~shape/benchMark/ Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

20. Results Precision-Recall plot for McGill database Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

21. Results McGill articulated shape database Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

22. Non-robustness of geodesic distances Non-robustness to outliers Limitations & Future Work Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

23. McGill 3D Shape Benchmark. Phil Shilane (LFD & SHD implementations). Outline Problem Overview Spectral Embeddings Shape Descriptors Results Future Work Acknowledgements

24. Thank You! for you attention