1. Structure Recovery by Part Assembly Chao-Hui Shen, Hongbo Fu, Kang Chen and Shi-Min Hu Tsinghua University City University of Hong Kong Presented by: Chenyang Zhu

2. Outline Background Overview Method Conclusion

3. Background Consumer level scanning devices Capture both RGB and depth Reconstruction is challenging ◦ Low resolution ◦ Noise ◦ Missing data ◦…◦…

4. Example-based Scan Completion Global-to-local and top-down [Kraevoy and Sheffer 2005; Pauly et al. 2005] Rely on the availability of suitable template model However … No suitable model! shape retrieval

5. Solution Recover the Structure by Part Assembly ◦ Structure recovery instead of geometry reconstruction ◦ Do NOT prepare a large database ◦ Retrieve and assemble suitable parts on the fly

6. Problem setup Input Point cloud + Image (Single view) Pre-segmented Repository Models (Parts + Labels) … Output ……

7. Algorithm Overview Candidate Parts SelectionStructure CompositionPart Conjoining …

8. Candidate Parts Employ 3D repository model as a global context ◦ Globally align the models with the input scan first Search in a 3D offset window around the part

9. Geometric contribution score Candidate Parts Employ 3D repository model as a global context ◦ Globally align the models with the input scan first Search in a 3D offset window around the part Geometric fidelity score 3D2D edge map 2D 3D

10. Candidate Parts Employ 3D repository model as a global context ◦ Globally align the models with the input scan first Select top K parts with highest score for each category Seat Back Arm Front leg … …… ………………

11. Structure Composition Search for promising compositions under constraints … Optimal composition average geometry fidelity of parts total geometry fidelity total geometry contribution Globally Evaluate the compositions

12. Part Conjoining Problem: the parts are loosely placed together Goal: generate a consistent & complete model

13. Part Conjoining identity scale i j transformed contact points

14. Conclusion A bottom-up structure recovery approach ◦ Effectively reuse limited repository models ◦ Automatically compose new structure ◦ Handle single-view inputs by the Kinect system Future work ◦ Multi-view inputs ◦ Include style/functional constraints ◦ Recover Indoor scenes

15. THANK YOU! Q&A