1. Key Laboratory of Machine Perception (MOE), Peking University Applications of 3D Digital Technologies in Archaeology and Digital Heritage Hongbin Zha ( 査紅彬） 3D Visual Computing and Robotics Group Key Lab on Machine Perception (MOE) Peking University, China

2. Key Laboratory of Machine Perception (MOE), Peking University Outline u Introduction u Task definition of 3D digitization u Applications u Process pipeline and challenges u Applications in Digital Heritage u Digital Longmen Grottos （竜門石窟） Project u Computer-assisted line drawing u Digital restoration u Maijishan Grottos （麦積山石窟） Project and Others

3. Key Laboratory of Machine Perception (MOE), Peking University 3D Modeling of Real Objects mini-statue meshrendered mesh  Copying machine  3D digital copy

4. Key Laboratory of Machine Perception (MOE), Peking University Objectives of 3D Digitization u Geometry Digitization: u Cover the whole surfaces of an object u Describe accurate shape and structure details u Digitization of Other Attributes: u Textures and colors u Transparency and glossiness u ……

5. Key Laboratory of Machine Perception (MOE), Peking University Characteristics of 3D Modeling Technologies u Objects: u With complex geometrical and topological structures, with dynamic deformation or movements -- architectures, human bodies or outdoor scenes. u Geometry Processing: u Model simplification, data compression and transmission for large-scale scenes. u Automatic Modeling: u 3D laser scanning, hybrid imaging systems, data acquisition using mobile platforms.

6. Key Laboratory of Machine Perception (MOE), Peking University Break down Wall between Real and Cyber Worlds

7. Key Laboratory of Machine Perception (MOE), Peking University u Emerging technology of computer vision, geometrical computation and computer graphics. u Integrated pipeline of processes: data acquisition, processing to model manipulation. u Wide range of applications: 3D surveying, digital city, digital heritage, virtual shops, digital human, medical and forensic applications. A New Field: Modeling Based on 3D Scanning Technology

8. Key Laboratory of Machine Perception (MOE), Peking University Digital Museum and Digital Heritage Digitization for museum and heritage objects Digitization for museum and heritage objects Interactive walkthrough of heritage sites Interactive walkthrough of heritage sites ̶ Digital recording and documentation ̶ Digital exhibition ̶ Digital reconstruction, measurement and analysis ̶ Remote visiting and education

9. Key Laboratory of Machine Perception (MOE), Peking University u Digital recording and duplication of heritage articles u Digital restoration of heritage relics u Behavior analysis of life forms based on skeleton fossils u Digital recording of archaeological excavation sites u Virtual reconstruction of historic sceneries 3D Digitization Assisted Archaeology and Heritage

10. Key Laboratory of Machine Perception (MOE), Peking University NRC Tylosaurus sp.Mosasaur fossils Stanford David IBM Piet à Tokyo Buddha Heritage Related Work

11. Key Laboratory of Machine Perception (MOE), Peking University Pipeline for 3D Imaging and Modeling

12. Key Laboratory of Machine Perception (MOE), Peking University 3D Scanners

13. Key Laboratory of Machine Perception (MOE), Peking University Terrestrial Fixed- Pose Laser Scanners Callidus GS 100 ILRIS-3D Cyrax2500 LMS-Z420

14. Key Laboratory of Machine Perception (MOE), Peking University 3D Photography or 3D TV uDevelop devices that can capture 3D images or videos as general cameras uimage-based modeling ucamera arrays or domes uhybrid imaging systems

15. Key Laboratory of Machine Perception (MOE), Peking University Complexity in 3D shapes and topological structures Complexity in 3D shapes and topological structures Discontinuities in data parameterization and irregularities in sampling Well-established digital processing tools (FFT, Wavelet analysis etc.) can not be taken into good use. Discontinuities in data parameterization and irregularities in sampling Well-established digital processing tools (FFT, Wavelet analysis etc.) can not be taken into good use. Difficulties in 3D Digital Geometry Processing sound image video

16. Key Laboratory of Machine Perception (MOE), Peking University Big Sizes of 3D Models 3D models become larger and larger: 3D models become larger and larger: High requirements for geometrical and other attribute details High requirements for geometrical and other attribute details Complexities of objects to be modeled Complexities of objects to be modeled Advance in 3D data acquisition technologies Advance in 3D data acquisition technologies

17. Key Laboratory of Machine Perception (MOE), Peking University u Acquisition of 3D geometry and attribute data u Registration of multi-view data u Gaze-guided LOD control u Texture mapping for creating photo-realistic scenes u Shape-based model retrieval u Craniofacial reconstruction using 3D models u Video-driven face animation u 3D Modeling of large scenes u Applications of 3D digital technology in digital heritage, medicine, and digital city Our Related Work

18. Key Laboratory of Machine Perception (MOE), Peking University Outline u Introduction u Task definition of 3D digitization u Applications u Process pipeline and challenges u Applications in Digital Heritage u Digital Longmen Grottos （竜門石窟） Project u Computer-assisted line drawing u Digital restoration u Maijishan Grottos （麦積山石窟） Project and Others

19. Key Laboratory of Machine Perception (MOE), Peking University Digital Heritage 19 Display Fundamental Research An input imageReconstruction result Modeling targets with complex geometry Computation on 3D models …… Transformation Digitization Projects Virtual Tourism of Maiji Grottoes (Collaborated with Toppan) Surveying and Mapping Caves (Collaborated with Longmen Grottoes) …… Data Acquisition Laser Scanners Laser Scanner GPS IMU Video Camera Laser Scanner Geometric reconstruction & Texture generation

20. Key Laboratory of Machine Perception (MOE), Peking University Projects Beijing XiangTang Shan Grottoes LongMen Grottoes MaiJiShan Grottoes Five-Layer Tower ChongQing DiaoYu Town YangCan Tomb NanYueWang Museum

21. Key Laboratory of Machine Perception (MOE), Peking University Joint Projects with Longmen Grottoes Computer assisted line drawing for archaeological research Computer assisted line drawing for archaeological research Virtual restoration of heritage sites and relics Virtual restoration of heritage sites and relics Digitization standards for grotto scanning and modeling Digitization standards for grotto scanning and modeling “Digital Longmen Grottoes” project “Digital Longmen Grottoes” project

22. Key Laboratory of Machine Perception (MOE), Peking University

23. Surveying and Mapping in Archaeology Research The purpose of surveying and mapping is to measure and record accurate, readable and parallel-projection feature curves of cultural heritage. Archeological reports: Photos: record current situations and information to infer knowledge such as its producing date Photos: record current situations and information to infer knowledge such as its producing date Texts: describe all explicit known information Texts: describe all explicit known information line drawings: reflect accurate, vivid and feature emphasized figures line drawings: reflect accurate, vivid and feature emphasized figures

24. Key Laboratory of Machine Perception (MOE), Peking University Line Drawing in Archeology Traditional methods: Traditional methods: ̶ Set baseline and grid, then measure using rulers ̶ Disadvantages: Low accuracy Low accuracy Time-consuming Time-consuming Difficult to draw lines on multiple sections and large areas Difficult to draw lines on multiple sections and large areas

25. Key Laboratory of Machine Perception (MOE), Peking University 3D Digitizing Pipeline Pre-Processing Post-Processing Registration Merging Display Measuring Restoration Applications ＋ ＝ … 3D Scanning

26. Key Laboratory of Machine Perception (MOE), Peking University 3D Scanners in Use Cyrax 2500 RIEGL LMS-Z210i OKIO VIVID 910

27. Key Laboratory of Machine Perception (MOE), Peking University 3D scanner Data Acquisition Get 3D range information of surfaces. Scan an object from different view points.

28. Key Laboratory of Machine Perception (MOE), Peking University Pre-processing Denoising Decimation

29. Key Laboratory of Machine Perception (MOE), Peking University Registration

30. Key Laboratory of Machine Perception (MOE), Peking University Data Merging

31. Key Laboratory of Machine Perception (MOE), Peking University Post-processing

32. Key Laboratory of Machine Perception (MOE), Peking University project Texture Mapping by Using Digital Color Images

33. Key Laboratory of Machine Perception (MOE), Peking University Line Drawing

34. Key Laboratory of Machine Perception (MOE), Peking University 3D Model Aided Line Drawing Photo Rendered 3D Model Line Drawing Based on 3D Model Advantage Advantage ̶ Precise measurement ̶ Fast ̶ Parallel projection, no distortion ̶ Multiple sections from any position ̶ Line drawing for large scale scenes

35. Key Laboratory of Machine Perception (MOE), Peking University Sections from Multiple Positions

36. Key Laboratory of Machine Perception (MOE), Peking University LeiGuTai, LongMen Grottoes Photo 8.5m 20m Traditional method 480 hours Our method 8 hours (for modeling) + 24 hours (for line drawing) Saved time 448 hours Parallel projection map Line drawing Efficiency of 3D Model Aided Line Drawing

37. Key Laboratory of Machine Perception (MOE), Peking University Buddha statue in the north cave PhotoParallel projection mapLine drawing 2.55m 0.9m 1.3m 1.8m Tradition al method 360 hours Our method 4 hours (for modeling)+32 hours (for line drawing) Saved time 324 hours Efficiency of 3D Model Aided Line Drawing

38. Key Laboratory of Machine Perception (MOE), Peking University Parallel projection map Line drawing Photo Bodhisattva statue in the middle cave 0.65m 0.4m 0.5m 1.85m Tradition al method 96 hours Our method 4 hours (for modeling)+40 hours (for line drawing) Saved time 52 hours Efficiency of 3D Model Aided Line Drawing

39. Key Laboratory of Machine Perception (MOE), Peking University 0.65m 0.40m 0.5m 1.85m Automatic Line Drawing image from 3D data automatically produced sketch model assisted drawing

40. Key Laboratory of Machine Perception (MOE), Peking University A large scene in Longmen Grottoes Automatic Line Drawing A Buddha statue

41. Key Laboratory of Machine Perception (MOE), Peking University Digital Restoration Budda in GuYang Cave, LongMen Grottoes

42. Key Laboratory of Machine Perception (MOE), Peking University Bodhisattva in HuoDing Cave, LongMen Grottoes Digital Restoration

43. Key Laboratory of Machine Perception (MOE), Peking University LongMen Grottoes

44. Key Laboratory of Machine Perception (MOE), Peking University MaiJiShan Grottoes A joint project: Toppan Printing Co., Waseda Univ., Peking Univ. Objective: to develop advanced 3D modeling, rendering, displaying and interaction methods for digital heritage A case study: Maijishan Grottos – production of 3D video for the Cave #123

45. Key Laboratory of Machine Perception (MOE), Peking University MaiJiShan Grottoes

46. Key Laboratory of Machine Perception (MOE), Peking University Budda in South Cave, XiangTangShan Grottoes Digital Restoration

47. Key Laboratory of Machine Perception (MOE), Peking University Statues in South Cave, XiangTangShan Grottoes Digital Restoration

48. Key Laboratory of Machine Perception (MOE), Peking University XiangTangShan Grottoes

49. Key Laboratory of Machine Perception (MOE), Peking University Conclusions u 3D digital technology is very useful for preservation, research, and knowledge dissemination for cultural heritages u It requires collaboration among different disciplines: archaeology, cultural heritage, information science and technology, … u Well-defined objectives u High-quality results u Useful in some aspects for the application u How to find research issues from the application work?

50. Key Laboratory of Machine Perception (MOE), Peking University

52. Thank You!

53. Key Laboratory of Machine Perception (MOE), Peking University Tomb Tomb ̶ 400m 2 Objects Objects ̶ 20 GuangZhou NanYueWang Museum

54. Key Laboratory of Machine Perception (MOE), Peking University Digital Archeology Report Photo Database management Database management ̶ Photos, 3D models, and relevant texts ̶ From large scenes to single statues ̶ Basic operations 3D measurement and analysis 3D measurement and analysis Section from any position Section from any position 3D ViewMeasurement

55. Key Laboratory of Machine Perception (MOE), Peking University Digital Museum   3D scanning and picturing   3D modeling   Texture mapping   Virtual display

56. Key Laboratory of Machine Perception (MOE), Peking University Other Cases