1. Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer Science, Hunter College, CUNY* 3D Modeling of Historic Sites Using Range and Image Data

2. Sites are subject to erosion, wear, vandalism Moving targets: many phases of construction, damage and repair over many years Need to track changes, foresee structural problems Modeling allows a wider audience to ”virtually” see and tour these sites Our Focus: methods that can reduce modeling time using automatic methods Preserving Cultural Heritage Sites

3. Cathedral St. Pierre, Beauvais, France

4. Modeling the Cathedral Goals: Cathedral on the World Monuments Fund's Most Endangered List. Create 3-D model to examine weaknesses in the building and proposed remedies Establish baseline for condition of Cathedral Visualize the building in previous contexts Basis for a new collaborative way of teaching about historic sites, in the classroom and on the Internet.

5. Commissioned in 1225 by Bishop Milon de Nanteuil Only the choir and transepts were completed - choir in 1272 In 1284 part of the central vault collapsed Area where the nave and façade would be is still occupied by the previous church constructed just before 1000. Completed in 16 th century, the transept was crowned by an ambitious central spire that allowed the cathedral to rival its counterpart in Rome. The tower collapsed on Ascension Day in 1573. History: 1200 - 1600

6. Rendition of original central spire

7. Cathedral survived intense incendiary bombing that destroyed much of Beauvais in WW II. Between 1950-80 many critical iron ties were removed from the choir buttresses in a damaging experiment. Temporary tie-and-brace system installed in the 1990s may have made the cathedral too rigid, increasing rather than decreasing stresses upon it. There continues to be a lack of consensus on how to conserve the essential visual and structural integrity of this Gothic wonder. History: 20 th Century

8. Problems with the Structure Wind Oscillation from English Channel winds Strange inner and outer aisle construction – can cause rotational moments in the structure Leaking Roof, foundation is settling Built in 3 campaigns over hundreds of years with differing attention to detail

9. Time-Lapse Image - Spire Movement Due to Wind

10. Technical Challenges Create Global and coherent geometric models: handle full range of geometries Reducing data complexity Registration of MANY million point data sets Range and intensity image fusion

11. Beauvais Cathedral: Exterior Scanning Session

12. Beauvais Cathedral: Interior

13. Beauvais Cathedral: Interior Scanning Session

14. Exterior: Raw Range Scan

15. Beauvais: Scan Detail

16. Range Registration 3 Step Process: 1. Pairwise registration between overlapping scans. Match 3D lines in overlapping range images. 2. Global registration using graph search to align all scans together. 3. Multi-scan simultaneous ICP registration algorithm (Nishino et. al.) Produces accurate registration.

17. Segmentation Algorithm Creates reduced data sets (~80%). Fit local plane to neighborhood of range points. Classify range points: planar, non-planar, unknown. Merge into connected clusters of co-planar points. Identify boundaries of planes. Used to find prominent linear features for matching.

18. N N 1 2 P P 1 2 R 12 Patches fit around points P1 and P2 P1 and P2 are coplanar if: a=cos (N 1. N 2 ) < angle threshold d=max(|R 12 N 1 |, |R 12 N 2 |) < distance threshold Local Planarity Comparison

19. Segmentation and 3-D Registration Lines

20. Pairwise Registration of Scans: Overview Segmenter creates planes and associated boundary lines Matching lines in 2 scans can be problematic Attributes of lines serve as a good filter for matching lines: length of lines, area of planar region that line bounds Use pairs of matched lines to compute transform Transform each line in one scan to the other. Results are graded by metric of number of matched lines after transform Choose best graded transform

21. Pairwise Registration

22. Registered Scans – Beauvais Cathedral

23. Global Registration

24. Graph Search Global Registration Create weighted graph of scans. Edges of graph are confidence in finding correct registration between pairs of scans Confidence (cost) is number of correctly aligned lines after applying registration (R,T) Global Registration: find max-cost path from pivot scan to each scan

25. Final ICP Registration

28. Beauvais Cathedral Model: Fly-Thru

29. Inside-Out: Beauvais Cathedral

30. Texture Mapped Models

33. Automating the Process: Robot can serve as a sensor platform containing ranging and imaging devices Onboard sensors can register sensor data with the environment: GPS, Odom., Vision Given a 2-D map of an environment, robot can augment 2-D map with 3-D models Planner can interact with robot navigator to move to new viewing sites Build a Mobile Site Modeling Robot

34. GPS DGPS Scanner Network Camera PTU Compass The AVENUE Mobile Platform UTONOMOUS EHICLE FOR XPLORATION AND AVIGATION IN RBAN NVIRONMENTS PC Sonars

35. Future Work Continue to automate the model building process Real-time texture mapping for realistic walk- throughs View planning: finding the right viewpoints and number of views Incorporate sensor planning with robot path planning to plan next view Modeling with non-planar segments. Merging multiple overlapping photos.