Faculty of Sciences and Technology from University of Coimbra Generating 3D Meshes from Range Data [1] Graphic Computation and Three-dimensional Modeling 2009/2010 Nuno Alexandre Neto Dias

2 Summary ● Objectives ● Range images ● Acquisition ● Registration ● Integration – Surface reconstruction ● Zippering method ● Volumetric method ● Final Notes

3 Objectives This work's objective is understand how can an object be acquired and represented into a 3D mesh.

4 Why to do this? ● Reverse engineering (Sometimes is needed...) ● Computer games (Use real objects modeled in 3D) ● Animation movies (Use objects, real people faces, body, etc) ● 3D fax machine (print a 3D object sent at distance – it needs both: a 3D scanner and a 3D printer) ● Biomedical (e.g. Prosthetic) ● Parts inspection (compare real parts to its CAD model) ● Arts (3D artifact generation)

5 3D modeling from range images ● General Steps ● Acquisition – sense the object and produce a range image (it is done multiple times). ● Registration – align points from two or more range images. ● Integration – combine points from two or more range images in order to create a single representation.

6 Acquisition

7 Aqcuisition ● This will be the process in which several range images will be acquired and the matrix of mxn points with depth information recorded.

8 Range data acquisition ● Tactile sensing ● Stereo cameras ● Laser scanner – Time of flight – Triangulation ● The optical triangulation method most described in literature when acquiring range data.

9 Advantages and Disadvantages ● Advantages – Non contact – Safe – Nowadays is becoming inexpensive – Very fast ● Disadvantages – Only acquire visible surface – As others optical sensors, may have problems with some material

10 Optical triangulation scanner The laser strikes the surface and the imaged reflection is triangulated by the CCD sensor. Image adapted from [4]

11 Examples

12 Standford repository Cyberware 3030MS scanne r (not all) ● “The Stanford Bunny has been used extensively as a test model for various research projects in computer graphics.”

13 Range images ● A range image is a matrix of m x n points which represents distances that describe a surface in cartesian coordinates. ● This range images can be obtained by the previous discussed 3D scanners that return an array of distances to the object surface.

14 Error sources – The surface reflectance varies – Corners – Occlusion – Others (e.g. beam angle variation, scanner calibration, inter-reflectance, CCD noise, quantification errors) Image adapted from [3]

15 Tessellation (I) ● After acquisition a tessellation method is applied ● A mesh of triangles is used at all stages – From four adjacent points can be created 0,1 or 2 triangles. – A tessellation threshold is used Image adapted from [2]

16 Tessellation (II) ● After the tessellation the range image will look like as follow: Happy Buddha – This dataset can be retrieved from the Standford Graphics Laboratory repository. Range imageAfter tessellation Real Image Image adapted from [1]

17 Tessellation - detail

18 Resume (back to the first example) This process is done several times

19 Registration

20 Registration ● After the triangle mesh is created for each range image acquired, the next step is to correspond portions of different range images in order to get them aligned with each other.

21 Registration problem ● With two overlapping range scans, which transformation (T), minimizes the distance between them. If correct correspondence between points is known it's easy to find the transformation that match both meshes. If not... use Iterated Closest-Point Algorithm (ICP) This is called a pairwaise registration.

22 Iterated Closest Point algorithm ICP algorithm – Find the nearest points – Compute the optimal T – Repeat until E is small qi are samples from the scan Q pi are the corresponding points on scan P T is the transformation 3D

23 ICP algorithm modified ● Find the nearest position on mesh A to each vertex of mesh B. ● Discard pairs of points that are too far apart. ● Eliminate pairs in which either points is on a mesh boundary. ● Find the rigid transformation that minimizes a weighted least-squared distance ● between the pairs of points. ● Iterate until convergence. Image adapted from [2]

24 Weight assignment ● The final surface will be weighted combination of all range images. ● Weights are assigned at each vertex (e.g. higher sampling rates)

25 Global registration ● Pairwise registration is difficult to perform when: ● The initial position of model is unknown ● Noisy system ● Accumulation errors ● The optimal solution minimizes distances between all range scans simultaneously.

26 Example after registration Image adapted from [1]

27 Surface consolidation (I) (Michael K. Reed [6]) ● No restrictions on topological object type – No assumptions should be made about the object type. ● No holes in the reconstruction – The result should be “watertight”. ● Incremental incorporation of new data – Able to update model acquired so far with new data. ● Robustness – Robust in representation ans integration of range data. ● Time and space efficiency – Computational efficient, when acquiring, a lot of data has to be processed.

28 Surface consolidation (II) ● Order independent modeling – The order of acquiring should be independent of the order of merging. ● Utilization of all range data – The accuracy should be increased by the redundant data. ● Representation of model surface quality – Distinguish which models are properly acquired from the ones that aren't.

29 Integration

30 Integration ● The integration step will produce a final triangulated mesh that will be the result of all merged range images.

31 Surface reconstruction - approaches ● Zippering method (Greg Turk and Marc Levoy) ● Volumetric Method (Brian Curless and Marc Levoy)

32 Zippering Method

33 Mesh Zippering ● Remove overlapping portions of the meshes. ● Clip one mesh against another. ● Remove the small triangles introduced during clipping.

34 Redundancy removal and zippering Image adapted from [1]

35 Redundancy Removal Image adapted from [2]

36 Zippering results (I) Image adapted from [2]

37 Zippering results (II) Image adapted from [2]

38 Consensus geometry ● Each vertex is moved to a consensus position given by a weighted average of positions from the original range images.

39 Consensus geometry results Image adapted from [2]

40 Volumetric Method

41 Volumetric Method (I) Image adapted from [4]

42 Volumetric Method (II) Image adapted from [4]

43 Volumetric Method (III) Image adapted from [4]

44 Hole filling - space carving Image adapted from [4]

45 Volumetric reconstruction results Image adapted from [4]

46 Zippering Vs Volumetric Real driller white painted Image adapted from [4]

47 Final comments ● This algorithms may be used in all 3D scanner. There are two main error sources, the scanner sensors error and the error created by the object (e.g. occlusion). If the scanner changes, the sensors errors will remain and must be considered. ● Generating weight to each vertex may be a hard task. It is not a describes a model to do this. ● The zippering method have the problem that they don't provide a hole filling method (drastic results can be seen in the last example). ● This techniques appeared in the mid-nineties but are still used.

48 References ● [1] Robert Kalnins, Robert Osada, Generating 3D Meshes from Range Data. Advanced Topics in Computer Science: Geometric Modeling for Computer Graphics. Available at ● [2] Greg Turk, Marc Levoy, Zippered Polygon Meshes from Range Images. ● [3] Brian Curless, Marc Levoy, Better Optical Triangulation through Spacetime Analysis. ● [4] Brian Curless, Marc Levoy, A Volumetric Method for Building Complex Models from Range Images. ● [5] Yi-Ping Hung, Chu-Song Chen†, Ing-Bor Hsieh†, Chiou-Shann Fuh, Reconstruction of complete 3D object model from multi-view range images. ● [6] Michael K. Reed, Solid Model from Range Imagery. Phd Thesis.