1. A Multiresolution Point Rendering System for Large Meshes Szymon Rusinkiewicz & Marc Levoy Stanford University Edited by Ingrid

2. Introduction Trend in computer graphics towards using sampled representations of real objects during renderingTrend in computer graphics towards using sampled representations of real objects during rendering Size of data meshes that can be scanned is on the order of hundreds of millions of pointsSize of data meshes that can be scanned is on the order of hundreds of millions of points Scanned data has a large number of verticies whose locations are often imprecise due to noiseScanned data has a large number of verticies whose locations are often imprecise due to noise

3. IntroductionIntroduction Evidenced by increased use of 3D scanners to produce three dimensional sampled models of objectsEvidenced by increased use of 3D scanners to produce three dimensional sampled models of objects This is FAR beyond what is currently displayable in real time This is FAR beyond what is currently displayable in real time Suggests an approach where individual points are relatively unimportant Suggests an approach where individual points are relatively unimportant

4. Introduction Examples of recent research employing this paradigmExamples of recent research employing this paradigm – Spline-fitting system by Krishnamurthy and Levoy – Range image merging system by Curless and Levoy – Octree particle rendering system by Yemez and Schmitt These algorithms do not preserve the original locations of the scanned pointsThese algorithms do not preserve the original locations of the scanned points

5. What is QSplat? System for representing and progressively displaying large scale meshes of scanned dataSystem for representing and progressively displaying large scale meshes of scanned data Combines a multi resolution hierarchy based on bounding spheres with a rendering system based on pointsCombines a multi resolution hierarchy based on bounding spheres with a rendering system based on points Does not preserve the connectivity of the original meshDoes not preserve the connectivity of the original mesh Intended to visualize scanned models that contain significant amounts of fine detail at scales near the scanning resolutionIntended to visualize scanned models that contain significant amounts of fine detail at scales near the scanning resolution

6. System Outline Preprocessing AlgorithmPreprocessing Algorithm – generates hierarchy from triangular mesh Data StructureData Structure – bounding spheres Node StructureNode Structure – sphere center, radius, normal, width of a normal cone, (optionally) a color Rendering AlgorithmRendering Algorithm – visibility culling & drawing a splat

7. if (begin == end) return Sphere(verticies[begin]) else midpoint = PartitionLongestAxis(verticies[begin..end]) leftstree = BuildTree(verticies[begin..midpoint]) rightstree = BuildTree(verticies[midpoint+1..end]) return BoundingSphere(leftsubtree, rightsubtree) Preprocessing Algorithm Bounding Box of vertices Recurse down each subtree, finding its bounding box

8. Preprocessing Algorithm Position and size of each bounding sphere is encoded relative to its parent and quantisized to 13 values to save spacePosition and size of each bounding sphere is encoded relative to its parent and quantisized to 13 values to save space Quantisized down the mesh, so errors do not propagateQuantisized down the mesh, so errors do not propagate Round up to ensure no holes introduced by quantizationRound up to ensure no holes introduced by quantization

9. Creating the Data Structure Start with a triangle mesh produced by aligning and integrating scans [Curless 96]Start with a triangle mesh produced by aligning and integrating scans [Curless 96]

10. Creating the Data Structure Place a sphere at each node, large enough to touch neighbor spheresPlace a sphere at each node, large enough to touch neighbor spheres

11. Creating the Data Structure Build up hierarchyBuild up hierarchy

12. QSplat Data Structure A single bounding sphere hierarchy used forA single bounding sphere hierarchy used for – Hierarchical frustum and backface culling – Level of detail control – Splat rendering [Westover 89]

13. QSplat Node Structure Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits 6 bytes

14. QSplat Node Structure Position and radius encoded relative to parent node – Hierarchical coding vs. delta coding along a path for vertex positions Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits 2 bits 16 bits Center Offset Radius Ratio

15. QSplat Node Structure Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits 2 bits 16 bitsUncompressed

16. QSplat Node Structure Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits 2 bits 16 bits Delta Coding [Deering 96]

17. QSplat Node Structure Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits 2 bits 16 bits Hierarchical Coding

18. QSplat Node Structure Number of children (0, 2, 3, or 4) – 2 bits Presence of grandchildren – 1 bit Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits

19. QSplat Node Structure Normal quantized to grid on faces of a cube Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits 52  52  6

20. QSplat Node Structure Each node contains bounding cone of children’s normals Hierarchical backface culling [Kumar 96] Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits

21. QSplat Node Structure Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits Culled Not Culled Viewer

22. QSplat Node Structure Per-vertex color is quantized 5-6-5 (R-G-B) Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits

23. QSplat Rendering Algorithm if (node not visible) Skip this branch else if (leaf node) Draw a splat else if (size on screen < threshold) Draw a splat else Traverse children Hierarchical frustum / backface culling Point rendering Adjusted to maintain desired frame rate Level of detail control

24. Rendering Algorithm Visibility CullingVisibility Culling – As recurse bounding spheres, cull nodes not visible – Backface culling if cone faces away, discard node & subtreeif cone faces away, discard node & subtree If cone faces viewer, mark node no cullingIf cone faces viewer, mark node no culling Frustum culling is performed by testing each sphere against planes of the view frustumFrustum culling is performed by testing each sphere against planes of the view frustum – discard nodes and subtrees that are outside the view frustum

25. Rendering Algorithm Determining when to recurseDetermining when to recurse – Based on projected size of screen – Node is subdivided if area of sphere when projected on viewing plane exceeds a threshold – Adjusted frame to frame to maintain user selected frame rate Type of data does not require smoothing or complex frame rate adjustmentType of data does not require smoothing or complex frame rate adjustment

26. Frame Rate Control Feedback-driven frame rate controlFeedback-driven frame rate control – During motion: adjust recursion threshold based on time to render previous frame – On mouse up: redraw with progressively smaller thresholds – Consequence: frame rate may vary Alternative:Alternative: – Predictive control of detail [Funkhouser 93]

27. PolygonsQSplat Tradeoffs of Splatting For rendering large 3D models, what are the tradeoffs of:For rendering large 3D models, what are the tradeoffs of: Good for large, flat or subtly curved regions Good for models with detail everywhere Highly-efficient rasterization with 3D graphics hardware Higher per-pixel cost, but less slowdown in absence of 3D hardware Decimation or creating LOD data structures is often expensive Fast preprocessing

28. Demo – St. Matthew 3D scan of 2.7 meter statue at 0.25 mm3D scan of 2.7 meter statue at 0.25 mm 102,868,637 points102,868,637 points File size: 644 MBFile size: 644 MB Preprocessing time: 1 hourPreprocessing time: 1 hour Demo on laptop (PII 366, 128 MB), no 3D graphics hardwareDemo on laptop (PII 366, 128 MB), no 3D graphics hardware

29. Future Work Splats as primitiveSplats as primitive – Unify rendering of meshes, volumes, point clouds – Compatible with shading after rasterization – Hybrid point/polygon systems High-level visibility / LOD frameworksHigh-level visibility / LOD frameworks – Store different kinds of data at each node: alpha, BRDF, scattering function, etc. – Potentially could be used to unify image- based-rendering (IBR) techniques

30. Acknowledgments Thanks to Gary King, Dave Koller, Jonathan Shade, Matt Ginzton, Kari Pulli, Lucas Pereira, James Davis, and the whole DMich gangThanks to Gary King, Dave Koller, Jonathan Shade, Matt Ginzton, Kari Pulli, Lucas Pereira, James Davis, and the whole DMich gang Digital Michelangelo Project sponsored by Stanford University, Interval Research Corporation, and the Paul Allen Foundation for the ArtsDigital Michelangelo Project sponsored by Stanford University, Interval Research Corporation, and the Paul Allen Foundation for the Arts

31. QSplat binaries and source codeQSplat binaries and source code Digital Michelangelo Project archive atDigital Michelangelo Project archive at QSplat Downloads http://graphics.stanford.edu/software/qsplat http://graphics.stanford.edu/projects/mich