1. computer graphics & visualization Point-Based Computer Graphics

2. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Why Points? huge geometry complexity of current CG models overhead introduced by connectivity of polygonal meshes acquisition devices generate point samples “digital 3D photography” points complement triangles

3. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Polynomials Rigorous mathematical concept Robust evaluation of geometric entities Shape control for smooth shapes Require proper parameterization Discontinuity modeling Topological flexibility

4. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Polynomilas  Triangles Piecewise linear approximations Irregular sampling of the surface No parameterization needed (geometry only)

5. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Triangles Simple and efficient representation Hardware pipelines support triangles Advanced geometric processing The widely accepted queen of graphics primitives Sophisticated modeling is difficult (Local) parameterizations still needed Complex LOD management Compression and streaming is highly non-trivial

6. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Triangles  Points Piecewise linear functions to Delta distributions Discrete samples of geometry No connectivity or topology – most simple Store all attributes per surface sample

7. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Points geometry complexity of current CG models connectivity overhead of polygonal meshes acquisition devices generate point samples points complement triangles holes compression

8. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group AcquisitionRendering RepresentationProcessing & Editing Point Based Graphics Taxonomy

9. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group How can we capture reality?

10. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Acquisition Contact digitizers – intensive manual labor Passive methods – require texture, Lambertian BRDF Active light imaging systems – restrict types of materials in general fuzzy, transparent, and refractive objects are difficult

11. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group First Method, Laser Range Scanner

12. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Basic Idea Laser Detector Laser Detector

13. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Computing the Distance Laser Detector H L O d d’ a’ a

14. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Scattering Issues How optically cooperative is marble?

15. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Image based Acquisition

16. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Image based Acquisition Acquisition Stage 2

17. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Image based Acquisition Acquisition Stage 3

18. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group IBA - Process

19. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Visual Hull

20. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Visual Hull the quality of the visual hull geometry is a function of the number of viewpoints / silhouettes the method is unable to capture all concavities  image based lighting

21. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Point Based Rendering Surfels (surface element)

22. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Extended Surfels

23. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Rendering Pipeline

24. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Uniform Reconstruction For uniform samples, use signal processing theory Reconstruction by convolution with low-pass (reconstruction) filter Exact reconstruction of band-limited signals using ideal low-pass filters

25. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Non-Uniform Reconstruction Signal processing theory not applicable for nonuniform samples Local weighted average filtering – Normalized sum of local reconstruction kernels

26. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Reconstruction 1D in 2D

27. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Reconstruction 2D in 3D

28. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Algorithm for each sample point { shade surface sample; splat = projected reconstruction kernel; rasterize and accumulate splat; } for each output pixel { normalize; }

29. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Results without Normalization with Normalization

30. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Visibility ε-z-buffering

31. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Implementation Use a three pass algorithm: 1.Draw depth image with a small depth offset ε away from the viewpoint Perform regular z-buffering (depth tests and updates), no color updates

32. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Second Pass Draw colored splats with additive blending enabled Perform depth tests, but no updates Accumulate – Weighted colors of visible splats in the color channels – Weights of visible footprint functions in the alpha channel

33. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Third Pass Normalization of the color channels by dividing through the alpha channel Implemented by – render to texture – drawing a screen filling quad with this texture – performing the normalization in the pixel shader

34. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Efficient Data Structures DuoDecim A Structure for Point Scan Compression and Rendering

35. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group a.d. 1500 created beautiful statues … but missed to make them portable David:434 cm Atlas:208 cm Barbuto:248 cm … Florence, Galleria dell'Accademia

36. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group a.d. 1999 Marc Levoy et al. (Stanford University) did a great job scanning the statues … but still “missed” to make them “portable” David:1.1 GB Atlas:10 GB … All in all 32 gigabytes raw data !!! Jens Krüger - Computer Graphics and Visualization Group, TU- München Image courtesy of Marc Levoy

37. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Today… … we present novel algorithms to make the statues of Michelangelo „portable“ and „viewable“ on PCs.

38. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group The Numbers The Atlas Statue Scan Size of raw scans: approx 600 mio. points Reconstruction: at 1/4 mm Size of reconstruction:approx 250 mio. vertices, approx 500 mio. triangles Filesize (without normals):9.94 GB „Your Home PC“ Size of Main Memory: 1 – 2 GB Size of Graphics Card Memory:0.125 – 0.5 GB

39. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group 230 MB vs. 10 GB Rendering and decoding time in full resolution 4 sec.

40. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group The Key Idea CPU Sample the point scan into a regular grid Divide the grid up into 2D slices Look for connected runs within the slices Store the starting position/normal per run Store position/normal delta for the rest GPU Store the compressed runs into textures Upload the compressed runs onto the GPU Decode the points with normals on the fly

41. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group 1. Sample the point scan into a regular grid Hexagonal close sphere packing (HCP) grid

42. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Cell Search (HCP) 2D Simplification

43. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group 2. Divide the grid up into 2D slices

44. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group 3. Look for connected runs within the slices l k f g i j h m n o q p d b e c a abc dklmn op oq def gh gij defghgijigfedklmnopoq opoq ghgij

45. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group 3. Connected runs (cont.)

46. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group 4. Store the starting position/normal per run Start Position  two (16bit) indices per point – one (16bit) index per slice Start Normal  one (16bit) index – one (16bit) Codebook per dataset

47. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group 5. Store position/normal delta for the rest Delta Position  6 cases  2.25 bit Delta Normal  5bit index – one (5bit) Codebook per dataset contains sin/cos of delta angles

48. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Rendering GPU Decoding and Rendering Store the compressed runs into textures Upload the compressed runs onto the GPU Decode the points with normals on the fly

49. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group 1. Store the compressed runs into textures Start Position/Normal Delta Position/Normal 16bit (RG)(B) 8bit (3+5) Per Slice: 16bit height 8 Floats to the GPU

50. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group n m 3. Decode the points with normals on the fly Start Position/Normal n m Position Normal Render as Points via: PBO/VBO Copy Cast to Vertex Array Vertex Texture Fetch

51. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group n m 3. Decode the points with normals on the fly Start Position/Normal n m Position Normal Delta Position/Normal 1 n m n m Position Normal Pixelshader Render as Points

52. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group One Image about quality…