1. Adaptive Point Cloud Rendering A Feasibility Study
Group May13-11
Christopher Jeffers - Communicator
Eric Jensen - Webmaster
Joel Rausch - Coordinator
Client: Siemens PLM Software
Advisor: Professor Simanta Mitra

2. Outline What Is Siemens PLM Software? What Is a Point Cloud?
What Are We Doing?
What Are the Requirements?
What Solutions are There?
How Are We Going to Do It?
What Have We Done?
What Will We Do?
Questions?
© Kean Walmsley

3. Siemens PLM Software
“A business unit of the Siemens Industry Automation Division”
“A leading global provider of product lifecycle management (PLM) software and services.”
“Nearly 6.7 million licensed seats and 69,500 customers worldwide”
“Headquartered in Plano, Texas.”
“Works collaboratively with companies to deliver open solutions that help them turn more ideas into successful products.”
“For more information on Siemens PLM Software products and services, visit
Images and text from Siemens PLM’s Goelus Brochure (

4. Star Wars: Rogue Squadron © LucasArts
What is a Point Cloud?
A collection of vertices
In computer graphics, a way to represent an object or scene as set of points
Polygons
Star Wars: Rogue Squadron © LucasArts
Lines/Vectors
Star Wars © Atari
Points
© Nghia Ho
Click Animation L: M: R:

5. © Stanford Computer Graphics Laboratory
Why Have Point Clouds?
3D scanners output points clouds
19-year old toy
© 1993 Bandai
Stanford Armadillo
© Stanford Computer Graphics Laboratory

6. Primary Business Use Case
Siemens PLM provides software that can simulation a layout of machinery on a factory floor.
Currently, the model of the factory is based on the building plans
The problem: these floor plans are not always maintained, as new machinery is added.
Siemens PLM’s idea: Scanned the build, so it is always correct.
Leica terrestrial LIDAR (Light Detection And Ranging)
© 2007 David Monniaux

7. How to Deal With a Point Cloud?
Convert it to a polygon mesh?
Readily available, but not designed for their needs and good results require creation-time knowledge
Convert it to multiple meshes?
Not an easy solution = = /

8. So What Are We Doing?
We are testing the feasibility rendering a point cloud as a collection of points.
Our Goals
Create a medium-scale (100s of millions to billions of points) point cloud rendering kernel using C++ and OpenGL.
Implement a data structure to organize the point cloud data.
Implement a traversal or selection algorithm to choose the correct points to render from the current viewpoint.
Create a viewer that gives the user basic input controls to manipulate the view of the point cloud.

9. Requirements Functional Non-Functional
Allow the user to load a point cloud in a TBD format .
Allow the user to rotate, zoom, and pan the view of the point cloud.
Non-Functional
Rendering kernel must be able to display millions of points concurrently.
The application must maintain an interactive frame rate (> 1 fps)
Selection algorithm should take no more than 10 sec to run.

10. Requirements, cont. Inverse Requirements Constraints
Mesh reconstruction from point clouds
Segmentation (RANSAC)
Point cloud compression
Collision detection
Brute Force Approaches
Constraints
Must be written in C++ programming language.
Must use OpenGL library for rendering graphics.
All data must fit within a few gigabytes of main memory.

11. How to Render A Billion Points?
We looked through 10+ papers.
We selected three methods to further investigate and test.
Splatting using Octrees (Kobbelt 02)
QSplats (Rusinkiewicz 00)
Ray Cast Filtering using K-d Trees
(Botsch 03) High-Quality Point-Based Rendering on Modern GPUs
(Kobbelt 02) Efficient High Quality Rendering of Point Samples Geometry
(Linsen 01) Point Cloud Representation
(Moenning 03) A New Point Cloud Simplification Algorithm
(Rusinkiewicz 00) QSplat: A Multiresolution Point Rendering System for Large Meshes
(Rusu 11) Point Cloud Library
(Wimmer 06) Instant Points Fast Rendering of Unprocessed Point Clouds
(Rusinkiewicz 01) Streaming QSplat: A Viewer for Networked Visualization of Large, Dense Models
(Havran 00) Heuristic Ray Shooting Algorithms
(Römisch 09) Sparse Voxel Octree Ray Tracing on the GPU
(Kashyap 10) Fast Raytracing of Point Based Models using GPUs
(Marques) GPU Ray Casting
Group: May13-11
Client: Siemens PLM

12. Splatting (Point Rendering)
Developed by Lee Westover - Vis 1989; SIGGRAPH 1990
Every point is treated as a glob of color and is splat on a the frame, like, as Westover said, a snow ball.
Requires front-to-back or back-to-front ordering when using alpha bending to smooth rendering.
We probably will be using a FastSplat or geometry shader method.
Auto animation

13. Splatting using Octrees – Octree
A tree structure is built by sub-dividing Euclidian space.
A node defines a virtual point at its center
Pros
Easily partitionable
Fixed depth
Cons
Requires knowledge of the bounding box at creation times
All information of
the point cloud
is lost.
Space: O(d*n)
© WhiteTimberwolf
Group: May13-11
Client: Siemens PLM

14. Splatting using Octrees – Algorithm
Based on Efficient High Quality Rendering of Point Samples Geometry by Mario Botsch, Andreas Wiratanaya, Leif Kobbelt
Method: Modified DFS of the octree. Traverse a node when is in the viewing frustum. Terminate branch when no children exist or are traversable or at the resolution boundary of the frustum. Order children in traversal back-to-front of the frustum when blending.
Pros
Back-to-front order for alpha blending.
Easy to off-load parts of the traversal to the GPU
Cons
Hidden points may be rendered
There is a resolution limit and fixed depth.
Space: O(d*n), where d is the depth
Time:
Average: O(A*8k), where k is the resolution boundary and A is the completeness of the octree
Worst: O(d*n) bound by O(8d)
Manual animation

15. Ray Cast Filtering – K-D Tree
BST in k dimensions.
A node defines a point and splitting plane.
Pros
All information of the point cloud is preserved.
Four paths of traversal (L, R, Both, None).
Cons
Creating a balanced tree requires median finding
Cannot be rebalance.
Space: O(n)
© KiwiSunset (upper)
© Myguel (lower)
Group: May13-11
Client: Siemens PLM

16. Ray Cast Filtering – Algorithm
Derived from ray tracing
Use view rays to select only
the points that can be seen.
Pros
Independent updating and
rendering.
No resolution limits.
Fixed number of points being
rendered (< 9M).
Cons
No filtering.
Viewport resolution changes are ill defined.
Back-to-front alpha bending is ill defined.
Space: O(n)
Time:
Average: O(R lg(n) + R) should be possible
Worst: O(R * n), where R is the number of pixels

17. Ray Cast Filtering – Ray Casting
S1: Test if the ray on the node’s splitting plane and store.
S2: If S1 is false, traverse the child closest to the origin. Else, if there is an equal-axis child closer to the origin, traverse the or-equal child.
S3: If S2’s child returns a point, return that point.
S4: Test if the ray hit this node’s point.
((𝑓 ≥ 0) && (𝑔 < 0)) ||
((𝑓 < 0) && (𝑓2 ≥ 𝑔))
S5: If S4 is true, return this node’s point.
S6: If S1 is false, return empty. Else, Test if the ray intersection with the node’s axis.
S7: If S6 is true, traverse remaining child.
S8: If S7’s child returns a point, return that point.
S9: Return empty.
𝑓 = 𝐸 ∙ 𝐷
𝑔 = ‖𝐷‖2 ∙ (‖𝐸‖2 − 𝑟2)
𝐸 = 𝑂 − 𝐶 r C O D 9 8 7 6 5 4 3 2 1

18. QSplat – Data Structure
Bounding sphere hierarchy
Property: Parent less detailed than children
Memory Layout
Nodes arranged breadth first in a contiguous memory region
Bounding sphere nodes
Requires 32 bits without color; 48 bits with color
Use special relations instead of pointers
(Rusinkiewicz 00)

19. QSplat – Algorithm
Based off the work of Szymon Rusinkiewicz and Marc Levoy
Level of detail scaling
Rough details are painted over with finer details
Quality / speed tradeoff
High quality while stationary
Low quality while moving
Refinement
Improve quality in subsequent stationary frames
Uses feedback system

20. Metrics *Memory/HDD Footprint *Load Time Size of backend
Graphics Performance
Backend Performance
Frame Rate (Hz)
The time it takes to draw a frame
*Update Rate (Hz)
The time it takes to update to a new static camera setting
Fragment Coverage (%)
The percentage of unique pixels that were determined
Response Time (ms)
The time it takes to begin updating update
*Memory/HDD Footprint
Size of backend
*Load Time
Time it takes to load X numbers of points into the backend
* Our primary concerns for our prototypes

21. Results of prototypes - Splatting
Node Size: 72 byte
Packing: 8 – 8-byte pointers, 4-byte color, 4-byte normal
Load Time
2,000,000 pts: 6 sec
10,000,000 pts: 32 sec
Update Time
10,000,000 pts: ~
For 1 billion points (estimated):
Memory Size:
Load Time: 1-2 hrs
Update Time: ~

22. Results of prototypes – Ray Casting
Node Size: 20 bytes
Packing: 2 – 4-byte pointers, 3-byte color, 3-byte normal, 6-byte point (3 – half floats)
Load Time (Simulated):
2,000,000 pts: 5 sec
20,000,000 pts: 53 sec
Ray Cast (Simulated):
2,000,000 pts: 2 sec
20,000,000 pts: 14 sec
200,000,000 pts: sec
Best Case for 20,000,000 pts: 0 sec
For 1 billion points (estimated):
Memory Size: 20 GB
Load Time: 1-2 hrs
Update Time (at 1080p fully coverage): 4 yrs (1 min / cast)

23. Results of prototypes - QSplat
Node Size: 6 bytes
Packing: 13 bits for position & radius, 3-bit pointer, 14-bit normal, 2-bit normal cone width, 16-bit color
Load Time
10,000,000 pts: 20 seconds
Update Time
10,000,000 pts: speed/quality tradeoff
For 1 billion points (estimated):
Memory Size: 6 GB
Load Time: 3-4 hrs
Update Time: speed / quality tradeoff

24. Results of prototypes - Summary
What We Learned:
Best option is QSplat
We will need to create the structure at a separate time and store it in permanent storage
We will want to cache splats to the GPU
We will want separate rendering and updating
Alg.
Mem.
Size
Load
Time
Up-date
Octrees
27 GB
1-2 hrs ~
Ray Casting
20 GB
4 yrs
QSplat
6 GB
3-4 hrs
Vari-able

25. System Description
State Diagram
Class Diagram

26. Accomplishments Figured out the problem.
Researched multiple solutions.
Selected the three most promising solutions.
Prototyped our solutions.
Selected the solution to be fully implemented based on our findings.
We design an architecture for our application.
Created our Project Plan, Design Document, and this presentation.
We still alive.

27. What’s Next? Fully developing our Qsplat implementation.
Creating our application.
Testing our implementation
Redesigning everything to improve performance
Design a GPU-side solution
Graduate and then sleep and then going to work.

28. Conclusion What is Siemens PLM Software? What is a Point Cloud?
What are We Doing?
What are the Requirements?
What Solutions are There?
How are We Going to Do It?
What Have We Done?
What Will We Do?
© Kean Walmsley

29. Questions Auto animation
© LucasArts X-Wing