Presentation on theme: "Adaptive Point Cloud Rendering A Feasibility Study"— Presentation transcript:
1 Adaptive Point Cloud Rendering A Feasibility Study Group May13-11Christopher Jeffers - CommunicatorEric Jensen - WebmasterJoel Rausch - CoordinatorClient: Siemens PLM SoftwareAdvisor: Professor Simanta Mitra
3 Siemens PLM Software“A business unit of the Siemens Industry Automation Division”“A leading global provider of product lifecycle management (PLM) software and services.”“Nearly 6.7 million licensed seats and 69,500 customers worldwide”“Headquartered in Plano, Texas.”“Works collaboratively with companies to deliver open solutions that help them turn more ideas into successful products.”“For more information on Siemens PLM Software products and services, visitImages and text from Siemens PLM’s Goelus Brochure (http://www.plm.automation.siemens.com/en_us/products/open/geolus/index.shtml#lightview%26uri=tcm: %26title=Geolus%20-%20PLM%20Components%20Brochure%20-%208123%26docType=.pdf)
7 How to Deal With a Point Cloud? Convert it to a polygon mesh?Readily available, but not designed for their needs and good results require creation-time knowledgeConvert it to multiple meshes?Not an easy solution==/
8 So What Are We Doing?We are testing the feasibility rendering a point cloud as a collection of points.Our GoalsCreate a medium-scale (100s of millions to billions of points) point cloud rendering kernel using C++ and OpenGL.Implement a data structure to organize the point cloud data.Implement a traversal or selection algorithm to choose the correct points to render from the current viewpoint.Create a viewer that gives the user basic input controls to manipulate the view of the point cloud.
9 Requirements Functional Non-Functional Allow the user to load a point cloud in a TBD format .Allow the user to rotate, zoom, and pan the view of the point cloud.Non-FunctionalRendering kernel must be able to display millions of points concurrently.The application must maintain an interactive frame rate (> 1 fps)Selection algorithm should take no more than 10 sec to run.
10 Requirements, cont. Inverse Requirements Constraints Mesh reconstruction from point cloudsSegmentation (RANSAC)Point cloud compressionCollision detectionBrute Force ApproachesConstraintsMust be written in C++ programming language.Must use OpenGL library for rendering graphics.All data must fit within a few gigabytes of main memory.
11 How to Render A Billion Points? We looked through 10+ papers.We selected three methods to further investigate and test.Splatting using Octrees (Kobbelt 02)QSplats (Rusinkiewicz 00)Ray Cast Filtering using K-d Trees(Botsch 03) High-Quality Point-Based Rendering on Modern GPUs(Kobbelt 02) Efficient High Quality Rendering of Point Samples Geometry(Linsen 01) Point Cloud Representation(Moenning 03) A New Point Cloud Simplification Algorithm(Rusinkiewicz 00) QSplat: A Multiresolution Point Rendering System for Large Meshes(Rusu 11) Point Cloud Library(Wimmer 06) Instant Points Fast Rendering of Unprocessed Point Clouds(Rusinkiewicz 01) Streaming QSplat: A Viewer for Networked Visualization of Large, Dense Models(Havran 00) Heuristic Ray Shooting Algorithms(Römisch 09) Sparse Voxel Octree Ray Tracing on the GPU(Kashyap 10) Fast Raytracing of Point Based Models using GPUs(Marques) GPU Ray CastingGroup: May13-11Client: Siemens PLM
12 Splatting (Point Rendering) Developed by Lee Westover - Vis 1989; SIGGRAPH 1990Every point is treated as a glob of color and is splat on a the frame, like, as Westover said, a snow ball.Requires front-to-back or back-to-front ordering when using alpha bending to smooth rendering.We probably will be using a FastSplat or geometry shader method.Auto animation
14 Splatting using Octrees – Algorithm Based on Efficient High Quality Rendering of Point Samples Geometry by Mario Botsch, Andreas Wiratanaya, Leif KobbeltMethod: Modified DFS of the octree. Traverse a node when is in the viewing frustum. Terminate branch when no children exist or are traversable or at the resolution boundary of the frustum. Order children in traversal back-to-front of the frustum when blending.ProsBack-to-front order for alpha blending.Easy to off-load parts of the traversal to the GPUConsHidden points may be renderedThere is a resolution limit and fixed depth.Space: O(d*n), where d is the depthTime:Average: O(A*8k), where k is the resolution boundary and A is the completeness of the octreeWorst: O(d*n) bound by O(8d)Manual animation
16 Ray Cast Filtering – Algorithm Derived from ray tracingUse view rays to select onlythe points that can be seen.ProsIndependent updating andrendering.No resolution limits.Fixed number of points beingrendered (< 9M).ConsNo filtering.Viewport resolution changes are ill defined.Back-to-front alpha bending is ill defined.Space: O(n)Time:Average: O(R lg(n) + R) should be possibleWorst: O(R * n), where R is the number of pixels
17 Ray Cast Filtering – Ray Casting S1: Test if the ray on the node’s splitting plane and store.S2: If S1 is false, traverse the child closest to the origin. Else, if there is an equal-axis child closer to the origin, traverse the or-equal child.S3: If S2’s child returns a point, return that point.S4: Test if the ray hit this node’s point.((𝑓 ≥ 0) && (𝑔 < 0)) ||((𝑓 < 0) && (𝑓2 ≥ 𝑔))S5: If S4 is true, return this node’s point.S6: If S1 is false, return empty. Else, Test if the ray intersection with the node’s axis.S7: If S6 is true, traverse remaining child.S8: If S7’s child returns a point, return that point.S9: Return empty.𝑓 = 𝐸 ∙ 𝐷𝑔 = ‖𝐷‖2 ∙ (‖𝐸‖2 − 𝑟2)𝐸 = 𝑂 − 𝐶rCOD987654321
18 QSplat – Data Structure Bounding sphere hierarchyProperty: Parent less detailed than childrenMemory LayoutNodes arranged breadth first in a contiguous memory regionBounding sphere nodesRequires 32 bits without color; 48 bits with colorUse special relations instead of pointers(Rusinkiewicz 00)
19 QSplat – AlgorithmBased off the work of Szymon Rusinkiewicz and Marc LevoyLevel of detail scalingRough details are painted over with finer detailsQuality / speed tradeoffHigh quality while stationaryLow quality while movingRefinementImprove quality in subsequent stationary framesUses feedback system
20 Metrics *Memory/HDD Footprint *Load Time Size of backend Graphics PerformanceBackend PerformanceFrame Rate (Hz)The time it takes to draw a frame*Update Rate (Hz)The time it takes to update to a new static camera settingFragment Coverage (%)The percentage of unique pixels that were determinedResponse Time (ms)The time it takes to begin updating update*Memory/HDD FootprintSize of backend*Load TimeTime it takes to load X numbers of points into the backend* Our primary concerns for our prototypes
24 Results of prototypes - Summary What We Learned:Best option is QSplatWe will need to create the structure at a separate time and store it in permanent storageWe will want to cache splats to the GPUWe will want separate rendering and updatingAlg.Mem.SizeLoadTimeUp-dateOctrees27 GB1-2 hrs~Ray Casting20 GB4 yrsQSplat6 GB3-4 hrsVari-able
26 Accomplishments Figured out the problem. Researched multiple solutions.Selected the three most promising solutions.Prototyped our solutions.Selected the solution to be fully implemented based on our findings.We design an architecture for our application.Created our Project Plan, Design Document, and this presentation.We still alive.
27 What’s Next? Fully developing our Qsplat implementation. Creating our application.Testing our implementationRedesigning everything to improve performanceDesign a GPU-side solutionGraduate and then sleep and then going to work.