Realtime 3D model construction with Microsoft Kinect and an NVIDIA Kepler laptop GPU Paul Caheny MSc in HPC 2011/2012 Project Preparation Presentation.

Today's Presentation Project motivation. Introduction to Microsoft Kinect and depth maps. A taster on algorithms for constructing 3D models from depth maps. Particular goals and constraints for this dissertation. Paul Caheny 2012

Project Motivation Proposal from Holoxica Ltd. Most holograms produced from 3D models which are computer generated ab initio. Some customers already posses high quality 3D models of real world objects. Commercial scanning solutions prohibitively expensive for Holoxica's purposes. A cheaper, good quality, portable 3D scanning & model construction system could make holograms a more viable and attractive proposition for customers. Paul Caheny 2012

Microsoft Kinect & Depth Maps Depth maps have been the subject of much research for 3D model construction over the past 20 years. The Kinect provides unprecedented quality to cost ratio as a depth sensor. Video Camera, IR Projector, IR Camera IR Projector/Camera constitute a depth sensor providing a 640 x 480 depth map at 30Hz frame rate. Paul Caheny 2012

3D Modelling from Depth Maps Put simply – fusing multiple viewpoints of the scene or object into a single unified 3D model. Early approaches involved highly calibrated motion of the object plus simple surface construction techniques (e.g. directly meshing the depth data points). Current state of the art replaces calibrated motion with software tracking & high quality 3D model construction techniques. Two techniques: Iterative Closest Point (ICP) algorithm for object tracking and Volumetric Integration for high quality model generation. Paul Caheny 2012

Iterative Closest Point (ICP) A method for the alignment of 3D surfaces. Introduced independently by Besl & McKay at GM and Chen & Medioni at Uni. Southern California in early '90s. If corresponding points on surface in two views are known, trivial to compute exact transform which aligns surfaces. But we don't know corresponding points on surfaces in distinct depth maps. Make a heuristic guess of corresponding points, compute transform with selected points, achieving closer alignment. Reselect points heuristically on transformed surfaces, iterate until close enough. Paul Caheny 2012

Volumetric Integration A technique for fusing data from multiple aligned depth maps. ICP alignment & depth sensors have inherent margin of error / measurement uncertainty. Results in noisy data points following ICP alignment. Reduce noise & improve result by combining multiple views using heuristics which minimises noise. Curless & Levoy, Stanford '96 Paul Caheny 2012

What's New? ICP and Volumetric Integration introduced in the '90s. Both techniques have have been subject of much refinement since. In the 90s workflow looked like: Scan -> ICP Batch process -> Volumetric Integration Batch process -> Finished 3D Model. Mid 2000s saw systems with realtime ICP phase plus batch Volumetric Integration phase to create finished model saw publication of research by Augmented Reality Group at Microsoft Research Cambridge demonstrating a realtime, synchronous ICP and Volumetric Integration system called KinectFusion running at Kinect full frame rate of 30Hz. Paul Caheny 2012

My Dissertation Focus on small object scanning and Laptop GPU. Implement the algorithms from scratch using CUDA. Plan to use NVIDIA's next generation Kepler architecture, recently launched for consumer desktop & laptop market. Laptop GPU TDP ~35W versus Tesla 2090 TDP of 225W – Fewer Cores, Less Memory, Less Bandwidth, Lower Clock Speed. Paul Caheny 2012

Wrap Up References: R. Newcombe et al. KinectFusion: Real-time dense surface mapping and tracking. In Proc. 10th IEEE Int. Symp. on Mixed and Augmented Reality, B. Curless and M. Levoy. A volumetric method for building complex models from range images. ACM Trans. on Graphics, P. Besl and N. McKay. A method for registration of 3D shapes. IEEE Trans. on Pattern Analysis & Machine Intelligence, 14:239–256, Y. Chen and G. Medioni. Object modeling by registration of multiple range images. Image and Vision Computing (IVC), 10(3):145–155, Paul Caheny 2012