Interactive Deformation and Visualization of Level-Set Surfaces Using Graphics Hardware Aaron Lefohn Joe Kniss Charles Hansen Ross Whitaker Aaron Lefohn Joe Kniss Charles Hansen Ross Whitaker

Scientific Computing and Imaging Institute, University of Utah Problem Statement  Goal Interactive system for manipulating level-set, deformable surfaces  Level-Set Challenges Computationally expensive Difficult to control  Solution New streaming narrow-band algorithm Unified computation and visualization

Scientific Computing and Imaging Institute, University of Utah Overview  Motivation and Introduction  A Streaming Narrow-Band Solution 1. Virtual memory model 2. Substreams for static branch resolution 3. Efficient GPU-to-CPU message passing 4. Direct volume rendering of compressed/sparse data  Application and Demo  Conclusions

Scientific Computing and Imaging Institute, University of Utah Level-Set Method  Deformable, implicit surfaces Surface deformation via partial differential equation General, flexible model Introduction Surface Processing Tasdizen et al. IEEE Visualization 2002 Physical Simulation Premoze et al. Eurographics 2003 Segmentation

Scientific Computing and Imaging Institute, University of Utah Level-Set Method  Implicit surface  Distance transform  denotes inside/outside  Surface motion F = Signed speed in direction of normal Introduction

Scientific Computing and Imaging Institute, University of Utah Level-Set Acceleration Initialize Domain Compute Update Domain  Narrow-Band/Sparse-Grid Compute PDE only near the isosurface –Adalsteinson et al –Whitaker et al –Peng et al Time-dependent, sparse-grid solver Introduction

Scientific Computing and Imaging Institute, University of Utah Level-Set Acceleration  Graphics Hardware (GPU) Implementations Strzodka et al –2D level-set solver on NVIDIA GeForce 2 Lefohn et al –3D level-set solver on ATI Radeon 8500 –1x – 2x faster than CPU, but 10x more computations –Unpublished work Introduction

Scientific Computing and Imaging Institute, University of Utah  GPUs Inexpensive, fast, data-parallel, streaming architecture Parallel “For-Each” call over data elements Combination of computation and visualization Scientific Computing on GPU Vertex & Texture Coordinates Vertex Processor Rasterizer Fragment Processor Texture Data Frame/Pixel Buffer(s) Introduction

Scientific Computing and Imaging Institute, University of Utah 2D computational domain Restricted, data-parallel programming model Slow GPU-to-CPU communication Limited (high-bandwidth) memory on GPU GPU Computational Capabilities Vertex & Texture Coordinates Vertex Processor Rasterizer Fragment Processor Texture Data Frame/Pixel Buffer(s) Introduction CPU

Scientific Computing and Imaging Institute, University of Utah Time-Dependent, Sparse Solver 1.2D computational domain Multi-dimensional virtual memory model 2.Restricted, data-parallel programming model Substream resolution of fragment-level conditionals 3.Slow GPU-to-CPU communication Efficient message passing algorithm 4.Limited, high-bandwidth memory on GPU Direct volume rendering of level-set solution on GPU A Streaming Narrow-Band Algorithm Algorithm

Scientific Computing and Imaging Institute, University of Utah  Virtual Memory 3D virtual memory -- Level-set computation 2D physical memory -- GPU optimizations 16 x 16 pixel memory pages -- Locality / Memory usage 1. Multi-Dimensional Virtual Memory Algorithm Virtual Memory Space Physical Memory Space Unused Pages Active Pages Inside Outside

Scientific Computing and Imaging Institute, University of Utah 1. Multi-Dimensional Virtual Memory  Cooperation between CPU and GPU CPU –Memory manager –Page table GPU –Performs level-set computation –Issues memory requests Algorithm CPU GPU Physical Addresses for Active Memory Pages Memory Requests PDE Computation passes

Scientific Computing and Imaging Institute, University of Utah 2. Static Resolution of Conditionals  Problem Neighbor lookups across page boundaries Branching slow on GPU  Solution Substreams –Create homogeneous data streams –Resolve conditionals with geometry : Points, Lines, Quads –Optimizes cache and pre-fetch performance Algorithm

Scientific Computing and Imaging Institute, University of Utah 3. Efficient Message Passing Algorithm  Problem: Time-Dependent Narrow Band GPU memory request mechanism Low bandwidth GPU-to-CPU communication  Solution Compress GPU memory request Use GPU computation to save GPU-to-CPU bandwidth Algorithm s+x-x+y-y+z-z  Mipmapping

Scientific Computing and Imaging Institute, University of Utah 4. Direct Volume Rendering of Level Set  Render from 2D physical memory Reconstruct 2D slice of virtual memory space On-the-fly on GPU Use 2D geometry and texture coordinates Algorithm

Scientific Computing and Imaging Institute, University of Utah 4. Direct Volume Rendering of Level Set  Fully general volume rendering of compressed data Tri-linear interpolation 2D slice-based volume rendering Full transfer function and lighting capabilities No data duplication Algorithm

Scientific Computing and Imaging Institute, University of Utah  Extract feature from volume  Two speed functions, F D and F H Data-based speed, F D Mean-curvature speed, F H –Smooth noisy solutions –Prevent “leaks” Segmentation Application Application F D = 0 F D (I) I (Intensity)

Scientific Computing and Imaging Institute, University of Utah Demo  Segmentation of MRI volumes scalar volume  Details ATI Radeon 9800 Pro ARB_fragment_program ARB_vertex_program 2.6 GHz Intel Xeon with 1 GB RAM Application

Scientific Computing and Imaging Institute, University of Utah Region-of-Interest Volume Rendering  Limit extent of volume rendering Use level-set segmentation to specify region Add level-set value to transfer function Application

Scientific Computing and Imaging Institute, University of Utah GPU Narrow-Band: Performance  Performance 10x – 15x faster than optimized CPU version Linear dependence on size of narrow band  Bottlenecks Fragment processor Conservative time step –Need for global accumulation register (min, max, sum, etc.) Results

Scientific Computing and Imaging Institute, University of Utah Summary  Interactive 3D Level-Set Computation/Visualization Integrated segmentation and volume rendering Intuitive parameter setting Quantified effectiveness, user study (MICCAI 2003)  Streaming Narrow-Band Solution 1. Virtual memory model 2. Substreams for static branch resolution 3. Efficient GPU-to-CPU message passing 4. Direct volume rendering of compressed/sparse data Conclusions

Scientific Computing and Imaging Institute, University of Utah Future Directions  Other level-set applications  User interface  Depth culling within active pages Sherbondy et al. talk at 3:15pm today “Fast Volume Segmentation With Simultaneous Visualization Using Programmable Graphics Hardware”  N-D GPU virtual memory system Separate memory layout from computation Conclusions

Scientific Computing and Imaging Institute, University of Utah Acknowledgements  Gordon Kindlmann –- “Teem” raster-data toolkit  Milan Ikits –- “Glew” OpenGL extension wrangler  SCI faculty, students, and staff  John Owens at UCDavis  Evan Hart, Mark Segal, Arcot Preetham, Jeff Royle, and Jason Mitchell at ATI Technologies, Inc.  Brigham and Women’s Hospital  CIVM at Duke University  Office of Naval Research grant #N  National Science Foundation grant #ACI and #CCR