Jeremy Meredith Lawrence Livermore National Laboratory UCRL-PRES This work was performed under the auspices of the U.S. Department of Energy by the University of California, Lawrence Livermore National Laboratory under contract No. W-7405-Eng-48. The GAIA Project: The GAIA Project: Evaluation of GPU-Based Programming Environments for Knowledge Discovery David Bremer, Lawrence Flath, John Johnson, Holger Jones, Sheila Vaidya, Randall Frank*

2 Motivation Trends in the graphics marketplace Trends in the graphics marketplace Inherent parallelism of graphics tasksInherent parallelism of graphics tasks Performance increasing faster than for CPUsPerformance increasing faster than for CPUs Move to programmable hardwareMove to programmable hardware Effects of mass marketsEffects of mass markets Not expected to end anytime soon… Not expected to end anytime soon… Today: 40GF, 2GB/s I/O, 30GB/s memoryToday: 40GF, 2GB/s I/O, 30GB/s memory 2006: 100GF, 8GB/s I/O, 60GB/s memory2006: 100GF, 8GB/s I/O, 60GB/s memory 2007: 1TF…2007: 1TF…

3 The NV40 and the Sony Playstation 3 Are graphics trends a glimpse of the future? Are graphics trends a glimpse of the future? The nVidia NV40 Architecture The nVidia NV40 Architecture 256MB+ RAM256MB+ RAM bit IEEE FP 400Mhz128 32bit IEEE FP 400Mhz 220M transistors, 110W of power220M transistors, 110W of power The PlayStation3 (patent application) The PlayStation3 (patent application) Core component is a cellCore component is a cell 1 “PowerPC” CPU + 8 APUs (“vectorial” processors) 1 “PowerPC” CPU + 8 APUs (“vectorial” processors) 4GHz, 128K RAM, 256GFLOP/cell 4GHz, 128K RAM, 256GFLOP/cell Multiple cells (Phone, PDA, PS3, …)Multiple cells (Phone, PDA, PS3, …) Four cell architecture (1TFLOP) Four cell architecture (1TFLOP) Central 64MB memory Central 64MB memory Keys Keys Streaming data modelsStreaming data models Cache-driven/cache-oblivious computingCache-driven/cache-oblivious computing nVidia NV30 nVidia NV40

4 Data representations for GPUs Programmable FP SIMD engines, GF today, 1TF by ’06 Programmable FP SIMD engines, GF today, 1TF by ’06 Where can they be exploited? Where can they be exploited? Many advantages for the data pipelineMany advantages for the data pipeline Data/algorithmic design challengesData/algorithmic design challenges Possible applicability for simulationPossible applicability for simulation Many current research projects on scientific computing, databases, audio processingMany current research projects on scientific computing, databases, audio processing Current projects Current projects Programmable rendering pipelineProgrammable rendering pipeline Multi-variate, interactive Multi-variate, interactive Increased graphics precision Increased graphics precision Image composition pipelineImage composition pipeline Implementation of physics based renderingImplementation of physics based rendering Simulated radiography, diffraction computation Simulated radiography, diffraction computation Large image geo-registrationLarge image geo-registration 100x performance improvement over CPU 100x performance improvement over CPU Texture RAM Vertex Program Volume A Volume B GPU Fragment Program

5 Specific Project Goals Investigate use of COTS technologies for computation Investigate use of COTS technologies for computation “Non-traditional” applications“Non-traditional” applications Image and speech Image and speech String, statistical, graph… String, statistical, graph… Mechanisms necessary for exploitationMechanisms necessary for exploitation Data infrastructure (e.g. cache coherent streaming…) Data infrastructure (e.g. cache coherent streaming…) Software abstractions Software abstractions Delineate some boundary conditions on their useDelineate some boundary conditions on their use Evaluation vs CPU based solutions Evaluation vs CPU based solutions Parameter-space investigation Parameter-space investigation

6 Data Infrastructure Forms the basis of a comparative framework Forms the basis of a comparative framework Support both GPU and CPU algorithmic implementationsSupport both GPU and CPU algorithmic implementations Targets multiple platformsTargets multiple platforms Provides data abstractionProvides data abstraction “Tile-based” streaming “Tile-based” streaming Cache coherency control Cache coherency control CPU to GPU to CPU glue layer CPU to GPU to CPU glue layer Utilizes higher-level languages for algorithmsUtilizes higher-level languages for algorithms Cg, Brook, GLSL, etc Cg, Brook, GLSL, etc

7 Image Processing Applications Common attributes Common attributes Large, streaming imagery on a single gfx cardLarge, streaming imagery on a single gfx card Parallel 1D and 2D applicationsParallel 1D and 2D applications Multi-spectral (four, possibly temporal channels)Multi-spectral (four, possibly temporal channels) Discrete convolution Discrete convolution Arbitrary kernelsArbitrary kernels Correlation Correlation Separate threshold, search, and detection phase includedSeparate threshold, search, and detection phase included

8 String Processing Applications Representation and bandwidth characteristics Representation and bandwidth characteristics String comparison String comparison “Bulk” comparison operations individual outputs“Bulk” comparison operations individual outputs String sorting String sorting Based on string comparisonBased on string comparison Batched sort based on radix algorithmsBatched sort based on radix algorithms String searching String searching “Wildcard” pattern matching“Wildcard” pattern matching Sort-based element searchSort-based element search

9 Other Application Targets Image transforms Image transforms FFT, WaveletFFT, Wavelet Many application domainsMany application domains Statistical functions on images Statistical functions on images Moments, regression (general linear model)Moments, regression (general linear model) Hypothesis/model driven image processing, texture characterization, etcHypothesis/model driven image processing, texture characterization, etc Hidden Markov ModelsHidden Markov Models Graph search Graph search Structured (fully connected) or unstructured graphs, detect and return lowest cost pathStructured (fully connected) or unstructured graphs, detect and return lowest cost path Many application domainsMany application domains

10 System Targets Constrained system targets based on resource limits Constrained system targets based on resource limits Hardware targets Hardware targets nVidia: NV3x, NV4x, NV5xnVidia: NV3x, NV4x, NV5x Focus on NV4x due to new branching capabilities Focus on NV4x due to new branching capabilities Dual CPU IA32 platform Dual CPU IA32 platform PCI-Express (PCIe) enhanced readback and async bandwidth PCI-Express (PCIe) enhanced readback and async bandwidth BG/L and MerrimacBG/L and Merrimac OS targets OS targets Primarily Linux, some Windows due to driver issuesPrimarily Linux, some Windows due to driver issues Language targets Language targets nVidia Cg, BrooknVidia Cg, Brook

11 Convolution Timing Results All timings count download, render, and readback All timings count download, render, and readback First render pass is excluded from the count First render pass is excluded from the count Overhead to load shader can be substantial Overhead to load shader can be substantial

12 Convolution Timing Results Software vs. two-texture hardware implementation Software vs. two-texture hardware implementation At all but the smallest kernel sizes, GPUs are much faster At all but the smallest kernel sizes, GPUs are much faster

13 Convolution Timing Results Software vs. two-texture hardware implementation Software vs. two-texture hardware implementation 32-bit textures use more memory bandwidth 32-bit textures use more memory bandwidth

14 Convolution Timing Results Two-texture vs. procedural hardware implementations Two-texture vs. procedural hardware implementations Two-texture implementation requires more memory bandwidth Two-texture implementation requires more memory bandwidth

15 Double Precision Port of David Bailey’s single-double Fortran library* to NVidia’s Cg language Port of David Bailey’s single-double Fortran library* to NVidia’s Cg language Can emulate double precision Can emulate double precision Use two single-precision floats Use two single-precision floats High order float is estimate to the double; Low order float is error of that estimate High order float is estimate to the double; Low order float is error of that estimate Resulting precision is almost double Resulting precision is almost double The exponent remains at single range available at htpp://crd.lbl.gov/~dhbailey/mpdist The exponent remains at single range available at htpp://crd.lbl.gov/~dhbailey/mpdist

16 Double Precision Results One Convolution Pass, Single vs Double Precision 32-bit Texture Size Convolution with single and emulated-double arithmetic Convolution with single and emulated-double arithmetic Double precision only 1.5x slower than single precision at the same texture depth Double precision only 1.5x slower than single precision at the same texture depth

17 Future Plans Obtain results for a variety of algorithms including strings, HMMs, and FFTs Obtain results for a variety of algorithms including strings, HMMs, and FFTs Include performance and accuracy Include performance and accuracy Extend to new architectures as available (e.g. Merrimac) Extend to new architectures as available (e.g. Merrimac) Explore other high-level languages (e.g. brook implementations and other streaming languages) Explore other high-level languages (e.g. brook implementations and other streaming languages) Launch a benchmarking web site: Launch a benchmarking web site: