Presentation is loading. Please wait.

Presentation is loading. Please wait.

CUDA Dynamic Parallelism

Published byClifford Harvey Modified over 8 years ago

Similar presentations

Presentation on theme: "CUDA Dynamic Parallelism"— Presentation transcript:

1 CUDA Dynamic Parallelism
©Jin Wang and Sudhakar Yalamanchili unless otherwise noted

2 Objective To understand the CUDA Dynamic Parallelism (CDP) execution model, including synchronization and memory model To understand the benefits of CDP in terms of productivity, workload balance and memory regularity To understand the launching path of child kernels in the microarchitectural level To understand the overhead of CDP

3 Reading CUDA Programming Guide. Appendix C “CUDA Dynamic Parallelism”.
S. Jones, “Introduction to Dynamic Parallelism”, GPU Technology Conference (presentation), Mar 2012. J. Wang and S. Yalamanchili. “Characterization and Analysis of Dynamic Parallelism in Unstructured GPU Applications.” 2014 IEEE International Symposium on Workload Characterization (IISWC). October 2014. J. Wang, A. Sidelink, N Rubin, and S. Yalamanchili, “Dynamic Thread Block Launch: A Lightweight Execution Mechanism to Support Irregular Applications on GPUs”, IEEE/ACM International Symposium on Computer Architecture (ISCA), June 2015.

4 Recap: CUDA Execution Model
Kernels/Grids are launched by host (CPU) Grids are executed on GPU Results are returned to CPU host device Kernel 1 Grid 1 Block (0, 0) Block (1, 1) Block (1, 0) Block (0, 1) Kernel 2 Grid 2

5 Launching from device Kernels/Grids can be launched by GPU host device
Block (0, 0) Block (1, 1) Block (1, 0) Block (0, 1) Grid 2

6 Code Example __global__ void k1() { } __global__ void k2() { int main() { … k1<<<1,1>>>(); … //get result of k1 if(result) { k2<<<1, 1>>>(); __global__ void k1() { if(result) k2<<<1, 1>>>(); } __global__ void k2() { int main() { … k1<<<1,1>>>(); Launching k2 from CPU Launching k2 from GPU

7 CUDA Dynamic Parallelism (CDP)
Introduced in NVIDIA Kepler GK110 Launch workload directly from GPU Launch can be nested Child Parent Finish Launch Grandchild Finish Launch

8 CDP (2) Launches are per thread Each thread can launch new kernels
Each thread can make nested launches __global__ void k1() { if(threadIdx.x == 0) k2<<<1, 1>>>(); } __global__ void k2() { int main() { … k1<<<1,32>>>(); Q1: How many k2 are launched? 32 Q2: How to change the code to launch only one k2?

9 Compilation Needs special option to compile
nvcc test.cu –o test –arch=sm_35 –rdc=true -lcudadevrt -sm_35: must be on device that have compute capability > 3.5 -rdc=true: generate relocatable device code, must be used for CDP program -lcudadevrt: link CUDA device runtime library. Can be omitted on >CUDA 6.0 Demo

10 PTX Interface The “<<< >>>” in device code will be compiled into two PTX APIs cudaGetParameterBuffer (cudaGetParameterBufferV2) cudaLaunchDevice (cudaLaunchDeviceV2) call.uni (retval0), cudaGetParameterBufferV2,( param0,param1,param2,param3); call.uni (retval0),cudaLaunchDeviceV2,( param0,param1); More on CUDA Programming Guide Appendix C

11 CDP Synchronization Child execution is not guaranteed unless synchronization Explicit Implicit __global__ void k1() { if(threadIdx.x == 0) k2<<<1, 1>>>(); } __global__ void k2() { int main() { … k1<<<1,32>>>(); Q1: Does k2 start execution immediately after it is launched? Don’t know!

12 CDP Synchronization (2)
Explicit synchronization API: cudaDeviceSynchronize() All child kernels launched in the thread block before the sync API immediately start execution If not enough GPU resources, parent kernel is suspended to yield to children Blocking parent until child kernels finish __global__ void k1() { if(threadIdx.x == 0) k2<<<1, 1>>>(); cudaDeviceSynchronize(); }

13 CDP Synchronization (3)
cudaDeviceSynchronize and __syncthreads Do NOT imply each other! __global__ void k1() { if(threadIdx.x == 0) k2<<<1, 1>>>(); //Only sync with child kernels, not other parent threads cudaDeviceSynchronize(); //still necessary for parent barrier __syncthreads(); //k2 result consumed by all parent threads consumeK2Result(); } K2 result visible to (same block) thread 0? Other threads? N N thread 0? Other threads? Y N thread 0? Other threads? Y Y

14 CDP Synchronization (3)
How about this? __global__ void k1() { if(threadIdx.x == 0) k2<<<1, 1>>>(); __syncthreads(); cudaDeviceSynchronize(); } K2 result visible to (same block) thread 0? Other threads? N N thread 0? Other threads? N N thread 0? Other threads? Y N

15 CDP Synchronization (4)
Implicit Synchronization If no explicit sync, child kernel must start execution when parent kernel execution ends Parent kernel is not finished until all child kernels are finished __global__ void k1() { if(threadIdx.x == 0) k2<<<B2, T2>>>(); otherOps(); } int main() { k1<<<B1,T1>>>(); k3<<<B3,T3>>>(); Implicit Sync Point Q: When can k3 start execution? After k1 and k2 finish.

16 CDP Memory Model Global memory: visible to both parent and child
Shared memory and local memory: private to parent/child, cannot be passed to each other. __global__ void k1(int * glmem) { __shared__ int shmem[…]; k2<<<B2,T2>>>(shmem) //undefined behavior k2<<<B2,T2>>>(glmem) //ok }

17 CDP Memory Model (2) Global memory is fully consistent when:
Child is invoked (parent->child) Child completes after cudaDeviceSynchronize is invoked (child->parent) __global__ void k1(int * glmem) { //glmem is fully consistent between k1 and k2 k2<<<B2,T2>>>(glmem) //No consistency guarantee cudaDeviceSyncrhonize(); }

18 CDP Benefits What are the issues with the following CUDA code?
Hint: control flow and memory accesses __global__ void k1(int * iterations, int * offsets, int * vals) { int it = iterations[threadIdx.x]; int offset = offsets[threadIdx.x]; for(int i = 0; i < it; i++) { val = vals[offset + i]; process(val); } Control diveregence Non-coalesced memory accesses

19 CDP Benefits (2) threadIdx.x iterations offset 3 1 4 2 5 7 12 19 21 6
3 1 4 2 5 7 12 19 21 6 11 23 34 2 3 6 1 4 5 7 19 21 34 12 23 Work load imbalance causes control flow divergence Different offsets cause non-coalesced memory accesses

20 CDP Benefits (3) Use CDP to implement
Launch a child kernel from each parent thread Launch only when sufficient parallelism Parent Kernel Child Kernels t0 t1 t2 t3 t4 t5 t6 t7 Launched by t1 Launched by t2 Launched by t3 Launched by t6 Launched by t7

21 CDP Benefits (3) Reduce control divergence
Reduce workload imbalance in parent kernel Uniform control flow in child kernels Increase coalesced memory accesses Parent Kernel Child Kernels t0 t1 t2 t3 t4 t5 t6 t7 3 4 5 6 8 9 10 11 2 1 7 12 19 21 23 Launched by t1 Launched by t2 Launched by t3 Launched by t6 Launched by t7 Uniform control flow More memory coalescing Reduced workload imbalance

22 CDP Benefits (4) Compared with other optimization to reduce control and memory divergence: Productivity Fewer lines of code __global__ void k1(int * iterations, int * offsets, int * vals) { int it = iterations[threadIdx.x]; int offset = offsets[threadIdx.x]; if(it>para_threshold) child<<<1,it>>>(vals, offset); }

23 CDP Benefits (5) Other benefits (from S. Jones’ slides)
Data-dependent execution Dynamic load balancing Dynamic library kernel calls Recursive kernel calls Simplify CPU/GPU divide

24 Recap: Host-launched Kernel Execution
Concurrent Kernel Execution Warps Warps K Kernel K K K Kernel K Kernel K Warps Warps TB TB TB TB TB TB TB TB TB Kernel Kernel

25 Device Kernel Launching
Manage Thousands of kernels Device Kernel Launch Warning: information speculated according to NVIDIA’s documentations

26 Device Kernel Launching (2)
Managed by Hardware (uArch) and Software (device driver) Pending kernel management in KMU Managing pending child kernels and suspended kernels (due to parent-child yielding) Kernel information are stored in memory Tracked by driver Warning: information speculated according to NVIDIA’s documentations

27 Device Kernel Launching (3)
New Launching path from SMX Driver (software) initializes the child kernels and stores their information in the memory SMX (hardware) notifies Kernel Management Unit Device kernel execution Same as host-launched kernels Warning: information speculated according to NVIDIA’s documentations

28 Device Kernel Launching (4)
Parent-Child Synchronization Device drive processes the synchronization request and perform necessary actions Suspend parent kernels Start execute child kernels Complete parent kernels SMX issues corresponding commands to KMU Warning: information speculated according to NVIDIA’s documentations

29 CDP Launching Overhead
Measured from CDP Program directly Device kernel launching time ~35us Recap: host kernel launching time ~10us Hardware (uArch) Extra launching path Software (device driver) Similar as driver for host-launched kernels, but executed on device Kernel preallocation Resource management No detailed breakdown Warning: information speculated according to NVIDIA’s documentations

30 CDP Launching Overhead (2)
Scales with kernel launch count 98.57ms for 256K kernel launches on K40 (VS nonCDP BFS kernel 3.27ms) 36.1%-80.6% of total execution time

31 CDP Launching Overhead (2)
Potential benefits in control flow and memory behavior However, overhead causes slow down instead of speedup From Wang and Yalamanchili, IISWC’14

32 CDP Memory Footprint Require preallocation for child kernel information ~8KB per child kernel launch Require preallocation for suspended parent status Register Shared memory ~100MB per explicit synchronization level (cudaDeviceSynchronize)

33 CDP memory footprint Reserved global memory for kernel launch
scale with kernel count Max 1.2GB reserved (K20c has 5GB global memory) From Wang and Yalamanchili, IISWC’14

34 Summary CUDA Dynamic Parallelism (CDP) supports launching kernels from the GPU Good for productivity, control flow and memory behavior (irregular applications) Supported by extended uArch and driver Non-trivial overhead overshadows the performance benefit

35 Dynamic Thread Block Launch (DTBL)
Wang, Rubin, Sidelnik and Yalamanchili, [ISCA 2015]

36 Objectives To understand DTBL execution model
To understand the microarchitectural support for DTBL To understand the benefits of DTBL

37 Example of Irregular Applications
Dynamically Formed structured pockets of Parallelism (DFP) Adaptive Mesh Refinement (AMR) Graph Traversal Pockets of Structured Parallelism 1 2 3 4 5 1 2 3 4 5 Workload Imbalance

38 Recap: CDP for Irregular Applications
Launch a child kernel from each parent thread when sufficient parallelism Parent Kernel Child Kernels t0 t1 t2 t3 t4 t5 t6 t7 Launched by t1 Launched by t2 Launched by t3 Launched by t6 Launched by t7

39 Features of DFP High Dynamic Workload Density: 1000~200,000 pockets of structured parallelism (# of dynamic kernels) High kernel launching overhead

40 Features of DFP (2) Low Compute Intensity: Average degree of parallelism in dynamic workload is ~40 (# of threads in dynamic kernels) Low GPU occupancy and utilization

41 Features of DFP (3) Workload Similarity: most dynamic workloads execute the same kernel code (with different configuration, i.e. TB size, shared memory size, etc…)

42 Dynamic Thread Block Launch (DTBL)
Motivation: Light-weight, efficient programming model and microarchitecture for DFP Propose: DTBL Extend the current GPU execution model Threads can launch fine-grained TBs on demand instead of kernels Goal: Increase SMX execution efficiency for DFP Minimize hardware and runtime overhead

43 DTBL Execution model Each thread in a kernel can launch new TBs
New TBs are coalesced with the existing kernel Coalescing: new TBs belong to the same kernel and scheduled together New Kernel 1 Kernel 1 Kernel 2 New Kernel 3 Kernel 3

44 User Interface and Code Example
Program Interface: One additional CUDA Device Runtime API cudaLaunchAggGroup __global__ void K1(…) { void * param = cudaGetParameterBuffer(…); cudaLaunchAggGroup(param, …); }

45 Revisit: Graph Traversal
1 2 3 4 5 Launch new TBs for DFP All the TBs are doing the same work Launch only when sufficient parallelism Parent Kernel Dynamically Launched TBs 1 2 3 4 5 1 2 3 4 5 Launched by t1 Launched by t2 Launched by t3 Launched by t4 Similar benefit as in CDP for control flow and memory behavior

46 Revisit: Graph Traversal
1 2 3 4 5 Launch new TBs for DFP All the TBs are doing the same work Launch only when sufficient parallelism Parent Kernel DTBL: Coalesced with a new Kernel 1 2 3 4 5 1 2 3 4 5 Launched by t1 Launched by t2 Launched by t3 Launched by t4

47 TB Coalescing and Scheduling
Achieve high SMX occupancy and utilization Recall DFP are fine-grained dynamic workloads with high density. TB coalescing allows enough TBs to be aggregated into one big kernel and scheduled to SMXs. In comparison: CDP only allows 32 concurrent kernel execution. SMXs are not fully occupied for small kernels.

48 Examples for TB coalescing
One TB has 64 threads or 2 warps (in CDP, that is one kernel) On K20c GK110 GPU: 13 SMX, max 64 warps/SMX, max 32 concurrent kernels CDP: All child kernels …… Kernel 1 Kernel 2 Kernel 3 Kernel 100 SMXs Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory …… SMX0 SMX1 SMX12

49 Examples for TB coalescing
One TB has 64 threads or 2 warps (in CDP, that is one kernel) On K20c GK110 GPU: 13 SMX, max 64 warps/SMX, max 32 concurrent kernels CDP: All child kernels …… Kernel 2 Kernel 3 Kernel 100 SMXs Kernel 1 Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory …… SMX0 SMX1 SMX12

50 Examples for TB coalescing
One TB has 64 threads or 2 warps (in CDP, that is one kernel) On K20c GK110 GPU: 13 SMX, max 64 warps/SMX, max 32 concurrent kernels CDP: All child kernels …… Kernel 3 Kernel 100 SMXs Kernel 1 Kernel 2 Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory …… SMX0 SMX1 SMX12

51 Examples for TB coalescing
One TB has 64 threads or 2 warps (in CDP, that is one kernel) On K20c GK110 GPU: 13 SMX, max 64 warps/SMX, max 32 concurrent kernels CDP: All child kernels …… Kernel 14 Kernel 100 SMXs Kernel 1 Kernel 2 Kernel 13 Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory …… SMX0 SMX1 SMX12

52 Examples for TB coalescing
One TB has 64 threads or 2 warps (in CDP, that is one kernel) On K20c GK110 GPU: 13 SMX, max 64 warps/SMX, max 32 concurrent kernels CDP: All child kernels …… Kernel 15 Kernel 100 SMXs Kernel 14 Kernel 1 Kernel 2 Kernel 13 Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory …… SMX0 SMX1 SMX12

53 Examples for TB coalescing
One TB has 64 threads or 2 warps (in CDP, that is one kernel) On K20c GK110 GPU: 13 SMX, max 64 warps/SMX, max 32 concurrent kernels CDP: All child kernels Wait till previous kernel finish …… Kernel 33 Kernel 100 SMXs Kernel 27 Kernel 28 Kernel 32 Kernel 14 Kernel 15 Kernel 26 Kernel 1 Kernel 2 Kernel 13 Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory …… SMX0 SMX1 SMX12

54 Examples for TB coalescing
One TB has 64 threads or 2 warps (in CDP, that is one kernel) On K20c GK110 GPU: 13 SMX, max 64 warps/SMX, max 32 concurrent kernels Achieved SMX Occupancy: 32 kernels * 2 warps/kernel / (64 warps/SMX * 13 SMXs) = 0.077 CDP: All child kernels Wait till previous kernel finish …… Kernel 33 Kernel 100 SMXs Kernel 27 Kernel 28 Kernel 32 Kernel 14 Kernel 15 Kernel 26 Kernel 1 Kernel 2 Kernel 13 Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory …… SMX0 SMX1 SMX12

55 Examples for TB coalescing
One TB has 64 threads or 2 warps (in CDP, that is one kernel) On K20c GK110 GPU: 13 SMX, max 64 warps/SMX, max 32 concurrent kernels DTBL: All TBs coalesced with one kernel. No restrictions in scheduling. …… TB 1 TB 2 TB 3 TB 100 SMXs Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory …… SMX0 SMX1 SMX12

56 Examples for TB coalescing
One TB has 64 threads or 2 warps (in CDP, that is one kernel) On K20c GK110 GPU: 13 SMX, max 64 warps/SMX, max 32 concurrent kernels DTBL: All TBs coalesced with one kernel. No restrictions in scheduling. SMXs TB 92 TB 93 TB 100 Kernel 27 Kernel 28 TB 91 Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory …… SMX0 SMX1 SMX12

57 Examples for TB coalescing
One TB has 64 threads or 2 warps (in CDP, that is one kernel) On K20c GK110 GPU: 13 SMX, max 64 warps/SMX, max 32 concurrent kernels Achieved SMX Occupancy: 100 TBs * 2 warps/TB / (64 warps/SMX * 13 SMXs) = 0.24 Can be increased if more TBs! DTBL: All TBs coalesced to one kernel. No restrictions in scheduling. SMXs TB 92 TB 93 TB 100 Kernel 27 Kernel 28 TB 91 Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory Register File Cores L1 Cache/Shared Memory …… SMX0 SMX1 SMX12

58 Microarchitecture (Baseline GPU)

59 Microarchitecture Extension
Tracking new TBs Coalescing and scheduling new TBs New TB launching information

60 TB Coalescing on Microarchitecture
K K K K Coalescing Newly launched TBs TB TB TB TB

61 TB Coalescing on Microarchitecture
K Scheduled together to increase SMX Occupancy TB TB TB TB TB TB TB TB

62 TB Launching Latency Exclusive launching path from SMXs
Simple runtime and driver support with low latency Kernel information only has to be managed once New TBs can reuse the kernel information In comparison, CDP uses Longer launching path More complicated runtime and driver for kernel management

63 Launch Path for CDP and DTBL
Manage Thousands of kernels Shorter path to KDE If launched as kernel TB TB TB TB

64 Launch Path for CDP and DTBL
Launch path for CDP: ~70k cycles Launch path for DTBL: ~10k cycles (32 launches per warp) Shorter path than to KMU If launched as kernel TB TB TB TB

65 Speedup of DTBL Ideal: excluding launch latency, 1.63x over flat implementations and 1.14x over CDP (Benefits from Higher GPU Utilization) Actual: including launch latency, 1.21x over flat implementations and 1.40x over CDP (Benefits from less launching overhead)

66 Summary DTBL: New extension to GPU execution Model
Efficient solution for Dynamic Parallelism in irregular applications Shows more benefit for dynamic, fine-grained but sufficient parallelism Dynamic launching latency is reduced from CDP

Download ppt "CUDA Dynamic Parallelism"

Similar presentations

Intermediate GPGPU Programming in CUDA

Intermediate GPGPU Programming in CUDA
Scalable Multi-Cache Simulation Using GPUs Michael Moeng Sangyeun Cho Rami Melhem University of Pittsburgh.

Scalable Multi-Cache Simulation Using GPUs Michael Moeng Sangyeun Cho Rami Melhem University of Pittsburgh.
Optimization on Kepler Zehuan Wang

Optimization on Kepler Zehuan Wang
Latency considerations of depth-first GPU ray tracing

Latency considerations of depth-first GPU ray tracing
1 ITCS 4/5010 CUDA Programming, UNC-Charlotte, B. Wilkinson, Feb 26, 2013, DyanmicParallelism.ppt CUDA Dynamic Parallelism These notes will outline CUDA.

1 ITCS 4/5010 CUDA Programming, UNC-Charlotte, B. Wilkinson, Feb 26, 2013, DyanmicParallelism.ppt CUDA Dynamic Parallelism These notes will outline CUDA.
Acceleration of the Smith– Waterman algorithm using single and multiple graphics processors Author : Ali Khajeh-Saeed, Stephen Poole, J. Blair Perot. Publisher:

Acceleration of the Smith– Waterman algorithm using single and multiple graphics processors Author : Ali Khajeh-Saeed, Stephen Poole, J. Blair Perot. Publisher:
L15: Review for Midterm. Administrative Project proposals due today at 5PM (hard deadline) – handin cs6963 prop March 31, MIDTERM in class L15: Review.

L15: Review for Midterm. Administrative Project proposals due today at 5PM (hard deadline) – handin cs6963 prop March 31, MIDTERM in class L15: Review.
L13: Review for Midterm. Administrative Project proposals due Friday at 5PM (hard deadline) No makeup class Friday! March 23, Guest Lecture Austin Robison,

L13: Review for Midterm. Administrative Project proposals due Friday at 5PM (hard deadline) No makeup class Friday! March 23, Guest Lecture Austin Robison,
Based on Silberschatz, Galvin and Gagne  2009 Threads Definition and motivation Multithreading Models Threading Issues Examples.

Based on Silberschatz, Galvin and Gagne  2009 Threads Definition and motivation Multithreading Models Threading Issues Examples.
CUDA Programming Lei Zhou, Yafeng Yin, Yanzhi Ren, Hong Man, Yingying Chen.

CUDA Programming Lei Zhou, Yafeng Yin, Yanzhi Ren, Hong Man, Yingying Chen.
Task Based Execution of GPU Applications with Dynamic Data Dependencies Mehmet E Belviranli Chih H Chou Laxmi N Bhuyan Rajiv Gupta.

Task Based Execution of GPU Applications with Dynamic Data Dependencies Mehmet E Belviranli Chih H Chou Laxmi N Bhuyan Rajiv Gupta.
Big Kernel: High Performance CPU-GPU Communication Pipelining for Big Data style Applications Sajitha Naduvil-Vadukootu CSC 8530 (Parallel Algorithms)

Big Kernel: High Performance CPU-GPU Communication Pipelining for Big Data style Applications Sajitha Naduvil-Vadukootu CSC 8530 (Parallel Algorithms)
To GPU Synchronize or Not GPU Synchronize? Wu-chun Feng and Shucai Xiao Department of Computer Science, Department of Electrical and Computer Engineering,

To GPU Synchronize or Not GPU Synchronize? Wu-chun Feng and Shucai Xiao Department of Computer Science, Department of Electrical and Computer Engineering,
An Introduction to Programming with CUDA Paul Richmond

An Introduction to Programming with CUDA Paul Richmond
Rahul Sharma (Stanford) Michael Bauer (NVIDIA Research) Alex Aiken (Stanford) Verification of Producer-Consumer Synchronization in GPU Programs June 15,

Rahul Sharma (Stanford) Michael Bauer (NVIDIA Research) Alex Aiken (Stanford) Verification of Producer-Consumer Synchronization in GPU Programs June 15,
CuMAPz: A Tool to Analyze Memory Access Patterns in CUDA

CuMAPz: A Tool to Analyze Memory Access Patterns in CUDA
Martin Kruliš 4 11. 2014 by Martin Kruliš (v1.0)1.

Martin Kruliš by Martin Kruliš (v1.0)1.
Extracted directly from:

Extracted directly from:
CUDA 5.0 By Peter Holvenstot CS6260. CUDA 5.0 Latest iteration of CUDA toolkit Requires Compute Capability 3.0 Compatible Kepler cards being installed.

CUDA 5.0 By Peter Holvenstot CS6260. CUDA 5.0 Latest iteration of CUDA toolkit Requires Compute Capability 3.0 Compatible Kepler cards being installed.
© David Kirk/NVIDIA and Wen-mei W. Hwu Taiwan, June 30-July 2, 2008 1 Taiwan 2008 CUDA Course Programming Massively Parallel Processors: the CUDA experience.

© David Kirk/NVIDIA and Wen-mei W. Hwu Taiwan, June 30-July 2, Taiwan 2008 CUDA Course Programming Massively Parallel Processors: the CUDA experience.

Similar presentations

Ads by Google