1. Lecture 1: Introduction
CS179: GPU Programming
Lecture 1: Introduction

2. Today Course summary Administrative details
Brief history of GPU computing
Introduction to CUDA

3. Course Summary GPU Programming What: Why:
GPU: Graphics processing unit -- highly parallel
APIs for accelerated hardware
Why:
Parallel processing

4. Course Summary: Why GPU?

5. Course Summary: Why GPU?
How many cores, exactly?
GeForce 8800 Ultra (2007) - 128
GeForce GTX 260 (2008) - 192
GeForce GTX 295 (2009) - 480*
GeForce GTX 480 (2010) - 480
GeForce GTX 590 (2011) *
GeForce GTX 690 (2012) *
GeForce GTX Titan Z (2014) *
* indicates these are cards shipped with 2 GPUs in them, effectively doubling the cores

6. Course Summary: Why GPU?

7. Course Summary: Why GPU?

8. Course Summary: Why GPU?

9. Course Summary: Why GPU?
What kinds of speedups do we get?

10. Course Summary: Overview
What will you learn?
CUDA
Parallelizing problems
Optimizing GPU code
CUDA libraries
What will we not cover?
OpenGL
C/C++

11. Administrative: Course Details
CS179: GPU Programming
Website:
Course Instructors/TA’s:
Connor DeFanti
Kevin Yuh
Overseeing Instructor:
Al Barr
Class time:
MWF 5:00-5:55PM

12. Administrative: Assignments
Homework:
8 assignments
Each worth 10% of your grade (100 pts. each)
Final Project:
2 weeks for a custom final project
Details are up to you!
20% of your grade (200 pts.)

13. Administrative: Assignments
Assignments will be due Wednesday, 5PM
Extensions may be granted…
Talk to TA’s beforehand!
Office Hours: located in 104 ANB
Connor: Tuesday, 8-10PM
Kevin: Monday, 8-10PM

14. Administrative: Assignments
Doing the assignments:
CUDA-capable machine required!
Must have NVIDIA GPU
Setting up environment can be tricky
Three options:
DIY with your own setup
Use provided instructions with given environment
Use lab machines

15. Administrative: Assignments
Submitting assignments:
Due date: Wednesday 5PM
Submit assignment as .tar/.zip, or similar
Include README file!
Name, compilation instructions, answers to conceptual questions on sets, etc.
Submit all assignments to
Receiving graded assignments:
Assignments should get back 1 week after submission
We will you back with grade and comments

16. GPU History: Early Days
Before GPUs:
All graphics run on the CPU
Each pixel drawn in series
Super slow! (CS171, anyone?)
Early GPUs:
1980s: Blitters (fixed image sprites) allowed fast image memory transfer
1990s: Introduction of DirectX and OpenGL
Brought fixed function pipeline for rendering

17. GPU History: Early Days
Fixed Function Pipeline:
“Fixed” OpenGL states
Phong or Gouraud shading?
Render as wireframe or solid?
Very limiting, made early games look similar

18. GPU History: Shaders Early 2000’s: shaders introduced
Allow for much more interesting shading models

19. GPU History: Shaders Shaders: expanded world of rendering greatly
Vertex shaders: apply operations per-vertex
Fragment shaders: apply operations per-pixel
Geometry shaders: apply operations to add new geometry

20. GPU History: Shaders These are great when dealing with graphics data…
Vertices, faces, pixels, etc.
What about general purpose?
Can trick GPU
DirectX “compute” shader may be an option
Anything slicker?

21. GPU History: CUDA 2007: NVIDIA introduces CUDA
C-style programming API for GPU
Easier to do GPGPU
Easier memory handling
Better tools, libraries, etc.

22. GPU History: CUDA New advantages on the table: Scattered reads
Shared memory
Faster memory transfer to/from the GPU

23. GPU History: Other APIs
Plenty of other API’s exist for GPGPU
OpenCL/WebCL
DirectX Compute Shader
Other

24. Using the GPU
Highly parallelizable parts of computational problems

25. A simple problem… Add two arrays On the CPU: A[] + B[] -> C[]
(allocate memory for C)
For (i from 1 to array length)
C[i] <- A[i] + B[i]
Operates sequentially… can we do better?

26. A simple problem… On the CPU (multi-threaded):
(allocate memory for C)
Create # of threads equal to number of cores on processor (around 2, 4, perhaps 8)
(Allocate portions of A, B, C to each thread...)
...
In each thread,
For (i from beginning region of thread)
C[i] <- A[i] + B[i]
//lots of waiting involved for memory reads, writes, ...
Wait for threads to synchronize...
Slightly faster – 2-8x (slightly more with other tricks)

27. A simple problem… How many threads? How does performance scale?
Context switching: High penalty on the CPU!

28. A simple problem… On the GPU: Speedup: Very high! (e.g. 10x, 100x)
(allocate memory for A, B, C on GPU)
Create the “kernel” – each thread will perform one (or a few) additions
Specify the following kernel operation:
For (all i‘s assigned to this thread)
C[i] <- A[i] + B[i]
Start ~20000 (!) threads
Wait for threads to synchronize...
Speedup: Very high! (e.g. 10x, 100x)

29. GPU: Strengths Revealed
Parallelism
Low context switch penalty!
We can “cover up” performance loss by creating more threads!

30. GPU Computing: Step by Step
Setup inputs on the host (CPU-accessible memory)
Allocate memory for inputs on the GPU
Copy inputs from host to GPU
Allocate memory for outputs on the host
Allocate memory for outputs on the GPU
Start GPU kernel
Copy output from GPU to host
(Copying can be asynchronous)

31. GPU: Internals Blocks: Groups of threads
Can cooperate via shared memory
Can synchronize with each other
Max size: 512, 1024 threads (hardware-dependent)
Warps: Subgroups of threads within block
Execute “in-step”
Size: 32 threads

32. GPU: Internals
Block
SIMD processing unit
Warp

33. The Kernel Our “parallel” function
Simple implementation (won’t work for lots of values)

34. Indexing Can get a block ID and thread ID within the block:
Unique thread ID!

35. Calling the Kernel …

36. Calling the Kernel (2)