1. Lecture 13: OpenGL Shading Language (GLSL)
COMP 175: Computer Graphics
April 18, 2018

2. Motivation
Last week, we discussed the many of the new “tricks” in Graphics require low-level access to the Graphics pipeline
For example, using textures, which is a pain in regular OpenGL.

3. Programmable Pipeline (~2001) CUDA/OpenCL (GPGPU) (~2008)
Graphics History
Software Rendering
Old-fashioned way (Use ray casting)
Fixed-Function
Immediate Mode (What we have used)
Programmable Pipeline (~2001)
GL Shading Language (What we will learn)
CUDA/OpenCL (GPGPU) (~2008)
General Purpose GPU Programming

4. Ray Casting (Software Rendering)
Wolfenstein
Quake 1

5. Fixed-Function
Quake 3
Half-Life

6. Programmable Pipeline (Shaders)
Doom 3
Far Cry

7. General Purpose Graphics Processing Unit
GPGPU
General Purpose Graphics Processing Unit
Utilize graphics card for computation
Do work other than graphics

8. Old Fixed-Function Not to exciting, what we’ve been using
A bunch of glEnable, glEnable

9. Programmable Pipeline
iPhone intentional, because you have to use shaders to do graphics for openGL.

10. Why GPU? Having two (CPU and GPU) is better than one…
GPU is dedicated to floating point operations (a traditionally expensive operation on a CPU)
It is especially optimized for linear algebra (e.g. matrix manipulation)!
GPU is multi-core (multi-processor)
Imagine that you have one processor per pixel, how would you program what the processor should do?
GPU is now optimized to work with textures
Modern GPUs come with large amounts of memory, almost comparable to CPUs

13. Floating Point Operations (FLOPs)

14. Versions of OpenGL and GLSL (and why you need to care)
Confusions galore…

15. Many versions of OpenGL
OpenGL 2.0 (2004) introduces the idea of shaders
OpenGL SL 1.0 (or 1.1) is a part of OpenGL 2.0
Currently, OpenGL is at 4.6 (2017). GLSL is also at 4.6
When OpenGL 3.1 was introduced, GLSL changed versioning to match (to 3.1), which was previous 1.3.1
3.0 -> 3.1 (2009) is said to be the “big move”
where fixed pipeline is considered deprecated.
Mac does split support (only 3.0+ or only <3.0). Mixed syntax is not allowed

16. OpenGL ES 2.0 is not backward compatible with previous versions
OpenGL ES (ES = Embedded Systems) is meant for smart phones, devices, etc.
OpenGL ES 1.0 (2003) implemented OpenGL 1.3
OpenGL ES 2.0 (2007) is a reduced set of OpenGL 2.0
But ES 2.0 “jumped ahead” and eliminated almost all fixed-pipeline operations (e.g. glBegin, glEnd)
OpenGL ES 2.0 is not backward compatible with previous versions
Current version is ES 3.2 (2015)

17. WebGL is OpenGL for browsers, in particular HTML5 canvas element.
WebGL 1.0 is based on OpenGL ES 2.0 (2011)
Current version is WebGL 2.0 (2017)
based on OpenGL ES 3.0
Largely backward compatible

18. CUDA / OpenCL Made for general purpose graphics programming (GPGPU)
Popular known use is bitcoin mining
CUDA is a language specific to nVidia (2007)
Obviously different from GLSL in that there are no graphics related keywords or concepts. But the general idea (of parallel computing) is similar
OpenCL is an open standard version (2009) of CUDA led by the Khronos Group
The Khronos Group is a non-profit consortium founded by major graphics companies
Currently Khronos is in charge of the OpenGL, OpenCL, OpenGL ES, WebGL, WebCL, and a bunch of related APIs.

19. Moving Forward in this Class…
Mac is a pain… It only supports up to OpenGL 4.1
Check your computer:
Mac is a pain… It has stopped supporting OpenGL as of June and now only supports their proprietary graphics language called “Metal”.
As a result, there are a lot of warnings when compiling OpenGL
Mac is a pain… The use of either gl.h or gl3.h means that there’s no backward compatibility
When programming in gl3.h, any fixed function call results in an error (e.g. glPushMatrix(), or glRotate3f(), etc.)
Windows is a pain… There are many graphics card vendors
… the graphics card vendors don’t support all aspects of OpenGL equally
As a result, the use of the GLEW library is almost strictly required

20. GLSL 1.2 vs. 3.3+
There’s a rather large departure between GLSL 1.2 and 3.3
GLSL 1.2 is easier to learn and get a program running.
However, it doesn’t work for GL 3.0 (so Mac is a problem because of its split support)
GLSL 3.3+ is more “clean”, but we need to learn about Vertex Buffer Object (VBO).
We’ll discuss later in passing… It’s a lot of:
Memory packing and alignment into the correct format for GLSL
Magic incantation to run it
Because of the differences in syntax, you need to specify the shader version in your GLSL code!

21. Quick Demo (of Lab 9)

22. Accessing Programmable Pipeline
Introducing Shaders!

23. CPU + GPU CPU: Typical C++ code GPU: OpenGL Shading Language (GLSL)
CPU passes data to GPU
Some processing happens on the CPU:
Get mouse movement
Load and parse data from disk
Some processing happens on the GPU:
Transform point from Object to World space
Solve lighting equation
Render

24. CPU + GPU (Continued)
Think of this as two separate processors. What we’ll learn today are:
What the GPU should do (and what the CPU should do).
In particular different “shaders” to fit into the different parts of the graphics pipeline
How to communicate between CPU and GPU
Sharing “chunks of memory”
Passing variable names and values
(and how to pass variables between different shaders)
The “language” used by the GPU (i.e. OpenGL Shading Language, or OpenGL SL, or GLSL).

25. Types of Shaders Most common: Newer: Vertex Fragment (Pixel) Geometry
Tessellation
Compute

26. Two Things to Learn: OpenGL Shading Language (GLSL)
Syntax
Sequence and processes
Vertex Shader
Fragment (Pixel) Shader
Communication between CPU and GPU
What type of information can be passed
How to pass information
Note that you also need to know how to load a shader program, but that’s “magic incantation” that (mostly) stays the same every time.

27. Interfacing with the GPU
Modern OpenGL
Mobile/Web Programming
Direct3D
Compute Languages
Stream, OpenCL, CUDA
Future
The goal of todays lecture is to show you what you can do to get ready to move onto the next tier of graphics.
We’ve covered a lot, most graphics courses end after ray tracer
You already have all the pieces you need to make great applications.

28. Client/Server Architecture
Think of like a network
Client = the CPU
Server = the GPU
The less they talk, the better
Less waiting for memory to transfer
Bandwidth is almost always the bottleneck
Send data over once, and have it ready to go
We’re going to see a common theme here, and need some vocabulary.
If we want to make things faster, we need to pass data around less.
Our goal is to minimize the number of openGL calls. Less calls means better performance.
This is the same model being used, but in the documentation and command names you will start seeing the word client and server.

29. Immediate Mode (Fixed-Function pipeline)
Every ‘gl’ command is being sent to the GPU every iteration.
Very inefficient
Think of it like having an interpreter versus a compiler for C++
Michael Shah

30. Fixed Function (Left) vs. Programmable (Right)

31. Optimizations Available
Vertex Array
Display Lists
Vertex Buffer Objects
Pixel Buffers
Frame Buffer Objects
All of these optimizations we’re going to see involve chunking a bunch of data together so that we can pass it between the GPU and CPU more efficiently.
Michael Shah

32. Minimizes OpenGL calls on Client (CPU)
Vertex Array
No more glBegin/glEnd
Store Vertices, indices, normals, colors, texture coordinates in an array
Minimizes OpenGL calls on Client (CPU)
Data still being passed from Client to Server every frame
A little better because it’s all at once
Stylistically, much cleaner code as well
Michael Shah

33. Send it to the Server(GPU) once, and it is stored in memory on Server
4/22/2014
Display List
Compiled geometry
Send it to the Server(GPU) once, and it is stored in memory on Server
Geometry cannot be updated, without deleting display list.
someGeometryDisplayList = glGenLists(1);
glNewList(someGeometryDisplayList,GL_COMPILE);
drawShape();
glEndList();
Once its on the server you cannot get it back, just like when you upload a picture on the internet
Can usually get an order of magnitude of performance improvement
Michael Shah

34. Vertex Buffer Object (VBO)
4/22/2014
Vertex Buffer Object (VBO)
Improves upon Vertex Array
We can store data on the Server (GPU)
Just like display lists!
We get a pointer into the Server (GPU) so that we can update it!
As good as display lists!
Because we have a pointer to the data, multiple clients can access data.
Multiple instances of a program can access data
Michael Shah

35. Modifiers to Variables Uniform/Varying/Attribute
Uniform – Read only data passed from CPU into GPU (vertex/fragment shaders)
Depending on GLSL version:
GLSL <=1.2: Attribues & Varying – Pass data from vertex to fragment shader.
GLSL >1.2 in / out – in is the variable being passed in, out is the variable being passed out
Read-only in fragment shader.
Read/Write in Vertex Shader
Attributes – Input value which change every vertex.
Read only – Used only in vertex shader
Varying – Variables passed from one shader to another
Uniform – Example pass in a texture
Attribute – Passing in different vertices or normals for every vertex, only used in the vertex shader

36. Order matters… Info is only passed in one direction (starting with vertex shader)
Note that we can run this loop multiple times (e.g. for shadow, or achieve specific shading effects (such as toon-shading))
4 types of shaders available to us.
We will cover 2, and the other newer ones we will not.
Part of this is because the new shaders require graphics cards often not found in laptops.
I encourage you to look into these features for your final projects however.
Verex, followd by tessellation, then geometry both dealing with geometry.
Finally fragment shader
Reference:

37. OpenGL Shading Language

38. Debugging – Still as difficult Domain Specific Language
GLSL
C-Like Language
Made for GPU Architecture
Because GPU’s are different than CPU’s
Still Cross Platform
Debugging – Still as difficult
Domain Specific Language
Compiled at runtime!!!!!
There is an equivalent for Microsoft called HLSL (High Level Shading Language)

39. GLSL Primitives float, int, bool Example: float f = 10.0;or
float f = float(10);
(Casting not allowed, choose your type wisely!)
No doubles in GLSL, they’re too slow for the architecture!
Later versions of GLSL and CUDA get doubles however

40. GLSL Types Vector: vec2, vec3, vec4 Matrix: mat2, mat3, mat4
Overloaded to take in each type
Example: vec4 temp= vec4(1.0, 1.0, 1.0, 1.0)
Matrix: mat2, mat3, mat4
Example: glModelViewMatrix is a mat4 type
Access first column with: mat[0]
Access second column with: mat[1]
Example: mat[1] = vec3(1.0, 1.0, 1.0);
Access individual element: mat[2][1];
Vectors just like in our Algebra.h file

41. Get components of vector
Some more examples
Pass in vec3 into a vec4
vec3 color = vec3(1.0,0.5,0.25)
vec4 temp = vec4(color, 1.0)
Get components of vector
color.x = 1.0; or color[0] = 1.0
color.y = 1.0; or color[1] = 1.0;

42. Swizzling [1]
Swizzling is the ability to rearrange a vectors components
vec2 pos = temp.xy; // retrieve x and y component, or any of xyzw
vec2 pos = temp.yz;
vec4 = color.agbr; // opposite of rgba

43. More GLSL Keywords [1][2]
Your cheat sheet to look at later to see what items you can access.
More datatypes available as you get into later versions of GLSL
Note that some of these default keywords might not be there depending on GLSL version number (for example: gl_ModelViewMatrix isn’t there in 3.0+)

44. Write a Simple Shader
Basic Vertex and Fragment Shader

45. Writing a shader, compiling a shader, and loading a shader
Compile at runtime
So we do not need to recompile entire source, if we only modify shader.

47. Simple Vertex Shader

48. Simple Fragment Shader

49. The cpp file

50. Drawing with immediate mode for now
For GLSL 3.0+, we will need to use a Vertex Buffer Object (VBO). But to keep it simple in this example:
Note that this will not work for a Mac! In Windows you will need to use GLSL 1.2
For now, still just drawing my triangle in the immediate mode. The point is we have replaced how we color our vertices with a fragment shader.
Soon we will need to communicate with the shader what vertices exist on the graphics card however.

51. Final Image

52. Communications Passing Vertex Buffer Objects from CPU -> Shaders
Passing variables from CPU -> Shaders
Passing variables between Shaders

53. Communications (from CPU -> GPU)

54. Communications (from CPU -> GPU)

55. Vertex Shader
Fragment Shader

56. Magic!! Vertex to Fragment Shader
If you are wondering how a vertex shader passes data to a fragment shader…
Because vertex shaders operate at a “per vertex” level and fragments operate at a “per pixel” level
What gets sent from one to another??
The magic process is “blending”
Defaults to linear blend. Can be changed to different blend funcs

57. Communications (Uniform)
Sometimes you want to pass just a single number (not per vertex (i.e. attribute) information via vertex buffer objects).
These are called “uniform” variables. These can include:
Rendering information (e.g. projection matrix)
Rendering resource (e.g. a texture ID)
Control variable (e.g. renderMode)
…. In the vertex shader below, the value of “Scale”

58. Communications (Uniform)
ProgramObject is the bound shader program
glUniform has many forms:
glUniform1f, glUniform2f, … 4f (float)
glUniform1i… 4i (int)
glUniform1ui … 4ui (unsigned int)
glUniform1fv… 4fv (floating array)
glUniformMatrix2fv…4fv

59. Questions?