Lecture 13: OpenGL Shading Language (GLSL) COMP 175: Computer Graphics April 18, 2018

Motivation Last week, we discussed the many of the new “tricks” in Graphics require low-level access to the Graphics pipeline For example, using textures, which is a pain in regular OpenGL.

Programmable Pipeline (~2001) CUDA/OpenCL (GPGPU) (~2008) Graphics History Software Rendering Old-fashioned way (Use ray casting) Fixed-Function Immediate Mode (What we have used) Programmable Pipeline (~2001) GL Shading Language (What we will learn) CUDA/OpenCL (GPGPU) (~2008) General Purpose GPU Programming

Ray Casting (Software Rendering) Wolfenstein Quake 1

Fixed-Function Quake 3 Half-Life

Programmable Pipeline (Shaders) Doom 3 Far Cry

General Purpose Graphics Processing Unit GPGPU General Purpose Graphics Processing Unit Utilize graphics card for computation Do work other than graphics

Old Fixed-Function Not to exciting, what we’ve been using A bunch of glEnable, glEnable

Programmable Pipeline iPhone intentional, because you have to use shaders to do graphics for openGL. http://www.lighthouse3d.com/tutorials/glsl-tutorial/pipeline-overview/

Why GPU? Having two (CPU and GPU) is better than one… GPU is dedicated to floating point operations (a traditionally expensive operation on a CPU) It is especially optimized for linear algebra (e.g. matrix manipulation)! GPU is multi-core (multi-processor) Imagine that you have one processor per pixel, how would you program what the processor should do? GPU is now optimized to work with textures Modern GPUs come with large amounts of memory, almost comparable to CPUs

Floating Point Operations (FLOPs)

Versions of OpenGL and GLSL (and why you need to care) Confusions galore…

Many versions of OpenGL OpenGL 2.0 (2004) introduces the idea of shaders OpenGL SL 1.0 (or 1.1) is a part of OpenGL 2.0 Currently, OpenGL is at 4.6 (2017). GLSL is also at 4.6 When OpenGL 3.1 was introduced, GLSL changed versioning to match (to 3.1), which was previous 1.3.1 3.0 -> 3.1 (2009) is said to be the “big move” where fixed pipeline is considered deprecated. Mac does split support (only 3.0+ or only <3.0). Mixed syntax is not allowed

OpenGL ES 2.0 is not backward compatible with previous versions OpenGL ES (ES = Embedded Systems) is meant for smart phones, devices, etc. OpenGL ES 1.0 (2003) implemented OpenGL 1.3 OpenGL ES 2.0 (2007) is a reduced set of OpenGL 2.0 But ES 2.0 “jumped ahead” and eliminated almost all fixed-pipeline operations (e.g. glBegin, glEnd) OpenGL ES 2.0 is not backward compatible with previous versions Current version is ES 3.2 (2015)

WebGL is OpenGL for browsers, in particular HTML5 canvas element. WebGL 1.0 is based on OpenGL ES 2.0 (2011) Current version is WebGL 2.0 (2017) based on OpenGL ES 3.0 Largely backward compatible

CUDA / OpenCL Made for general purpose graphics programming (GPGPU) Popular known use is bitcoin mining CUDA is a language specific to nVidia (2007) Obviously different from GLSL in that there are no graphics related keywords or concepts. But the general idea (of parallel computing) is similar OpenCL is an open standard version (2009) of CUDA led by the Khronos Group The Khronos Group is a non-profit consortium founded by major graphics companies Currently Khronos is in charge of the OpenGL, OpenCL, OpenGL ES, WebGL, WebCL, and a bunch of related APIs.

Moving Forward in this Class… Mac is a pain… It only supports up to OpenGL 4.1 Check your computer: https://support.apple.com/en-us/HT202823 Mac is a pain… It has stopped supporting OpenGL as of June 2018 and now only supports their proprietary graphics language called “Metal”. As a result, there are a lot of warnings when compiling OpenGL Mac is a pain… The use of either gl.h or gl3.h means that there’s no backward compatibility When programming in gl3.h, any fixed function call results in an error (e.g. glPushMatrix(), or glRotate3f(), etc.) Windows is a pain… There are many graphics card vendors … the graphics card vendors don’t support all aspects of OpenGL equally As a result, the use of the GLEW library is almost strictly required

GLSL 1.2 vs. 3.3+ There’s a rather large departure between GLSL 1.2 and 3.3 GLSL 1.2 is easier to learn and get a program running. However, it doesn’t work for GL 3.0 (so Mac is a problem because of its split support) GLSL 3.3+ is more “clean”, but we need to learn about Vertex Buffer Object (VBO). We’ll discuss later in passing… It’s a lot of: Memory packing and alignment into the correct format for GLSL Magic incantation to run it Because of the differences in syntax, you need to specify the shader version in your GLSL code!

Quick Demo (of Lab 9)

Accessing Programmable Pipeline Introducing Shaders!

CPU + GPU CPU: Typical C++ code GPU: OpenGL Shading Language (GLSL) CPU passes data to GPU Some processing happens on the CPU: Get mouse movement Load and parse data from disk Some processing happens on the GPU: Transform point from Object to World space Solve lighting equation Render

CPU + GPU (Continued) Think of this as two separate processors. What we’ll learn today are: What the GPU should do (and what the CPU should do). In particular different “shaders” to fit into the different parts of the graphics pipeline How to communicate between CPU and GPU Sharing “chunks of memory” Passing variable names and values (and how to pass variables between different shaders) The “language” used by the GPU (i.e. OpenGL Shading Language, or OpenGL SL, or GLSL).

Types of Shaders Most common: Newer: Vertex Fragment (Pixel) Geometry Tessellation Compute

Two Things to Learn: OpenGL Shading Language (GLSL) Syntax Sequence and processes Vertex Shader Fragment (Pixel) Shader Communication between CPU and GPU What type of information can be passed How to pass information Note that you also need to know how to load a shader program, but that’s “magic incantation” that (mostly) stays the same every time.

Interfacing with the GPU Modern OpenGL Mobile/Web Programming Direct3D Compute Languages Stream, OpenCL, CUDA Future The goal of todays lecture is to show you what you can do to get ready to move onto the next tier of graphics. We’ve covered a lot, most graphics courses end after ray tracer You already have all the pieces you need to make great applications.

Client/Server Architecture Think of like a network Client = the CPU Server = the GPU The less they talk, the better Less waiting for memory to transfer Bandwidth is almost always the bottleneck Send data over once, and have it ready to go We’re going to see a common theme here, and need some vocabulary. If we want to make things faster, we need to pass data around less. Our goal is to minimize the number of openGL calls. Less calls means better performance. This is the same model being used, but in the documentation and command names you will start seeing the word client and server.

Immediate Mode (Fixed-Function pipeline) Every ‘gl’ command is being sent to the GPU every iteration. Very inefficient Think of it like having an interpreter versus a compiler for C++ Michael Shah

Fixed Function (Left) vs. Programmable (Right)

Optimizations Available Vertex Array Display Lists Vertex Buffer Objects Pixel Buffers Frame Buffer Objects All of these optimizations we’re going to see involve chunking a bunch of data together so that we can pass it between the GPU and CPU more efficiently. Michael Shah

Minimizes OpenGL calls on Client (CPU) Vertex Array No more glBegin/glEnd Store Vertices, indices, normals, colors, texture coordinates in an array Minimizes OpenGL calls on Client (CPU) Data still being passed from Client to Server every frame A little better because it’s all at once Stylistically, much cleaner code as well Michael Shah

Send it to the Server(GPU) once, and it is stored in memory on Server 4/22/2014 Display List Compiled geometry Send it to the Server(GPU) once, and it is stored in memory on Server Geometry cannot be updated, without deleting display list. someGeometryDisplayList = glGenLists(1); glNewList(someGeometryDisplayList,GL_COMPILE); drawShape(); glEndList(); Once its on the server you cannot get it back, just like when you upload a picture on the internet Can usually get an order of magnitude of performance improvement Michael Shah

Vertex Buffer Object (VBO) 4/22/2014 Vertex Buffer Object (VBO) Improves upon Vertex Array We can store data on the Server (GPU) Just like display lists! We get a pointer into the Server (GPU) so that we can update it! As good as display lists! Because we have a pointer to the data, multiple clients can access data. Multiple instances of a program can access data Michael Shah

Modifiers to Variables Uniform/Varying/Attribute Uniform – Read only data passed from CPU into GPU (vertex/fragment shaders) Depending on GLSL version: GLSL <=1.2: Attribues & Varying – Pass data from vertex to fragment shader. GLSL >1.2 in / out – in is the variable being passed in, out is the variable being passed out Read-only in fragment shader. Read/Write in Vertex Shader Attributes – Input value which change every vertex. Read only – Used only in vertex shader Varying – Variables passed from one shader to another Uniform – Example pass in a texture Attribute – Passing in different vertices or normals for every vertex, only used in the vertex shader

Order matters… Info is only passed in one direction (starting with vertex shader) Note that we can run this loop multiple times (e.g. for shadow, or achieve specific shading effects (such as toon-shading)) 4 types of shaders available to us. We will cover 2, and the other newer ones we will not. Part of this is because the new shaders require graphics cards often not found in laptops. I encourage you to look into these features for your final projects however. Verex, followd by tessellation, then geometry both dealing with geometry. Finally fragment shader Reference: http://ogldev.atspace.co.uk/www/tutorial30/tutorial30.html

OpenGL Shading Language

Debugging – Still as difficult Domain Specific Language GLSL C-Like Language Made for GPU Architecture Because GPU’s are different than CPU’s Still Cross Platform Debugging – Still as difficult Domain Specific Language Compiled at runtime!!!!! There is an equivalent for Microsoft called HLSL (High Level Shading Language)

GLSL Primitives float, int, bool Example: float f = 10.0; or float f = float(10); (Casting not allowed, choose your type wisely!) No doubles in GLSL, they’re too slow for the architecture! Later versions of GLSL and CUDA get doubles however

GLSL Types Vector: vec2, vec3, vec4 Matrix: mat2, mat3, mat4 Overloaded to take in each type Example: vec4 temp= vec4(1.0, 1.0, 1.0, 1.0) Matrix: mat2, mat3, mat4 Example: glModelViewMatrix is a mat4 type Access first column with: mat[0] Access second column with: mat[1] Example: mat[1] = vec3(1.0, 1.0, 1.0); Access individual element: mat[2][1]; Vectors just like in our Algebra.h file

Get components of vector Some more examples Pass in vec3 into a vec4 vec3 color = vec3(1.0,0.5,0.25) vec4 temp = vec4(color, 1.0) Get components of vector color.x = 1.0; or color[0] = 1.0 color.y = 1.0; or color[1] = 1.0;

Swizzling [1] Swizzling is the ability to rearrange a vectors components vec2 pos = temp.xy; // retrieve x and y component, or any of xyzw vec2 pos = temp.yz; vec4 = color.agbr; // opposite of rgba

More GLSL Keywords [1][2] Your cheat sheet to look at later to see what items you can access. http://www.neatware.com/lbstudio/web/glsl.html More datatypes available as you get into later versions of GLSL Note that some of these default keywords might not be there depending on GLSL version number (for example: gl_ModelViewMatrix isn’t there in 3.0+)

Write a Simple Shader Basic Vertex and Fragment Shader

Writing a shader, compiling a shader, and loading a shader Compile at runtime So we do not need to recompile entire source, if we only modify shader.

Simple Vertex Shader

Simple Fragment Shader

The cpp file

Drawing with immediate mode for now For GLSL 3.0+, we will need to use a Vertex Buffer Object (VBO). But to keep it simple in this example: Note that this will not work for a Mac! In Windows you will need to use GLSL 1.2 For now, still just drawing my triangle in the immediate mode. The point is we have replaced how we color our vertices with a fragment shader. Soon we will need to communicate with the shader what vertices exist on the graphics card however.

Final Image

Communications Passing Vertex Buffer Objects from CPU -> Shaders Passing variables from CPU -> Shaders Passing variables between Shaders

Communications (from CPU -> GPU) https://www.khronos.org/opengl/wiki/Tutorial2:_VAOs,_VBOs,_Vertex_and_Fragment_Shaders_(C_/_SDL)

Communications (from CPU -> GPU)

Vertex Shader Fragment Shader

Magic!! Vertex to Fragment Shader If you are wondering how a vertex shader passes data to a fragment shader… Because vertex shaders operate at a “per vertex” level and fragments operate at a “per pixel” level What gets sent from one to another?? The magic process is “blending” Defaults to linear blend. Can be changed to different blend funcs

Communications (Uniform) Sometimes you want to pass just a single number (not per vertex (i.e. attribute) information via vertex buffer objects). These are called “uniform” variables. These can include: Rendering information (e.g. projection matrix) Rendering resource (e.g. a texture ID) Control variable (e.g. renderMode) …. In the vertex shader below, the value of “Scale”

Communications (Uniform) ProgramObject is the bound shader program glUniform has many forms: glUniform1f, glUniform2f, … 4f (float) glUniform1i… 4i (int) glUniform1ui … 4ui (unsigned int) glUniform1fv… 4fv (floating array) glUniformMatrix2fv…4fv

Questions?