Real-Time Shading Using Programmable Graphics Hardware Introduction, Setup and Examples Wan-Chun Ma National Taiwan University

Course Infomation Instructor  Wan-Chun Ma, Alex  Dept. of Computer Science and Information Engineering, National Taiwan University  Course  4/28, 5/5, 5/12, 5/14  Suggested readings R. Fernando and M. J. Kilgard. The Cg Tutorial: The Definitive Guide to Programmable Real-Time Graphics, Addison-Wesley, 2003 (beginners only!) R. J. Rost. OpenGL Shading Language, Addison-Wesley, 2004  hp?id=tech:courses:dci_rts:home hp?id=tech:courses:dci_rts:home

The Student... The student should be familiar with  C/C++ programming  Graphics basics Transformations in 3D (translation, rotation, modelview, projection) Rasterization Texturing  OpenGL GLUT, GLUI Use texturing in OpenGL

Today’s Schedule Introduction Setup of Programming Environment Real-Time Shading Examples

Introduction

Evolution of GPUs Virtual Fighter SEGA Dead or Alive 3 Temco Dawn Demo NVIDIA NV1Xbox (NV2A)GeForce FX (NV30) 50K triangles/sec 1M pixel ops/sec 1M transistors 100M triangles/sec 1G pixel ops/sec 20M transistors 200M triangles/sec 2G pixel ops/sec 120M transistors

The 5 Generations of GPU 1 st generation (up to 1998)  NVIDIA TNT2, ATI Rage, 3dfx Voodoo3  Lack of transform vertices of 3D objects. Vertex transformation are done by CPU  Limited math operations for combining textures to compute the color of pixels 2 nd generation ( )  NVIDIA GeForce 256, GeForce 2, ATI Radeon 7500  GPU has the ability to do transformation and lighting. Both OpenGL and DirectX 7 support vertex transformation by hardware  Configurable (in driver level) but not programmable

The 5 Generations of GPU 3 rd generation (2001)  NVIDIA GeForce 3, GeForce 4 Ti, Xbox, ATI Radeon 8500  Vertex programmability: DirectX 8 vertex shader and OpenGL ARB vertex program  Pixel-level configurable 4 th generation (2002)  NVIDIA GeForce FX, ATI Radeon 9700  Vertex and pixel programmability  High-level shading language (NVIDIA Cg, Microsoft HLSL, OpenGL GLSL) 5 th generation (2004)  NVIDIA GeForce 6, ATI Radeon X  Infinite length shader program  Dynamic flow control

GPU Model (Old) Fixed function pipeline

GPU Model (Current) Programmability!

GPU Process Vertex Processing Fragment Processing

Programming GPU However, programming in assembly is painful DP3 R0, c[11].xyzx, c[11].xyzx; RSQ R0, R0.x; MUL R0, R0.x, c[11].xyzx; MOV R1, c[3]; MUL R1, R1.x, c[0].xyzx; DP3 R2, R1.xyzx, R1.xyzx; RSQ R2, R2.x; MUL R1, R2.x, R1.xyzx; ADD R2, R0.xyzx, R1.xyzx; DP3 R3, R2.xyzx, R2.xyzx; RSQ R3, R3.x; MUL R2, R3.x, R2.xyzx; DP3 R2, R1.xyzx, R2.xyzx; MAX R2, c[3].z, R2.x; MOV R2.z, c[3].y; MOV R2.w, c[3].y; LIT R2, R2; DX8 shader instructions Basic: mov, add, mul, mad, rsq… Vector operation: dp3, dp4… Miscellaneous: lit, exp, log, min, max, tex… Oh my god!

Programming GPU The need of high level shading language Compile DP3 R0, c[11].xyzx, c[11].xyzx; RSQ R0, R0.x; MUL R0, R0.x, c[11].xyzx; MOV R1, c[3]; MUL R1, R1.x, c[0].xyzx; DP3 R2, R1.xyzx, R1.xyzx; RSQ R2, R2.x; MUL R1, R2.x, R1.xyzx; ADD R2, R0.xyzx, R1.xyzx; DP3 R3, R2.xyzx, R2.xyzx; RSQ R3, R3.x; MUL R2, R3.x, R2.xyzx; DP3 R2, R1.xyzx, R2.xyzx; MAX R2, c[3].z, R2.x; MOV R2.z, c[3].y; MOV R2.w, c[3].y; LIT R2, R2; // A Phong model shader COLOR c = k_a + k_d * dot(N, L) + k_s * pow(max(0, dot(N, H)), k_exp); High level shading language Easier to read and modify Cross-platform Code reuse

Cg: A Shading Language Cg is a high level language from NVIDIA for programming GPUs, developed in close collaboration with Microsoft Cg stands for “C for Graphics” Cg enables a dramatic productivity increase for graphics development developers of:  Games  CAD tools  Scientific visualizations

Cg: A C-like Language Syntax, operators, functions from C Conditionals and flow control (for, if) Particularly suitable for GPUs:  Express data flow of pipeline/stream architecture of GPUs (e.g. vertex-to-pixel)  Vector and matrix operations  Support hardware data types for maximum performance  Exposes GPU functions for convenience and speed: Intrinsic: (mul, dot, sqrt, exp, pow) Built-in: extremely useful and GPU optimized math, utility and geometric functions (noise, ddx, ddy, reflect) Compiler uses hardware profiles to subset Cg as required for particular hardware feature sets

Cg Workflow Architecture

Cg Workflow Shader Development Application Cg program source code // Diffuse lighting float d = dot(normalize(N), normalize(L)); if (d < 0) d = 0; c = d*tex2D(texture, T)*diffuse; 1.Load/bind program 2.Specify program parameter 3.Specify vertex inputs 4.Render Cg Compiler Shader program assembly code DP3 r0.x, f[TEX0], f[TEX0]; RSQ r0.x, r0.x; MUL r0, r0.x, f[TEX0]; DP3 r1.x, f[TEX1], f[TEX1]; RSQ r1.x, r1.x; MUL r1, r1.x, f[TEX1]; DP3 r0, r0, r1; MAX r0.x, r0.x, 1.0; MUL r0, r0.x, DIFFUSE; TEX r1, f[TEX1], 0, 2D; MUL r0, r0, r1; Shader Compiler Shader binary 0000h: D C h: F 6E E D 0020h: 32 2E D 69 4B E h: C 6F F 72 6D 0040h: D 6C E 36 2E

What Cg can do? Real-time visual effects

What Cg can do? Lots of effects…

Coffee Break Next section: Setup of Programming Environment

Setup of Programming Environment

Requirement Hardware  The computer should be equipped with programmable graphics hardware  NVIDIA FX, NVIDIA 6, ATI 9x00, ATI X series Software  Microsoft Visual Studio.NET 2003  GLUT, GLUI...

Installation Cg Toolkit 1.3 (10MB)  ml ml  Check the “Cg Installer for Windows” NVIDIA SDK 9.0 (340MB, not required)  ml ml FX Composer 1.6 (60MB, not required)  _home.html _home.html  Check the “FX Composer 1.6 Installer”

Installation If default installation locations are used, all the packages are installed in the folder of C:\Program Files\NVIDIA Corporation\ C:\Program Files\NVIDIA Corporation\  Cg\  NVIDIA FX Composer\  SDK 9.0\

My Stuff Several useful codes I collect  download/dci_rts/class.zip download/dci_rts/class.zip Download it and unpack it into a folder, say  D:\My Projects\Class\  Any folder is ok, but remember where you put it

VC++ Directories Execute visual studio  Tools, Options, Projects, VC++ Directories  Show the directories for:  Include files D:\My Project\Class (remember My Stuff?) C:\Program Files\NVIDIA Corporation\Cg\include  Library files C:\Program Files\NVIDIA Corporation\Cg\lib

Ready to Go A small engine  download/dci_rts/env.zip download/dci_rts/env.zip  I will use this engine for shader development during these courses The first example  download/dci_rts/ex01.zip download/dci_rts/ex01.zip

Compilation cgc –profile profiles filename  profiles: graphics hardware profiles Vertex: arbvp1, vp20, vp30, vp40... Fragment: arbfp1, fp20, fp30, fp40...  filename: filename of the shader Examples  cgc –profile vp30 test_vtx.cxx  cgc –profile fp30 test_frg.cxx

Debugging Debugging is very hard (it is GPU, not CPU) However, you may still use intermediate visualization to debug your program  Output intermediate data (e.g. position, normal, textures…) as color

Coffee Break Next section: Real-time Shading Examples

Real-Time Shading Examples

Progression Games push hardware, hardware advances games

Effects in Games Shadows Level of detail Reflection Shading Smoke

Effects in Games Bump mapping Light mapping Per-pixel lighting Multi-texturing

Multi-pass Rendering The rendering pass is not fixed anymore. A single rendering pass may consists of many functional programs

Multi-pass Rendering Each different program (effect) is handled individually, and finally summed up to become rendering result

Cg Samples Check out the effect samples in NVIDIA SDK Browser

The First Cg Example A Phong model shader with color texture Shaders  Vertex: ex1_vtx.cxx  Fragment: ex1_frg.cxx Texture  Diffuse: wood.bmp

Vertex Shader struct v2f { float4 P2D: POSITION;// projected 2D position float4 C: COLOR0;// color float4 T: TEXCOORD0;// texture coord float3 P3D: TEXCOORD1;// vertex 3D position float3 N: TEXCOORD2;// normal float3 G: TEXCOORD3;// tangent float3 B: TEXCOORD4;// binormal }; Vertex-to-fragment data structure

Vertex Shader Main (application-to-vertex) arguments v2f main( float4 C: COLOR, float4 P: POSITION, float4 N: NORMAL, float4 T: TEXCOORD0, uniform float4x4 ModelViewProj, uniform float4x4 ModelView, uniform float4x4 ModelViewIT)

Vertex Shader Main body { v2f OUT; OUT.P2D = mul(ModelViewProj, P); OUT.P3D = P.xyz; OUT.T = T; OUT.N= normalize(N.xyz); // normal OUT.G= normalize(2.0*C.xyz - 1.0);// tangent OUT.B= normalize(cross(OUT.G, OUT.N)); return OUT; }

Fragment Shader Fragment-to-screen data structure struct f2s { float4 C: COLOR0; };

Fragment Shader Main (vertex-to-fragment) arguments f2s main( v2f IN, uniform sampler2Dtex01, // texture 01 uniform float3L, uniform float3V)

Fragment Shader Main body { f2s OUT; OUT.rgb = 0; L = normalize(L); V = normalize(V); float3 H = normalize(L+V); float diff = dot(normalize(IN.N), L); if(diff > 0) { float spec = 2*pow(dot(IN.N, H), 128); OUT.C.rgb = diff*tex2D(tex01, IN.T.xy) + spec; } return OUT; }

Result

Try This... Output red color for all fragments { f2s OUT; OUT.rgb = 0; // L = normalize(L); V = normalize(V); // float3 H = normalize(L+V); // float diff = dot(normalize(IN.N), L); // if(diff > 0) // { // float spec = 2*pow(dot(IN.N, H), 128); OUT.C.rgb = float3(1.0, 0.0, 0.0); // } return OUT; }

Try This... Visualize normal vectors { f2s OUT; OUT.rgb = 0; // L = normalize(L); V = normalize(V); // float3 H = normalize(L+V); // float diff = dot(normalize(IN.N), L); // if(diff > 0) // { // float spec = 2*pow(dot(IN.N, H), 128); OUT.C.rgb = (IN.N+1)/2; // } return OUT; }

End