1. Texture Mapping Kurt Akeley CS248 Lecture 10 25 October 2007 http://graphics.stanford.edu/courses/cs248-07/

2. CS248 Lecture 10Kurt Akeley, Fall 2007 Texture mapping demo

3. CS248 Lecture 10Kurt Akeley, Fall 2007 Texture mapping Paints images onto triangles Paints images onto points, lines, and other images

4. CS248 Lecture 10Kurt Akeley, Fall 2007 Complete OpenGL pipeline Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipelinePixel pipeline Texture mapping ties the vertex and pixel pipelines together Application

5. CS248 Lecture 10Kurt Akeley, Fall 2007 Texture mapping Paints images onto triangles Paints images onto points, lines, and other images Ties the vertex and pixel pipelines together n Rendered images can be used as textures n To modify the rendering of new images –That can be used as textures … Implements general functions of one, two, or three parameters n Specified as 1-D, 2-D, or 3-D tables (aka texture images) n With interpolated (aka filtered) lookup Drives the hardware architecture of GPUs n Multi-thread latency hiding n “shader” programmability Adds many capabilities to OpenGL n Volume rendering n Alternate color spaces n Shadows …

6. CS248 Lecture 10Kurt Akeley, Fall 2007 Complete OpenGL pipeline Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipelinePixel pipeline Application

7. CS248 Lecture 10Kurt Akeley, Fall 2007 Fundamentals of texture mapping Texture mapping requires specification of n The texture image n The mapping from object to texture coordinates n The sampling mechanism n The application of the resulting value(s)

8. CS248 Lecture 10Kurt Akeley, Fall 2007 The OpenGL Pipeline (texture mapping example)

9. CS248 Lecture 10Kurt Akeley, Fall 2007 OpenGL textured-quad code LoadTexture(“Textures\\rooster.tga”, 1);// lots hidden here glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE); glEnable(GL_TEXTURE_2D); glBindTexture(GL_TEXTURE_2D, 1); glClearColor(1, 1, 1, 1);// white glClear(GL_COLOR_BUFFER_BIT); glLoadIdentity(); glOrtho(0, 100, 0, 100, -1, 1); glColor3f(1, 1, 1);// white glBegin(GL_TRIANGLE_STRIP); glTexCoord2f(0, 0);glVertex2i(11, 31); glTexCoord2f(0, 1);glVertex2i(37, 71); glTexCoord2f(1, 0);glVertex2i(91, 38); glTexCoord2f(1, 1);glVertex2i(65, 71); glEnd(); glFlush(); Texture mapping is not corrected for perspective!

10. CS248 Lecture 10Kurt Akeley, Fall 2007 Texture image specification Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipelinePixel pipeline Application struct { float r,g,b,a; } pixel; Scale and offset Color table lookup Optional: Convolution Histogram Min/max computation Unpack from memory format into canonical format (pixel structures)

11. CS248 Lecture 10Kurt Akeley, Fall 2007 Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipelinePixel pipeline Application struct { float x o,y o,z o,w o ; float nx o,ny o,nz o ; float r,g,b,a; float s o,t o,r o,q o ; } vertex; struct { float 11,31,0,1; float 0,0,1; float 1,1,1,1; float 0,0,0,1; } vertex; glColor3f(1,1,1); glBegin(GL_TRIANGLE_STRIP); glTexCoord2f(0, 0); glVertex2i(11, 31);

12. CS248 Lecture 10Kurt Akeley, Fall 2007 Vertex operations Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipelinePixel pipeline Application struct { float x c,y c,z c,w c ; float r,g,b,a; float s c,t c,r c,q c ; } clipvertex;

13. CS248 Lecture 10Kurt Akeley, Fall 2007 Primitive assembly Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipelinePixel pipeline Application struct { float x c,y c,z c,w c ; float r,g,b,a; float s c,t c,r c,q c ; } clipvertex; struct { clipvertex v0,v1,v2; } triangle;

14. CS248 Lecture 10Kurt Akeley, Fall 2007 Primitive operations Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipelinePixel pipeline Application struct { float x w,y w,z w,w c ; float r,g,b,a; float s c,t c,r c,q c ; } winvertex; struct { winvertex v0,v1,v2; } triangle;

15. CS248 Lecture 10Kurt Akeley, Fall 2007 Rasterization Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipelinePixel pipeline Application struct { float x w,y w,z w,w c ; float r,g,b,a; float s c,t c,r c,q c ; } winvertex; struct { winvertex v0,v1,v2; } triangle;

16. CS248 Lecture 10Kurt Akeley, Fall 2007 Perspective-correct attribute evaluation (x 0, y 0, w 0, f 0 ) (x 1, y 1, w 1, f 1 ) (x 2, y 2, w 2, f 2 ) (x, y, f ) a2a2 a1a1 a0a0 All w ’s are w c ’s

17. CS248 Lecture 10Kurt Akeley, Fall 2007 Perspective-correct tex coord evaluation (x 0, y 0, w 0, s 0 ) (x 1, y 1, w 1, s 1 ) (x 2, y 2, w 2, s 2 ) (x, y, s ) a2a2 a1a1 a0a0 w c ’s, s c ’s, q c ’s x w ’s, y w ’s Texture coordinates homogenized here Supports image remapping (from q-specified warp to render-specified warp)

18. CS248 Lecture 10Kurt Akeley, Fall 2007 Rasterization Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipelinePixel pipeline Application struct { float x w,y w,z w ; float r,g,b,a; float s w,t w,r w ; } fragment;

19. CS248 Lecture 10Kurt Akeley, Fall 2007 Fragment operations Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipelinePixel pipeline Application struct { float x w,y w,z w ; float r,g,b,a; float s w,t w,r w ; } fragment;

20. CS248 Lecture 10Kurt Akeley, Fall 2007 Texture lookup struct { float x w,y w,z w ; float r,g,b,a; float s w,t w,r w ; } fragment; 01 0 1 struct { float r t,g t,b t,a t ; } color; If s w or t w is outside the range [0, 1] : - Clamp to edge color - Clamp to border color - Wrap repeatedly - Wrap with mirror reflections swsw twtw

21. CS248 Lecture 10Kurt Akeley, Fall 2007 Texture application struct { float x w,y w,z w ; float r,g,b,a; float s w,t w,r w ; } fragment; 01 0 1 struct { float r t,g t,b t,a t ; } color; swsw twtw struct { float x w,y w,z w ; float r’,g’,b’,a’; } fragment; GL_MODULATE: Alternatives include: - GL_REPLACE - GL_BLEND - GL_ADD

22. CS248 Lecture 10Kurt Akeley, Fall 2007 Fragment / framebuffer operations Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipelinePixel pipeline Application struct { float x w,y w,z w ; float r’,g’,b’,a’; } fragment;

23. CS248 Lecture 10Kurt Akeley, Fall 2007 OpenGL textured-quad code LoadTexture(“Textures\\rooster.tga”, 1);// lots hidden here glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE); glEnable(GL_TEXTURE_2D); glBindTexture(GL_TEXTURE_2D, 1); glClearColor(1, 1, 1, 1);// white glClear(GL_COLOR_BUFFER_BIT); glLoadIdentity(); glOrtho(0, 100, 0, 100, -1, 1); glColor3f(1, 1, 1);// white glBegin(GL_TRIANGLE_STRIP); glTexCoord2f(0, 0);glVertex2i(11, 31); glTexCoord2f(0, 1);glVertex2i(37, 71); glTexCoord2f(1, 0);glVertex2i(91, 38); glTexCoord2f(1, 1);glVertex2i(65, 71); glEnd(); glFlush(); run

24. CS248 Lecture 10Kurt Akeley, Fall 2007 Texture Sampling

25. CS248 Lecture 10Kurt Akeley, Fall 2007 Texel coordinates struct { float x w,y w,z w ; float r,g,b,a; float s w,t w,r w ; } fragment; 01 0 1 struct { float r t,g t,b t,a t ; } color; 02n2n 0 2m2m For this image n = m = 256 256 swsw twtw u v

26. CS248 Lecture 10Kurt Akeley, Fall 2007 Image resampling We are resampling the texture image at the pixel rate We have seen resampling before …

27. CS248 Lecture 10Kurt Akeley, Fall 2007 Supersample antialiasing Supersampling algorithm: 1. Over-sample, e.g., at 4x the pixel rate 2. Reconstruct at the over-sampled rate 3. Band-limit to match the pixel rate 4. Resample at the pixel rate to yield pixel values 5. Reconstruct to display

28. CS248 Lecture 10Kurt Akeley, Fall 2007 Texture resampling (to avoid aliasing) Texture resampling algorithm: 1. Over-sample, e.g., at 4x the pixel rate 2. Reconstruct at the texture-sample rate 3. Band-limit to match the pixel rate 4. Resample at the pixel rate to yield pixel values 5. Reconstruct to display

29. CS248 Lecture 10Kurt Akeley, Fall 2007 Supersampling optimizations The over-sample reconstruction convolution and the band-limiting convolution steps can be combined: n Convolution is associative (f * g) * h = f * (g * h) And g and h are constant n f * (g * h) = f * filter The filter convolution can be reduced to a simple weighted sum of sample values: n The result is sampled at pixel rate n So only values at pixel centers are needed n These are weighted sums of the 4x samples

30. CS248 Lecture 10Kurt Akeley, Fall 2007 Texture resampling optimizations The reconstruction convolution and the band-limiting convolution steps can be combined: n Convolution is associative (f * g) * h = f * (g * h) And g and h are constant n f * (g * h) = f * filter The filter convolution can be reduced to a simple weighted sum of sample values: n The result is sampled at pixel rate n So only values at pixel centers are needed n These are weighted sums of the texels How many texels need to be summed to get a good answer?

31. CS248 Lecture 10Kurt Akeley, Fall 2007 Size of the filter n typically differs for every fragment!

32. CS248 Lecture 10Kurt Akeley, Fall 2007 Magnification One pixel corresponds to less than one texel Resulting image is “larger” (more pixels than corresponding texels) Efficient to implement: Rule 1

33. CS248 Lecture 10Kurt Akeley, Fall 2007 Minification One pixel corresponds to many texels Resulting image is “smaller” (fewer pixels than corresponding texels) Difficult to implement efficiently Could require visiting every texel in the entire texture image!

34. CS248 Lecture 10Kurt Akeley, Fall 2007 Two approaches to antialiasing Supersampling Pre-filtering

35. CS248 Lecture 10Kurt Akeley, Fall 2007 MIPmap pre-filtering Prefilter repeatedly to ½ resolution Reduce resolution equally in all dimensions Stop at a single texel

36. CS248 Lecture 10Kurt Akeley, Fall 2007 MIPmap filtering Choose the pre-filtered images that most closely match the pixel sample rate Sample each of these images Combine the resulting values with a MIPmap filter whose weights are determined by the fractional value of p Notes: n Any filter can be chosen for the in-image sampling n Usually a bilinear filter is chosen for the MIPmap filter n Hence two pre-filtered images are chosen and sampled n Choose the single nearest MIPmap level improves preformance n But violates rule 1 ! n Four or more MIPmap levels can also be chosen

37. CS248 Lecture 10Kurt Akeley, Fall 2007 Tri-linear filtering A horrible term! Usually means sampling a 2-D MIPmapped texture image using bilinear interpolation for both the in-image filtering and the MIPmap filtering OpenGL allows separate specification of in-image and MIPmap filters

38. CS248 Lecture 10Kurt Akeley, Fall 2007 2-D and 3-D textures Spatial rates typically differ in different dimensions: This conservative approach will result in image blurring Can we do better?

39. CS248 Lecture 10Kurt Akeley, Fall 2007 RIPmap pre-filtering

40. CS248 Lecture 10Kurt Akeley, Fall 2007 Anisotropic texture filtering Combine pre-filtering and supersampling Typical algorithm: n Pre-filter to MIPmap images n Supersample the pixel area n For each sample perform a full MIPmap sample operation n Combine the MIPmap samples using pixel-customized filter weights

41. CS248 Lecture 10Kurt Akeley, Fall 2007 Summary Texture mapping is a powerful, general-purpose mechanism n It’s not just painting pictures onto triangles! Texture mapping requires specification of n The texture image n The mapping from object to texture coordinates n The sampling mechanism n The application of the resulting value(s) Of these, sampling is the greatest challenge n Use pre-filtering and supersampling approaches to n Avoid aliasing n Avoid blurring n Maintain high performance

42. CS248 Lecture 10Kurt Akeley, Fall 2007 Assignments Tuesday 30 October n Kurt will be out of town n Saturday morning until Tuesday late n Lecture with be given by Andrew Adams Reading assignment for Tuesday’s class n Work through the tutorial at: n http://www.lighthouse3d.com/opengl/glsl/http://www.lighthouse3d.com/opengl/glsl/ Project 3: n Write a computer-graphics game n Will be assigned this coming Tuesday Office hours today ?

43. CS248 Lecture 10Kurt Akeley, Fall 2007 End