1. Texture Mapping Aaron Bloomfield CS 445: Introduction to Graphics Fall 2006 (Slide set originally by David Luebke)

2. 2 Overview  Motivation and Examples Fundamentals Algorithms Perspective-Correct Texturing Transparency and Anti-Aliasing MIP-Maps Bump and Displacement Maps Illumination Maps

3. 3 Texture Mapping: Motivation Scenes created with diffuse lighting look convincingly three-dimensional, but are flat, chalky, and “cartoonish” Phong lighting lets us simulate materials like plastic and (to a lesser extent) metal, but scenes still seem very cartoonish and unreal Big problem: polygons are too coarse-grained to usefully model fine surface detail Solution: texture mapping

4. 4 Texture Mapping: Motivation Adding surface detail helps keep CG images from looking simple and sterile Explicitly modeling this detail in geometry can be very expensive Zebra stripes, wood grain, writing on a whiteboard Texture mapping pastes images onto the surfaces in the scene, adding realistic fine detail without exploding the geometry

5. 5

6. 6 Texture Mapping: Examples

7. 7 Doom III (ID Software)

8. 8 Texture Mapping In short: it is impractical to explicitly model fine surface detail with geometry Solution: use images to capture the “texture” of surfaces Texture maps can modulate many factors that affect the rendering of a surface Color or reflectance (diffuse, ambient, specular) Transparency (smoke effects) What else?

9. 9 Overview Motivation and Examples  Fundamentals Algorithms Perspective-Correct Texturing Transparency and Anti-Aliasing MIP-Maps Bump and Displacement Maps Illumination Maps

10. 10 Texture Mapping: Fundamentals A texture is typically a 2-D image Can also be A 2-D procedural texture A 3-D procedural texture See video A 3-D volumetric texture (rarely used)

11. 11 Texture Mapping: Fundamentals A texture is typically a 2-D image Image elements are called texels Value stored at a texel affects surface appearance in some way Example: diffuse reflectance, shininess, transparency… The mapping of the texture to the surface determines the correspondence, i.e., how the texture lies on the surface Mapping a texture to a triangle is easy (why?) Mapping a texture to an arbitrary 3-D shape is more complicated (why?)

12. 12 Texturing Fundamentals A texture is typically a 2-D array of texels Mapping the texture to an arbitrary 3-D shape is complex:

13. 13 Texturing Fundamentals A texture is typically a 2-D array of texels Mapping the texture to an arbitrary 3-D shape is complex:

14. 14 Texturing Fundamentals A texture is typically a 2-D array of texels Mapping the texture to an arbitrary 3-D shape is complex:

15. 15 Texturing Fundamentals A texture is typically a 2-D array of texels Mapping the texture to an arbitrary 3-D shape is complex! http://www.cs.virginia.edu/~gfx/Courses/2003/Intro.spring.03/animations/unfold.mo v

16. 16 How to map a texture There are a number of possibilities: Flat Cube Tube (i.e. cylinder) Sphere The slides on the next slide are from http://mediawiki.blender.org/index.php/Manual/Map_Input#2D_to_3D_Mapping http://mediawiki.blender.org/index.php/Manual/Map_Input#2D_to_3D_Mapping

17. 17

18. 18 Overview Motivation and Examples Fundamentals  Algorithms Perspective-Correct Texturing Transparency and Anti-Aliasing MIP-Maps Bump and Displacement Maps Illumination Maps

19. 19 Texture Mapping: Rendering Rendering uses the mapping: Find the visible surface at a pixel Find the point on that surface corresponding to that pixel Find the point in the texture corresponding to that point on the surface Use the parameters associated with that point on the texture to shade the pixel Using triangulated meshes reduces the problem to mapping a portion of the image to each triangle:

20. 20 Texture Mapping: Rendering

21. 21 Texture Mapping: User-Generated Mappings For complex 3-D objects, mapping textures is still something of an art…so we often let the user do it

22. 22 Texture Mapping: Rendering We typically parameterize the texture as a function in (u, v) For simplicity, normalize u & v to [0, 1] Associate each triangle with a texture Give each vertex of the triangle a texture coordinate (u, v) For other points on the triangle, interpolate texture coordinate from the vertices Much like interpolating color or depth But there’s a catch...

23. 23 Naïve Texture Mapping A first cut at a texture-mapping rasterizer: For each pixel: Interpolate u & v down edges and across spans Look up nearest texel in texture map Color pixel according to texel color (possibly modulated by lighting calculations) McMillan’s demo of this is at http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide05.html http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide05.html What artifacts do you see in this demo?

24. 24 Naïve Texturing Artifacts Another serious artifact is warping at the edges of triangles making up the mesh A more obvious example: http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide06.html http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide06.html To address this, need to consider the geometry of interpolating parameters more carefully

25. 25 Interpolating Parameters The problem turns out to be fundamental to interpolating parameters in screen-space Uniform steps in screen space  uniform steps in world coords

26. 26 Interpolating Parameters Perspective foreshortening is not getting applied to our interpolated parameters Parameters should be compressed with distance Linearly interpolating them in screen-space doesn’t do this Is this a problem with Gouraud shading? Is everything I taught wrong? A: It can be, but we usually don’t notice (why?)

27. 27 Overview Motivation and Examples Fundamentals Algorithms  Perspective-Correct Texturing Transparency and Anti-Aliasing MIP-Maps Bump and Displacement Maps Illumination Maps

28. 28 Perspective-Correct Interpolation Skipping a bit of math to make a long story short… Rather than interpolating u and v directly, interpolate u/z and v/z These do interpolate correctly in screen space Also need to interpolate z and multiply per-pixel Problem: we don’t know z anymore Solution: we do know w  1/z So…interpolate uw and vw and w, and compute u = uw/w and v = vw/w for each pixel This unfortunately involves a divide per pixel (Just 1?)

29. 29 Perspective-Correct Texturing Known as perspective-correct texture mapping Early PC cards and game consoles didn’t support it So how did they avoid the warping problem? http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide15.html As mentioned, other interpolation schemes really ought to use perspective correction E.g., Gouraud shading Generally get away without it because it is more important to be smooth than correct Java code fragment from McMillan’s edge- equation triangle rasterizer:

30. 30 Perspective-Correct Texturing: Code... PlaneEqn(uPlane, (u0*w0), (u1*w1), (u2*w2)); PlaneEqn(vPlane, (v0*w0), (v1*w1), (v2*w2)); PlaneEqn(wPlane, w0, w1, w2);... for (y = yMin; y <= yMax; y += raster.width) { e0 = t0; e1 = t1; e2 = t2; u = tu; v = tv; w = tw; z = tz; boolean beenInside = false; for (x = xMin; x <= xMax; x++) { if ((e0 >= 0) && (e1 >= 0) && (e2 >= 0))) { int iz = (int) z; if (iz <= raster.zbuff[y+x]) { float denom = 1.0f / w; int uval = (int) (u * denom + 0.5f); uval = tile(uval, texture.width); int vval = (int) (v * denom + 0.5f); vval = tile(vval, texture.height); int pix = texture.getPixel(uval, vval); if ((pix & 0xff000000) != 0) { raster.pixel[y+x] = pix; raster.zbuff[y+x] = iz; } } beenInside = true; } else if (beenInside) break; e0 += A0; e1 += A1; e2 += A2; z += Az; u += Au; v += Av; w += Aw; } t0 += B0; t1 += B1; t2 += B2; tz += Bz; tu += Bu; tv += Bv; tw += Bw; }

31. 31 Texture Tiling It is often handy to tile a repeating texture pattern onto a surface The previous code does this via tile() : int uval = (int) (u * denom + 0.5f); uval = tile(uval, texture.width); int vval = (int) (v * denom + 0.5f); vval = tile(vval, texture.height); int pix = texture.getPixel(uval, vval); int tile(int val, int size) { while (val >= size) val -= size; while (val < 0) val += size; } See http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide18.html http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide18.html

32. 32 Overview Motivation and Examples Fundamentals Algorithms Perspective-Correct Texturing  Transparency and Anti-Aliasing MIP-Maps Bump and Displacement Maps Illumination Maps

33. 33 Texture Transparency McMillan’s code also includes a “quick fix” for handling transparent texture: if ((pix & 0xff000000) != 0) { raster.pixel[y+x] = pix; raster.zbuff[y+x] = iz; } Note that this doesn’t handle partial transparency (How might such partial transparency arise?) Demo at: http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide19.html http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide19.html

34. 34

35. 35 Texture Map Aliasing Naive texture mapping looks blocky, pixelated Problem: using a single texel to color each pixel: int uval = (int) (u * denom + 0.5f); int vval = (int) (v * denom + 0.5f); int pix = texture.getPixel(uval, vval); Actually, each pixel maps to a region in texture If the pixel is larger than a texel, we should average the contribution from multiple texels somehow If the pixel is smaller than a texel, we should interpolate between texel values somehow Even if pixel size  texel size, a pixel will in general fall between four texels An example of a general problem called aliasing

36. 36 Texture Map Antialiasing Use bilinear interpolation to average nearby texel values into a single pixel value (Draw it) Find 4 nearest texture samples Round u & v up and down Interpolate texel values in u Interpolate resulting values in v Also addresses the problem of many pixels projecting to a single texel (Why?)

37. 37 Texture Map Antialiasing What if a single pixel covers many texels? Problem: sampling those texels at a single point (the center of the pixel): Produces Moire patterns in coherent texture (checkers) Leads to flicker or texture crawling as the texture moves

38. 38 Moire Patterns

39. 39 Texture Map Antialiasing What if a single pixel covers many texels? Approach: blur the image under the pixel, averaging the contributions of the covered texels But calculating which texels every pixel covers is way too expensive, especially as the texture is compressed Solution: pre-calculate lower-resolution versions of the texture that incorporate this averaging

40. 40 Overview Motivation and Examples Fundamentals Algorithms Perspective-Correct Texturing Transparency and Anti-Aliasing  MIP-Maps Bump and Displacement Maps Illumination Maps

41. 41 MIP-maps For a texture of 2 n x 2 n pixels, compute n-1 textures, each at ½ the resolution of previous: This multiresolution texture is called a MIP-map

42. 42 Generating MIP-maps Generating a MIP-map from a texture is easy For each texel in level i, average the values of the four corresponding texels in level i-1 If a texture requires n bytes of storage, how much storage will a MIP-map require? Answer: 4n/3

43. 43 GR B RG B RG B B GR Representing MIP-maps Trivia: MIP = Multum In Parvo (many things in a small place)

44. 44 Using MIP-maps Each level of the MIP-map represents a pre- blurred version of multiple texels A texel at level n represents 2 n original texels When rendering: Figure out the texture coverage of the pixel (i.e., the size of the pixel in texels of the original map) Find the level of the MIP map in which texels average approximately that many original texels Interpolate the value of the four nearest texels

45. 45 Using MIP-maps Even better: Likely, the coverage of the pixel will fall somewhere between the coverage of texels in two adjacent levels of the MIP map Find the pixel’s value in each of the two textures using two bilinear interpolations Using a third interpolation, find a value in between these two values, based on the coverage of the pixel versus each of the MIP-map levels This is (misleadingly?) called trilinear interpolation

46. 46 MIP-map Example No filtering: MIP-map texturing:

47. 47 Can We Do Better? What assumption does MIP-mapping implicitly make? A: The pixel covers a square region of the texture More exactly, the compression or oversampling rate is the same in u and v Is this a valid assumption? Why or why not?

48. 48 MIP-maps and Signal Processing An aside: aliasing and antialiasing are properly topics in sampling theory Nyquist theorem, convolution and reconstruction, filters and filter widths Textures are particularly difficult because a tiled texture can easily generate infinite frequencies E.g., a checkered plane receding to an infinite horizon Using a MIP-map amounts to prefiltering the texture image to reduce artifacts caused by sampling at too low a rate

49. 49 Summed-Area Tables A technique called summed-area tables lets us integrate texels covered by the pixel more exactly (but still quickly) Details in the book Example: MIP-map texturingSummed-area table texturing

50. 50 Overview Motivation and Examples Fundamentals Algorithms Perspective-Correct Texturing Transparency and Anti-Aliasing MIP-Maps  Bump and Displacement Maps Illumination Maps

51. 51 Texture Mapping Variations In addition to the texture, we add the lighting model also: Texture as R,G,B: Texture with lighting:

52. 52 Bump Mapping The texture map can modulate the surface normal used for shading Sphere w/ diffuse textureSwirly bump map Sphere w/ diffuse texture and swirly bump map

53. 53 + = More Bump Mapping How can you tell a bumped-mapped object from an object in which the geometry is explicitly modeled?

54. 54 Another Bump Mapping Example From Wikipedia +=

55. 55 Displacement Map A displacement map actually displaces the geometry Treats the texture as a height field to be applied to the surface Starting to appear in the interactive graphics pipeline First supported in Matrox Parhelia card Can sort of implement with beta drivers in ATI & NVIDIA cards Will soon appear in all cards Implemented by recursive subdivision of triangles/quads

56. 56 Displacement Map Example What is the biggest visual difference between displacement mapping and bump mapping?

57. 57 Overview Motivation and Examples Fundamentals Algorithms Perspective-Correct Texturing Transparency and Anti-Aliasing MIP-Maps Bump and Displacement Maps  Illumination Maps

58. 58 Illumination Maps Quake introduced illumination maps or light maps to capture lighting effects in video games Texture map: Texture map + light map: Light map

59. 59 Illumination Maps Illumination maps differ from texture maps in that they: Usually apply to only a single surface Are usually fairly low resolution Usually capture just intensity (1 value) rather than color (3 values) Illumination maps can be: Painted by hand: Disney’s Aladdin ride Calculated by a global illumination process: Nintendo64 demo, modern level builders

60. 60 Other Texture Applications Lots of other interesting applications of the texture- map concept (we’ll return to some): Shadow maps 3-D textures (marble, wood, clouds) Procedural textures Environment maps & cube maps For a neat explanation of the first three (with cool applets, as usual) check out: http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture22/Slide21.html