1. 1 Texture Maps Jeff Parker Oct 2013

2. 2 Objectives Introduce Mapping Methods Texture Mapping Environment Mapping Bump Mapping Billboards Consider basic strategies Forward vs backward mapping Point sampling vs area averaging

3. 3 Limits of Geometric Modeling We can create well lit spheres and cones, but geometry lacks visual interest Needed to spice things up

4. Bricks 4

5. Iron Maiden Demo 5

6. Commands ' ' – Toggle animation p – Toggle performance t – Toggle texture mapping m – Toggle Texture Mode f – Toggle filter Mode b – Toggle Background 1-4 – Change Background c - Toggle Culling * - Toggle starfield r – Print pixels/frame Arrow keys left/right change number of spheres Arrow keys up/down change level of details change # of textures Nate Robins - 1997

7. 7 Basic Stragegy Three steps to applying a texture Specify the texture Read or generate image Assign to texture Enable texturing Assign texture coordinates to vertices "Tack it up" Proper mapping function is left to application Specify texture parameters Wrapping, filtering, sampling

8. 8 Texture Mapping and the OpenGL Pipeline geometry pipeline vertices pixel pipeline image rasterizer Images and geometry flow through separate pipelines Join at the rasterizer “Complex” textures do not affect geometric complexity

9. 9 Define a texture image from an array of texels (texture elements) in CPU memory Define as any other pixel map Scanned image Generate by application code Enable texture mapping OpenGL supports 1-4 dimensional texture maps WebGL is limited to 1-2 dimensional texture maps Specifying a Texture Image

10. "Tacking it up" Based on parametric texture coordinates Mapping a Texture

11. var texCoord = [ vec2(0, 0), vec2(0, 1), vec2(1, 1), vec2(1, 0) ]; pointsArray.push(vertices[a]); colorsArray.push(vertexColors[a]); texCoordsArray.push(texCoord[0]);

12. 12 Nate Robin's Tutor

13. 13 Applying Textures II 1.specify textures in texture objects 2.set texture filter (minification/magnification) 3.set texture function 4.set texture wrap mode 5.set optional perspective correction hint 6.bind texture object 7.enable texturing 8.supply texture coordinates for vertex coordinates can also be generated

14. 14 In practice Let's walk through a simple example We create two textures As we image the object (a cube) we tie a texture coordinate to each vertex We enable texture mapping

15. textureCubev2.js // Create a checkerboard pattern var image1 = new Array() for (var i =0; i<texSize; i++) image1[i] = new Array(); for (var i =0; i<texSize; i++) for ( var j = 0; j < texSize; j++) image1[i][j] = new Float32Array(4); for (var i =0; i<texSize; i++) for (var j=0; j<texSize; j++) { var c = (((i & 0x8) == 0) ^ ((j & 0x8) == 0)); image1[i][j] = [c, c, c, 1]; }

16. textureCubev2.js // Create a checkerboard pattern using floats... // Convert floats to ubytes for texture var image2 = new Uint8Array(4*texSize*texSize); for ( var i = 0; i < texSize; i++ ) for ( var j = 0; j < texSize; j++ ) for(var k =0; k<4; k++) image2[4*texSize*i+4*j+k] = 255*image1[i][j][k];

17. 17 Texture Objects function configureTexture(image) { texture = gl.createTexture(); gl.activeTexture( gl.TEXTURE0 ); gl.bindTexture( gl.TEXTURE_2D, texture ); gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true); gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, texSize, texSize, 0, gl.RGBA, gl.UNSIGNED_BYTE, image); gl.generateMipmap( gl.TEXTURE_2D ); gl.texParameteri( gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST_MIPMAP_LINEAR); gl.texParameteri( gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST ); }

18. 18 Define Image as a Texture void texImage2D(GLenum target, GLint level, GLenum internalformat, GLenum format, GLenum type, ImageData? pixels); gl.texImage2D( gl.TEXTURE_2D, 0, gl.RGB, gl.RGB, gl.UNSIGNED_BYTE, image ); target: type of texture, e.g. GL_TEXTURE_2D level: used for mipmapping (discussed later) internalformat, format, type: describe texels pixels: pointer to texel array

19. 19 Converting A Texture Image OpenGL requires texture dimensions to be powers of 2 Even more limited in WebGL If dimensions of image are not powers of 2 gluScaleImage( format, w_in, h_in, type_in, *data_in, w_out, h_out, type_out, *data_out ); data_in is source image data_out is for destination image Image interpolated and filtered during scaling Check your system: GL_ARB_texture_non_power_of_two

20. In practice var cBuffer = gl.createBuffer(); gl.bindBuffer( gl.ARRAY_BUFFER, cBuffer); gl.bufferData( gl.ARRAY_BUFFER, flatten(colorsArray), gl.STATIC_DRAW ); var vColor = gl.getAttribLocation( program, "vColor" ); gl.vertexAttribPointer(vColor, 4, gl.FLOAT, false, 0, 0); gl.enableVertexAttribArray(vColor); var vBuffer = gl.createBuffer(); gl.bindBuffer( gl.ARRAY_BUFFER, vBuffer); gl.bufferData( gl.ARRAY_BUFFER, flatten(pointsArray), gl.STATIC_DRAW); var vPosition = gl.getAttribLocation( program, "vPosition" ); gl.vertexAttribPointer( vPosition, 4, gl.FLOAT, false, 0, 0); gl.enableVertexAttribArray(vPosition); var tBuffer = gl.createBuffer(); gl.bindBuffer( gl.ARRAY_BUFFER, tBuffer); gl.bufferData( gl.ARRAY_BUFFER, flatten(texCoordsArray), gl.STATIC_DRAW ); var vTexCoord = gl.getAttribLocation( program, "vTexCoord"); gl.vertexAttribPointer(vTexCoord, 2, gl.FLOAT, false, 0, 0); gl.enableVertexAttribArray(vTexCoord); configureTexture(image2); Passing in three attributes per vertex

21. Pin Textures function quad(a, b, c, d) { pointsArray.push(vertices[a]); colorsArray.push(vertexColors[a]); texCoordsArray.push(texCoord[0]); pointsArray.push(vertices[b]); colorsArray.push(vertexColors[a]); texCoordsArray.push(texCoord[1]);...

22. Parameters: Wrapping Mode Clamping: if s, t > 1 use 1, if s, t < 0 use 0 Wrapping: use s, t modulo 1 texParameteri( target, type, mode ) texParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP ) texParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT ) texture s t GL_CLAMP wrapping GL_REPEAT wrapping s, t: parameter space

23. 23 Magnification and Minification TexturePolygon MagnificationMinification PolygonTexture More than one texel can cover a pixel (minification) or more than one pixel can cover a texel (magnification) Can use point sampling (nearest texel) or linear filtering ( 2 x 2 filter) to obtain texture values

24. 24 Filter Modes Modes defined by calls to texParameteri( target, type, mode ) texParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); texParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); Linear filtering requires a border of an extra texel for filtering at edges (border = 1)

25. 25 Mipmapped Textures Mipmapping define prefiltered texture maps of decreasing resolutions Lessens interpolation errors for smaller objects Declare mipmap level of image when defining texture texImage2D( GL_TEXTURE_2D, 0, … ) Mipmap generate will build all the textures from image generateMipmap( gl.TEXTURE_2D ); Can also load your own…

26. MipMapping 26

27. Reading in a Texture OpenGL does not have any way to read images It is equally bad at JPG, TIFF, etc In the past I have spent time showing how to read.ppm files, a very simple graphics format. 27

28. PPM Files Portable Pixel Map (PPM) files are a simple, uncompressed format Can be read by xv and gimp (GNU Image Manipulation Program). I use GraphicConverter from Lemksoft Header holds Version String # One or more comments width height maxval Example P6 # Created by Paint Shop Pro 5 128 # Could have more comments between values 255 &@#$5%... OpenGL does not support reading or writing graphical images (JPG, PNG, etc) The.ppm format is simple enough for us to create utility to read a file

29. ReadPPMFile Header /* Read a P6 PPM File */ int readPPMFile(GLubyte image[MAX][MAX][3], char *filename) { FILE* fp; int i, w, h, m; char head[70]; /* max line <= 70 in PPM (per spec). */ fp = fopen(filename, "rb"); if (!fp) { perror(filename); exit(1); } /* Check for the PPM Magic number, P6 */ fgets(head, 70, fp); if (strncmp(head, "P6", 2)) { fprintf(stderr, "%s: Not a raw PPM file\n", filename); exit(1); }

30. ReadPPMFile Header /* grab the three elements in the header (width, height, maxval). */ i = 0; while (i < 3) { fgets(head, 70, fp); if (head[0] == '#') /* skip comments. */ continue; if (i == 0) i += sscanf(head, "%d %d %d", &w, &h, &m); else if (i == 1) i += sscanf(head, "%d %d", &h, &m); else if (i == 2) i += sscanf(head, "%d", &m); } if ((w != MAX) || (h != MAX) || (m > 255)) 30

31. The work in ReadPPMFile /* Read a P6 PPM File */ int readPPMFile(GLubyte img[MAX][MAX][3], char *fname) {... fread(image, sizeof(unsigned char), w*h*3, fp); fclose(fp); return 1; } int main(int argc, char **argv) {... if ((argc > 1) && (argv[1][0] != '-')) readPPMFile(image, argv[1]);... 31

32. Where can I get Textures? Paul Bourke has a large collection of images at http://paulbourke.net/ 32 http://paulbourke.net/texture_colour/leaf/

33. 33 Paul Bourke

34. 34 Texture Parameters OpenGL has a variety of parameters that determine how texture is applied We have seen wrapping parameters which determine what happens if s and t are outside the (0,1) range Filter modes allow us to use area averaging instead of point samples Mipmapping allows us to use textures at multiple resolutions Environment parameters determine how texture mapping interacts with shading

35. Texture Functions Controls how texture is applied glTexEnv{fi}[v]( GL_TEXTURE_ENV, prop, param ) GL_TEXTURE_ENV_MODE modes GL_MODULATE: modulates with computed shade GL_BLEND: blends with an environmental color GL_REPLACE: use only texture color GL(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE); Set blend color with GL_TEXTURE_ENV_COLOR

36. 36 Interpolation OpenGL uses interpolation to find proper texels from specified texture coordinates Can be distortions good selection of tex coordinates poor selection of tex coordinates texture stretched over trapezoid showing effects of bilinear interpolation

37. 37 Aliasing Point sampling of the texture can lead to aliasing errors point samples in u,v (or x,y,z) space point samples in texture space miss blue stripes

38. 38 Aliasing Point sampling of the texture can lead to aliasing errors The Nyquist limit point samples in u,v (or x,y,z) space point samples in texture space miss blue stripes

39. 39 Nyquist Limit To reconstruct a signal, must sample at over twice the frequency

40. Aliasing 40

41. Undersampling - holes 41

42. 42 Area Averaging Note that preimage of pixel is curved pixel preimage A better but slower option is to use area averaging Acts as a low pass filter

43. Prefiltering 43

44. Compare 44

45. Supersampling 45

46. Some Algorithms 46 We could look for the nearest textel – GL_NEAREST We could average 4 – GL_LINEAR We could MipMap and take best fit – GL_NEAREST_MIPMAP_X Then apply NEAREST of LINEAR We could MipMap and take bracketing maps GL_LINEAR_MIPMAP_X Then apply NEAREST of LINEAR

47. With change in Z 47

48. Nearest Neighbor 48

49. Checkerboard 49

50. What is the problem? 50

51. How can we fix? 51

52. How can we fix? 52

53. How can we fix? 53

54. Perlin Noise 54 A special random number generator http://mrl.nyu.edu/~perlin/doc/oscar.html#noise www.noisemachine.com/talk1/ Widely used to make realistic messes Marble Dirty surfaces Perlin won an Oscar for his work on Tron http://www.youtube.com/watch?v=- 3ODe9mqoDE&feature=player_embedded

56. 56 Coordinate Systems Although the idea is simple---map an image to a surface---there are 4 coordinate systems involved Parametric coordinates May be used to model curves and surfaces Texture coordinates Used to identify points in the image to be mapped Object or World Coordinates Conceptually, where the mapping takes place Window Coordinates Where the final image is really produced

57. 57 Texture Mapping parametric coordinates texture coordinates world coordinates window coordinates

58. 58 Mapping Functions We often have a natural map from texture coordinates to a point on a surface For example, a torus (surface of a donut) But we really want to go the other way From (x, y, z) to (s, t) s t (x,y,z)

59. Where is my color? 59

60. 60 Backward Mapping Given a pixel in screen space, we want to know to which point on an object it corresponds Given a point on an object, we want to know to which point in the texture it corresponds Need a map of the form s = s(x,y,z) t = t(x,y,z) Such functions are difficult to find in general In our cube example, we can interpolate over the corners Issues with Perspective, Aliasing

61. 61 Two-part mapping One solution to the mapping problem is to first map the texture to a simple intermediate surface Example: map to cylinder Look at the man page for glTexGenX() I do not see evidence for support in WebGL for this

62. Cylinder 62

63. Spherical Map 63

64. Cube Map 64

65. Applications: Environment Map 65

66. Result 66

67. 67 Environment Map Example An example from NVIDIA Now difficult to find Uses cube environment map, mipmaps Options – ' ', c, s, m, a, z Also Menu choices

68. 68 Main (start) http://www.alecjacobson.com/weblog/?p=959

69. 69 You can use different geometry

70. Environment Map 70 Geri's Game – Pixar http://vimeo.com/42073764

71. Environment Map 71 Geri's Game – Pixar http://vimeo.com/42073764

72. Second Mapping 72

73. 73

74. 74 Perspective Correction Hint Texture coordinate and color interpolation either linearly in screen space or using depth/perspective values (slower) Noticeable for polygons “on edge” glHint(GL_PERSPECTIVE_CORRECTION_HINT, hint) where hint is one of GL_DONT_CARE GL_NICEST GL_FASTEST

75. Ray Tracing vs Env Map 75

76. Bump Mapping 76

77. 77

78. 78

79. Displacement Maps 79 Bump maps are an illusion Do not affect the silhouette To change the surface itself, we have to work harder True "Displacement Mapping" is more work We could move each vertex of a finely triangulated mesh

80. 80 Billboard Used to create complex images Draw the image perpendicular to ray from viewer As you move, the billboard rotates to face you Background of billboard is transparent

81. 81 Summary Texture mapping is an efficient way to add visual interest The texture can be procedural, or defined by an image It is not easy to paste an image onto a curved surface Other problems arise when using perspective OpenGL provides many options to work around these issues. Resources: NVIDIA tutorial (links page) – cubemap example Nate Robins "Iron Maiden" demo – under his website, in SGI section