1. Texture Mapping

2. Typical application: mapping images on geometry 3D geometry (quads mesh) + RGB texture 2D (color-map) =

3. More examples

4. Texture Mapping The fragment operations can access a specialized RAM –The Texture RAM –Organized in a set of Textures Each texture is an array 1D, 2D o 3D of Texels (texture elements) of the same type

5. Texels Typical examples of texels: –each texel is a color (components: R-G-B, or R-G-B-A) The texture is a "color-map" –each texel is an alpha value the texture is an "alpha-map" –each texel is a normal (components: X-Y-Z) the texture is a "normal-map" or "bump-map" –each texel contains a specularity value the texture is a "shininess-map" –...

6. More examples +=

7. Texture Mapping: History 1974 introduced by Ed Catmull –In its Phd Thesis Only in 1992 (!) we have texture mapping hardware –Silicon Graphics RealityEngine 1992 on: increasingly used and integrated in graphic cards –First of all by low end graphic boards Today: a fundamental rendering primitive –the main image-based technique Ed Catmull (MEGA-MEGA- GURU)

8. Notation Texture 2D u v texel Texture Space (or "parametric space" or "u-v space") A Texture is defined In the region [0,1] x [0,1] of the "parametric space" 512 texels 1024 texels 1.0

9. Texture Mapping We associate to each vertex (of each triangle) its u,v coordinates in the texture space Screen Space x 0,y 0 x 2,y 2 x 1,y 1 u 0,v 0 u 1,v 1 u 2,v 2 Position of the 1st vertex Attributes of the 1st vertex u 0,v 0 u 1,v 1 u 2,v 2

10. Texture Mapping More precisely, we define a mapping between the 3D triangle 3D and a texture triangle Texture Space Screen Space

11. Texture Mapping each fragment has its own u,v coordinates in the texture space Texture Space Screen Space texture look-up

12. Texture Mapping Screen buffer Texture RAM interpolation texture coordinates texture coordinates interpolated including: texture coordinates (per vertex!) texture look-up Fragments & interpolated attributes Vertices & their attributes Projected Vertices & computed attributes Triangles rasterizer set- up Segments rasterizer set- up points rasterizer set- up vertex computation Fragment computations

13. Problem: linear interpolation of texture coordinates Not true for perspective projection! –It is only an approximation –It works fine to interpolate colors, normals,.. –Not applicable to interpolate texture coordinates... V1V1 V2V2 V3V3 p f(p) f( v 1 ) f( v 2 ) f( v 3 ) projection f p has barycentric coordinates a,b,c In the triangle v 1 v 2 v 3 f(p) has barycentric coordinates a,b,c in the triangle f(v 1 ) f(v 2 ) f(v 3 )

14. Problem: linear interpolation of texture coordinates Example: u v 1 1 u,v= (1,0) u 1,v 1 = (1,1) u 1,v 1 = (0,1) u 1,v 1 = (0,0)

15. Problem: linear interpolation of texture coordinates Example:

16. Solution: Perspective Correction p has barycentric coordinates c 0 c 1 c 2 V0V0 V2V2 V1V1 A 0,B 0... A 1,B 1... A 2,B 2... p p = c 0 v 0 + c 1 v 1 + c 2 v 2 Attributes of p: (not considering the “ perspective correction ") A p = c 0 A 0 + c 1 A 1 + c 2 A 2 B p = c 0 B 0 + c 1 B 1 + c 2 B 2 = ( x 0, y 0, z 0, w 0 )

17. Solution: Perspective Correction p has barycentric coordinates c 0 c 1 c 2 V0V0 V2V2 V1V1 A 0,B 0... A 1,B 1... A 2,B 2... p p = c 0 v 0 + c 1 v 1 + c 2 v 2 Attributes of p: (not considering the “ perspective correction ") w0w0 w1w1 A p = c 0 A 0 + c 1 A 1 + c 2 A 2 A0A0 w0w0 A1A1 w1w1 A2A2 w2w2 Ap =Ap = 1 11 w2w2 = ( x 0, y 0, z 0, w 0 )

18. Solution: Perspective Correction Screen buffer Original attribute A Apply transformations compute: A' = A / w and w' = 1 / w c 0 + c 1 + c 2 A0A0 w0w0 A1A1 w1w1 A2A2 w2w2 Ap =Ap = 1 w0w0 1 w1w1 1 w2w2 interpolate A' and w' Final fragment attribute: A' / w' c 0 + c 1 + c 2 Fragments & interpolated attributes Vertices & their attributes Projected Vertices & computed attributes Triangles rasterizer set- up Segments rasterizer set- up points rasterizer set- up vertex computation Fragment computations

19. Perspective Correction WithoutWith

20. Perspective Correction Texture mapping with perspective correction –Also known as Perfect texture mapping

21. Note: the texture must be loaded Screen buffer Texture RAM LOADLOAD Fragments & interpolated attributes Vertices & their attributes Projected Vertices & computed attributes Triangles rasterizer set- up Segments rasterizer set- up points rasterizer set- up vertex computation Fragment computations

22. Note: the texture must be loaded 1.From hard disk to main RAM memory (in the motherboard) 2.From main RAM memory to Texture RAM (on board of the graphics HW) Both steps are quite slow. It is not possible to accomplish them once per frame!

23. In OpenGL As an example: glEnable(GL_TEXTURE_2D); glBindTexture (GL_TEXTURE_2D, ID); glTexImage2D ( GL_TEXTURE_2D, 0, // mipmapping GL_RGB, // original format imageWidth, imageHeight, 0, // border GL_RGB, // RAM format GL_UNSIGNED_BYTE, imageData);

24. Assigning texture coordinates to vertices Screen buffer Texture RAM interpolation texture coordinates Interpolated texture coordinates including: coordinates texture (per vertex!) texture look-up including: texture coordinates (per vertex!) Fragments & interpolated attributes Vertices & their attributes Projected Vertices & computed attributes Triangles rasterizer set- up Segments rasterizer set- up points rasterizer set- up vertex computation Fragment computations

25. Assigning texture coordinates to vertices 2 possibilities: –Computing textures coordinates on the fly During the rendering… –Precomputing (and store them within the mesh) The choice is application-dependent!

26. Difficult problem: u-v mapping Associate texture coordinates to each vertex of the mesh –During preprocessing u v u v

27. Difficult problem: u-v mapping Hand-made or automated

28. In OpenGL Like any other attribute TexCoord2d( u,v )

29. Assigning texture coordinates to vertices Screen buffer Texture RAM interpolating texture coordinates texture coordinates interpolated texture look-up texture coordinates (transformed) including: texture coordinates Fragments & interpolated attributes Vertices & their attributes Projected Vertices & computed attributes Triangles rasterizer set- up Segments rasterizer set- up points rasterizer set- up vertex computation Fragment computations

30. Assigning texture coordinates to vertices 2 possibilities: –Computing textures coordinates on the fly During the rendering… –Precomputing (and store them within the mesh)

31. Assigning texture coordinates to vertices Screen buffer Texture RAM interpolating texture coordinates Interpolated texture coordinates texture look-up texture coordinates compute texture coordinates Using the position Fragments & interpolated attributes Vertices & their attributes Projected Vertices & computed attributes Triangles rasterizer set- up Segments rasterizer set- up points rasterizer set- up vertex computation Fragment computations

32. Automatically computed Idea: from (x,y,z) to (u,v) - Linearly Using object or view coordinate –(before or after the trasformation) Examples:

33. Automatically computed Even 1D 1D texture!

34. Assigning texture coordinates to vertices 2 possibilities: –Computing textures coordinates on the fly During the rendering… –Precomputing (and store them within the mesh)

35. Environment mapping: spherical Environment map: a texture containing the color of the environment “reflexed by each normal of the half-sphere”. The texture coordinate is the transformed normal!

36. Environment mapping: spherical Simulates a mirror-like object reflecting a far-away background simulates a complex material (fixed lighting)

37. Environment mapping: cube frontrightback below above left

38. Environment mapping: cube Screen buffer Texture RAM interpolating 3D texture coordinates interpolated coordinates 3D texture Project on the cube, look-up the corresponding face compute 3D Texture coordinates [-1,+1] x [-1,+1] x [-1,+1] As view ray reflexed by the normal Fragments & interpolated attributes Vertices & their attributes Projected Vertices & computed attributes Triangles rasterizer set- up Segments rasterizer set- up points rasterizer set- up vertex computation Fragment computations

39. Environment mapping: cube frontrightback below above left

40. Environment mapping: cube and spherical Spherical: –one texel for each direction in the half- sphere Projected on a circle –the texture coordinate is the normal –It has the "headlight“ effect: I can only rotate the object while the viewpoint does not change Cube –one texel for each direction in the sphere Projected on the surface of the cube –the texture coordinate is the view direction reflexed by the normal –The viewpoint can rotate around a steady object

41. Automated computation of texture coordinates glEnable(GL_TEXTURE_GEN_S); 1- abilitate: 2- choice of the mode: glTexGeni(GL_S, GL_TEXTURE_GEN_MODE, mode ) S, T, R, Q mode = GL_OBJECT_LINEAR GL_EYE_LINEAR GL_SPHERE_MAP Computes the texture coordinates from the position in object coordinates (before the trasformation) Computes the texture coordinates from the position in view coordinates (after the MODEL-VIEW) The texture coordinates is the reflexed view ray (using the normal) (after the MODEL-VIEW)

42. Automated computation of texture coordinates glTexGenfv(GL_S, GL_EYE_PLANE, v); 3- choice of the plane S, T, R, Q EYE OBJECT or 4 elements vector The resulting texture coordinate = v T pos_vertex (It’s the distance from the plane!)

43. u Texture Look-up out of bounds: “clamp” mode if (u 1) u←1; if (v 1) v←1; 1 1 v

44. Texture Look-up out of bounds: “repeat” mode u v 1 1 u ← u – [ u ] v ← v – [ v ]

45. Repeated textures Typical use: Very space-efficient! Note: the texture must be TILEABLE

46. In OpenGL glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP ); glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT ); or note: u and v treated separately Texture parameters. each texture loaded in memory has its own parameters.

47. Texture Look-up A fragment can have non-integer coordinates (in texels) Texture Space Screen Space texture look-up

48. pixel Texture Look-up Texture SpaceScreen Space pixel texel one pixel = less than one texel one pixel = more than one texel minification magnification

49. Magnification Solution 1: Use the texel containg the pixel (that is, the texel whose center is closest to the u,v coordinates of the fragment) Equivalent to rounding up the texel coordinates to the nearest integer "Nearest Filtering" 0.51.5 2.5 3.54.55.56.5 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 u v

50. Magnification texture 128x128 Nearest Filtering: result "texels are visible !"

51. Magnification Solution 2: Compute the average of the four closest texels Bilinear Interpolation 0.51.5 2.5 3.54.55.56.5 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 u v

52. Magnification texture 128x128 Bilinear Interpolation: result

53. Magnification Nearest filtering: –Texels are visible –Ok if texel borders are useful –More efficient Bilinear Interpolation –Usually provides better quality –Less efficient –Sometimes there is an “out-of-focus“ effect

54. Minification Nearest Filtering Bilinear interpolation Does not solve the problem

55. Minification: MIP-mapping MIP-map level 0 MIP-map level 1 MIP-map level 2 MIP-map level 3 MIP-map level 4 (only one texel) MIP-mapping: "Multum In Parvo"

56. Mipmap Math Define a scale factor,  =texels/pixel –  is the maximum between  x and  y –It can vary in the same triangle –Can be derived from the transformation matrices, computed for the Vertices and interpolated for the fragments The mipmap level to use is: log 2  –level 0 = maximum resolution –if level<0 what is the reason? –note: the level might not be an integer

57. Minification: MIP-mapping Bilinear interpolation MIP-mapping

58. Minification: MIP-mapping 0 1 2 3 4 5 Other example

59. In OpenGL glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR ); or Choose the magnification filter:

60. In OpenGL glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, mode ); mode = GL_NEAREST GL_LINEAR GL_NEAREST_MIPMAP_NEAREST GL_LINEAR_MIPMAP_NEAREST GL_NEAREST_MIPMAP_LINEAR GL_LINEAR_MIPMAP_LINEAR where Choose the minification filter: Trilinear interpolation

61. In OpenGL Load on the graphics card all the mipmapping levels. –One-by-one: glTexImage2D ( GL_TEXTURE_2D, i, // MIP-map level GL_RGB, // original format imageWidth, imageHeight, 0, // border GL_RGB, // RAM format GL_UNSIGNED_BYTE, imageData);

62. In OpenGL Load on the graphics card all the mipmapping levels. –All together (using the glu library): glTexImage2D ( GL_TEXTURE_2D, 0, // MIP-map level GL_RGB, // original format imageWidth, imageHeight, 0, // border GL_RGB, // RAM format GL_UNSIGNED_BYTE, imageData); gluBuild2DMipmaps (