Photo-realistic Rendering and Global Illumination in Computer Graphics Spring 2012 Texturing K. H. Ko School of Mechatronics Gwangju Institute of Science and Technology

2 Overview  Texturing is a process that takes a surface and modifies its appearance at each location using some images, functions, or other data source.  Huge modeling, memory and speed savings are obtained by combining images and surfaces.  Color image texturing also provides a way to use photographic images and animations on surfaces.

3 Generalized Texturing  Texturing is a technique for efficiently modeling the surface’s properties.  Generalized Texture Pipeline

3 Generalized Texturing

4  A location in space is the starting point for the texturing process.  It is more often in the model’s frame of reference.  When the model moves, the texture moves along with it.  Projector Function  The goal of the projector function is to generate texture coordinates.  It is a function to give parameter space values, which will be used for accessing the texture.  It typically works by converting a three-dimensional point in space into texture coordinates.: projection of a point in 3D to a point in 2D.  Spherical, cylindrical and planar projections  Natural projections

5 Generalized Texturing  Projector Function  Spherical projection casts points onto an imaginary sphere centered around some point.  Cylindrical projection computes the u texture coordinate the same as spherical projection, with the v texture coordinate computed as the distance along the cylinder’s axis.  The planar projection is like an x-ray slide projector, projecting along a direction and applying the texture to all surfaces. It uses orthographic projection.

6 Generalized Texturing  Corresponder Functions  They convert parameter-space values to texture-space locations.  They provide flexibility in applying textures to surfaces.  An optional matrix transformation: translate, rotate, scale, shear and even project the texture on the surface.  Another class of corresponder functions controls the way an image is applied.  Determine the behavior when values are outside of the range.  Wrap, mirror, clamp, border.

7 Generalized Texturing  Once the texture values have been retrieved, they may be used directly or further transformed.  The resulting values are used to modify one or more surface attributes.  Almost all real-time systems use Gouraud shading, meaning that only certain values are interpolated across a surface.  So these are the only values that the texture can modify.  We modify the RGB result of the lighting equation, since this equation was evaluated at each vertex and the color is then interpolated.

8 Generalized Texturing  Combine Functions or Texture Blending Operations: They glue an image texture onto a surface.  Replace: simply replace the original surface color with the texture color.  Decal: like replace, but when an alpha texture value is available, the texture is blended with the underlying color but the original color is not modified.  Modulate: multiply the surface color by the texture color.

9 Generalized Texturing  Summary with a brick wall texture example.  A modeler sets the (u,v) parameter values once in advance for the wall model vertices.  The texture is read into the renderer, and the wall polygons are sent down the rendering pipeline.  A white material is used in computing the illumination at each vertex.  This color and (u,v) values are interpolated across the surface.  At each pixel, the proper brick image’s texel is retrieved and modulated by the illumination color and displayed.

10 Image Texturing  In image texturing, a two-dimensional image is effectively glued onto the surface of a polygon and rendered.  Issues  We have an image of size 256×256 and want to use it as a texture on a square.  Case 1: The projected square on the screen is roughly the same size as the texture.  Case 2: The projected square on the screen is larger than the texture.  Magnification  Case 3: The projected square on the screen is smaller than the texture.  Minification  Solution depends on what kind of sampling and filtering methods are used.

11 Magnification  Nearest Neighbor  The method takes the value of the nearest texel to each pixel center when magnifying, resulting in a blocky appearance.  One characteristic of this magnification technique is that the individual texels may become apparent.

12 Magnification  Bilinear Interpolation  Find the four neighboring texels and linearly interpolates in two dimensions to find a blended value for the pixel.  The result is blurrier.  Much of the jaggedness from using the nearest neighbor method has disappeared.

13 Magnification  Bilinear Interpolation

14 Minification  When a texture is minimized, several texels may cover a pixel’s cell.  To get a correct color value for each pixel, integrate the effect of the texels influencing the pixel.  However, it is difficult to determine precisely the exact influence of all texels near a particular pixel.  It is effectively impossible to do so perfectly in real time.

15 Minification  One method  Use the nearest neighbor.  It selects the texel which is visible at the very center of the pixel’s cell.  This filter may cause severe aliasing problems.  Temporal aliasing: only one of the many texels influencing a pixel is chosen to represent the surface.

16 Minification  Bilinear Interpolation  The same as the magnification filter.  It is only slightly better than the nearest neighbor approach for minification.  It blends four texels instead of using just one.  When a pixel is influenced by more than four texels, the filter soon fails and produces aliasing.

17 Minification  Increase the pixel’s sampling frequency or decrease the texture frequency.  The Nyquist limit.  We need to make sure that the texture’s signal frequency is no greater than half the sample frequency.  In general, for textures, there should be at most one texel per pixel to avoid aliasing.  All texture antialiasing algorithms  To preprocess the texture and create data structure that will help compute a quick approximation of the effect of a set of texels on a pixel.  A single sample will retrieve the effects of one or more texels.

18 Minification  Mipmapping  The most popular antialisaing method for textures.  The texture (level zero) is downsampled to a quarter of the original area.  Each new texel value is often computed as the average of the four neighbor texels.  The new, level-one texture is called a subtexture of the original texture.  The reduction is performed recursively until one or both of the dimensions of the texture equals 1 texel.

19 Minification  Mipmapping  The goal is to determine roughly how much of the texture influences the pixel.  We compute d for the mipmap.  The intent of computing the coordinate d is to determine where to sample along the mipmap’s pyramid axis.  A pixel-to-texel ratio is at least 1:1 to achieve the Nyquist rate.  As the pixel cell comes to include more texels, d increaes.  A smaller, blurrier version of the texture is accessed.

20 Minification  Mipmapping  The result of mipmapping is that, instead of trying to sum all the texels which affect the pixel individually, precombined sets of texels are access and interpolated.  Causes overblurring.

21 Minification  Ripmapping  To avoid some or all of the overblurring  The idea is to extend the mipmap to include downsampled rectangular areas as subtextures that can be accessed.  Need four coordinates for access

22 Minification  Summed-Area Table  Create an array that is the size of the texture but contains more bits of precision for the color stored.  At each location in this array, one must compute and store the sum of all the corresponding texture’s texels in the rectangle formed by this location and texel (0,0).  During texturing, the pixel cell’s projection onto the texture is bound by a rectangle.  The summed-area table is then accessed to determine the average color of this rectangle, which is passed back as the texture’s color for the pixel.

23 Minification  Unconstrained Anisotropic Filtering  For current graphics hardware, the most common method to further improve texture filtering is to reuse existing mipmap hardware.  The basic idea  The pixel cell is back-projected.  This quadrilateral on the texture is then sampled a number of times.  The samples are combined.  Each mipmap sample has a location and a squarish area associated with it.  Instead of using a single mipmap sample to approximate this quad’s coverage, the algorithm uses a number of squares to cover the quad.

24 Minification  Unconstrained Anisotropic Filtering A line of anisotropy is formed between the longer sides.

25 Multipass Texture Rendering  The various parts of the lighting equation can be evaluated in separate passes, with each successive pass modifying the previous results.  Motion blur, antialiasing, shadows, etc.  Example  To have the diffuse color modulated by a texture and want the specular highlight to be unmodified by the texture.  In the first pass, compute and interpolate the diffuse illumination contribution and modulate it by the texture.  Then compute and interpolate the specular part and render the scene again.  This result then would be added to the existing diffusely lit, textured image.

26 Multipass Texture Rendering  The basic idea behind multipass rendering is that each pass computes a piece of the lighting equation and the frame buffer is used to store intermediate results.  On the fastest machine, up to 10 passes are done on some objects to render a single frame.  Passes 1-4 : accumulate bump map  Pass 5 : diffuse lighting  Pass 6 : base texture with specular component  Pass 7 : specular lighting  Pass 8 : emissive lighting  Pass 9 : volumetric/atmospheric effects  Pass 10: screen flashes

27 Multitexturing  Most graphics hardware today allows two or more textures to be applied in a single rendering pass. -> Multitexturing.  To combine the results of these texture accesses, a texture blending cascade is defined that is made up of a series of texture stages, or texture units.  The first texture stage combines two texture values, typically RGB and alpha, and this result is then passed on to the next texture stage.  Second and successive stages blend another texture’s or interpolant’s values with the previous result.

28 Texturing Methods  We will cover various other forms of texturing beyond gluing simple color images onto surfaces.  Alpha blending in texturing  Reflections via environment mapping  Rough surface simulation using bump mapping  Etc.

29 Alpha Mapping  The alpha value can be used for many interesting effects.  Decaling  You wish to put a picture of a flower on a teapot.  By properly setting the decal texture’s alpha, you can replace or blend the underlying surface with the decal.  By assigning an alpha of 0 to a texel, you make it transparent so that it has no effect.  Making cutouts  You make a decal image of a tree, but you do not want the background of this image to affect the scene at all.  If an alpha is found to be fully transparent, the textured surface itself does not affect that pixel.

30 Alpha Mapping  The alpha value can be used for many interesting effects.  Combining alpha blending and texture animation can produce convincing special effects  Flickering torches, plant growth, explosions, atmospheric effects, etc.

31 Light Mapping  For static lighting in an environment, the diffuse component on any surface remains the same from any angle.  Because of this view-independence, the contribution of light to a surface could be captured in a texture structure attached to a surface.  By using a separate, pre-computed texture that captured the lighting contributions, and multiplying it with the underlying surface, one can achieve Phong-like shading.

32 Light Mapping  If the lighting will never change, or will only change in overall brightness, the light texture can simply be multiplied by the surface’s material texture during the modeling stage, and the single resulting texture can be used.  Using textures to represent shadows and projective textures: both are related with light maps.

33 Gloss Mapping  Not all objects are uniformly shiny over their surface.  This can be simulated by using a technique called gloss mapping and the texture that makes this happen is called a gloss map.  A gloss map is a texture that varies the contribution of the specular component over the surface.  The key idea is that all material properties can be supplied by textures rather than by constants or per- vertex values.

34 Environment Mapping  Environment Mapping (EM), also called reflection mapping, is a simple yet powerful method of generating approximations of reflections in curved surfaces.  All EM methods start with a ray from the viewer to a point on the reflector.  This ray is then reflected with respect to the normal at that point.  Instead of finding the intersection with the closest surface, EM uses the direction of the reflection vector as an index to an image containing the environment.

35 Environment Mapping  The steps of an EM algorithm are  Generate or load a two-dimensional image representing the environment.  For each pixel that contains a reflective object, compute the normal at the location on the surface of the object.  Compute the reflection vector from the view vector and the normal.  Use the reflection vector to compute an index into the environment map that represents the objects in the reflection direction.  Use the texel data from the environment map to color the current pixel.

36 Cubic Environment Mapping  Far and away the most popular EM method implemented in modern graphics hardware, due to its speed and flexibility.  The cubic environment map is obtained by placing the camera in the center of the environment and then projecting the environment onto the sides of a cube positioned with its center at the camera’s location.  The images of the cube are then used as the environment map.  In practice, the scene is rendered six times with the camera at the center of the cube, looking at each cube face with a 90-degree view angle.

37 Cubic Environment Mapping  Cubic maps have no singularities, and are view- independent. They can be used for any view direction.

38 Sphere Mapping  The texture image is derived from the appearance of the environment as viewed orthographically in a perfectly reflective sphere.  This texture is called a sphere map.  One way to make a sphere map of a real environment is to take a photograph of a shiny sphere.  Sphere map textures for synthetic scene can be generated using ray tracing or by warping the images generated for a cubic environment map.

39 Lighting Using Environment Mapping  An important use of EM techniques is generating specular reflections and refractions.  Gouraud shading can miss highlights (reflections of lights).  EM can solve this problem by representing the lights in the texture.  We can simulate highlighting on a per-pixel basis for any number of lights at a fixed cost.

40 Lighting Using Environment Mapping  Recursive reflections of objects in a scene can be performed using EM.  Compute only one environment map per frame using the environment maps from the previous frame.

41 Bump Mapping  Bump mapping is a technique that makes a surface appear uneven in some manner: bumpy, wrinkled, wavy, etc.  Bump maps can simulate features that would otherwise take many polygons to model.  The basic idea is that instead of using a texture to change a color component in the illumination equation, we access a texture to modify the surface normal.  The geometric normal of the surface remains the same. We merely modify the normal used in the lighting equation.  We perform changes on the surface normal, but the surface itself remains smooth in the geometric sense.

42 Bump Mapping  Method 1(offset vector bump map or offset map): Store in a texture two signed values, b u and b v, at each point.  These two vectors are added to the normal to change its direction.  Method 2: Use a heightfield to modify the surface normal’s direction.  The heightfield is used to derive u and v signed values.

43 Bump Mapping  Two drawbacks  The illusion breaks down around the silhouettes of objects.  The viewer notices that there are no real bumps, just smooth outlines.  The bumps do not cast shadows onto their own surface, which can look unrealistic.  More advanced real-time rendering methods can be used to provide self-shadowing effects.

44 Bump Mapping  Emboss Bump Mapping  It is a way to give a chiseled look to an image.  Render the surface with the heightfield applied as a diffuse monochrome texture.  Shift all the vertex (u,v) coordinates in the direction of the light.  Render this surface with the heightfield again applied as a diffuse texture, subtracting from the first-pass result. This gives the emboss effect.  Render the surface again with no heightfield, diffusely illuminated and Gouraud-shaded. Add this shaded image to the result.

45 Bump Mapping  Dot Product Bump Mapping  Instead of storing heights or slopes, the actual normals for the surface are stored as (x,y,z) vectors in a normal map.  The bump texture, which consists of normals, is then combined with the interpolated light vector at each pixel.  These are combined by taking their dot product, which is a special texture-blending function provided precisely for this purpose.

46 Bump Mapping  Environment Map Bump Mapping (EMBM)  One way to give the appearance of bumpiness to a shiny surface.  The idea is to perturb (u,v) environment-mapping coordinates by u and v differentials found in the bump texture.  This gives the effect of wobbling the reflection vector, thereby distorting the look of the reflected surface.

47 Bump Mapping