1. Normal Map Compression with ATI 3Dc™ Jonathan Zarge jzarge@ati.com ATI Research Inc.

2. Normal Map Compression with ATI 3Dc™ Overview Normal mapping Normal map compression techniques DXT compression ATI 3Dc™ Swizzled DXT5 compression

3. Normal Map Compression with ATI 3Dc™ Normal Mapping Increase visual realism Reduce geometric size of models Create detail by simulating geometry More efficient than dense geometry Textures provide lighting (normal) information Used in Half-Life 2, Doom 3, and Far Cry

4. Normal Map Compression with ATI 3Dc™ Car Paint Demo

5. Normal Map Compression with ATI 3Dc™ Traditional Lighting Model n0 n1 n2 Lighting values calculated at vertices and then interpolated across triangle Normals interpolated across triangle and lighting values calculated at each pixel

6. Normal Map Compression with ATI 3Dc™ Normal Map Lighting 3 component normal map texture Lighting values calculated at each pixel using normal read from normal map (nx,nx,nz) nx nx nz

7. Normal Map Compression with ATI 3Dc™ World Space vs. Tangent Space Triangle normal = (-0.6,-0.3,0.74) z y x Triangle normal can be any normalized vector x y z Triangle normal = (0,0,1) Coordinate space of normal defined by triangle orientation z component of normal map texel must be positive

8. Normal Map Compression with ATI 3Dc™ Example Tangent Space Normal Maps Tangent space normal maps can be efficiently compressed

9. Normal Map Compression with ATI 3Dc™ Generating Normal Maps

10. Normal Map Compression with ATI 3Dc™ Normal Mapping Drawbacks Low angle views distorted Silhouette edges do not have detail High texture bandwidth required Large amount of texture memory DXT compression not optimal for normal maps

11. Normal Map Compression with ATI 3Dc™ Benefits of Normal Map Compression More detailed normal maps More normal maps Memory and bandwidth can be diverted to other resources Smaller media size

12. Normal Map Compression with ATI 3Dc™ Normal Map Compression Techniques Lower precision texture formats Texture formats with fewer channels Denormalization Palletization Direct DXT compression ATI 3Dc™ Swizzled DXT5

13. Normal Map Compression with ATI 3Dc™ DXT Compression Designed to compress color data No suited for arbitrary data –One channel dependent on another Lossy format –Image divided into 4x4 pixel blocks –16 possible colors in block –Only 3 or 4 colors in compressed data Local similarity in small region enables high quality color compression

14. Normal Map Compression with ATI 3Dc™ DXT Compression Color Block Image divided into 4x4 blocks Two 16-bit 5.6.5 colors stored representing endpoints of linear color-ramp One or two interpolated colors calculated 16 two-bit indices stored representing index into color ramp Effective bit-rate: – 64 (2*16 +16*2) / 16 = 4 bits per pixel

15. Normal Map Compression with ATI 3Dc™ DXTC Color Compression BLACK CYAN WHITE MAGENTA BLUE GREEN YELLOW RED Color0 Color1 Interpolated Colors

16. Normal Map Compression with ATI 3Dc™ DXTC Color Decompression For the four color case, given color[0], color[1], and index[1..15]: // calculating interpolated colors color[2] = (2*color[0] + color[1] + 1)/3 color[3] = (color[0] + 2*color[1] + 1)/3 // decompressed texel values for n = 0 to 15 texel[n] = color[index[n]]

17. Normal Map Compression with ATI 3Dc™ DXT5 Compression Alpha Block Like color, image divided into 4x4 blocks Two 8-bit scalars stored representing endpoints of linear alpha-ramp 4 or 6 intermediate alpha values calculated 16 three-bit indices stored representing index into alpha ramp Effective bit-rate: – 64 (2*8 +16*3) / 16 = 4 bits per pixel

18. Normal Map Compression with ATI 3Dc™ DXTC Alpha Decompression For the 8 alpha value case, given alpha[0], alpha[1], and index[1..15]: // calculating interpolated alpha values alpha[2] = (6*alpha[0] + 1*alpha[1] + 3)/7 alpha[3] = (5*alpha[0] + 2*alpha[1] + 3)/7.. alpha[7] = (1*alpha[0] + 6*alpha[1] + 3)/7 // decompressed texel values for n = 0 to 15 texel[n] = alpha[index[n]]

19. Normal Map Compression with ATI 3Dc™ Using DXT Compression for Normal Maps Not a good idea! Block artifacts –Normals describe curved surfaces, not lines –Especially visible in diagonal gradients Colors stored in 5.6.5 format –Fine detail lost Errors are amplified for specular lighting Storage is inefficient because normalized vectors can be stored in two values

20. Normal Map Compression with ATI 3Dc™ ATI 3Dc™ Texture Format Two independent channel texture format Exposed in D3D via the FOURCC code “ATI2” Composed of 2 DXT5 alpha blocks Extremely useful for normal textures Compression: –4:1 for 32 bit textures –2:1 for 16 bit textures

21. Normal Map Compression with ATI 3Dc™ Ruby Demo

22. Normal Map Compression with ATI 3Dc™ ATI 3Dc™ Texture Format Decompression accomplished in hardware –Almost no impact on performance Can be used for two unrelated scalars –e.g. shininess, refractive index, transparency Library for compressing to 3Dc™ available from ati.com/developerati.com/developer Available in OpenGL using the GL_ATI_texture_compression_3dc extension

23. Normal Map Compression with ATI 3Dc™ ATI 3Dc™ Texture Format

24. Normal Map Compression with ATI 3Dc™ Using ATI 3Dc™ with Normal Maps x and y (of normal) stored in the texture z value calculated by x 2 + y 2 + z 2 = 1 – z = ±sqrt(1 – x 2 – y 2 ) –Only positive root used because normal is in tangent space –Calculation done in pixel shader –Can also be derived by a texture lookup Single channel z component texture

25. Normal Map Compression with ATI 3Dc™ Using ATI 3Dc™ Pixel shader 2.0 code: ; scale, bias, half, one def c0, 2.0f, -1.0f, 0.5f, 1.0f. texld r1, t1, s1 ;normal map mad r1, r1, c0.x, c0.y ; scale/bias to -1,1 ; Compute third component dp2add r1.z, r1, -r1, c0.w ; 1-x*x-y*y rsq r1.z, r1.z ; 1 / sqrt(1-x*x-y*y) rcp r1.z, r1.z ; sqrt(1-x*x-y*y)

26. Normal Map Compression with ATI 3Dc™ Using ATI 3Dc™ Pixel shader code (HLSL): sampler bump_map; float4 ps_main( PS_INPUT_STRUCT psInStruct ) :COLOR0 { // Unpack two component bump map float3 bump = tex2D( bump_map, psInStruct.bump_map ); // Put x and y in -1..+1 range and compute z bump = ( bump * 2.0f ) - 1.0f; bump.z = sqrt(1 - dot(bump.xy, bump.xy));.

27. Normal Map Compression with ATI 3Dc™ ATI 3Dc™ Compression Comparison

28. Normal Map Compression with ATI 3Dc™ ATI 3Dc™ Normal Compression Sample

29. Normal Map Compression with ATI 3Dc™ Drawbacks of ATI 3Dc™ Can only represent tangent space normal maps Cannot represent high precision normal maps –Large areas of low curvature (car hood) may require 16 bits per component More pixel shader instructions required Not available on all hardware

30. Normal Map Compression with ATI 3Dc™ Normal Map Compression Using Swizzled DXT5 For any hardware with at least ps 1.4 support Store x and y in alpha and green channels of the texture map Shader similar to 3Dc™ with one extra pixel shader instruction Results better than using DXTC directly but not as good as 3Dc™

31. Normal Map Compression with ATI 3Dc™ Using DXT5 Compression Pixel shader 2.0 code: ;scale, bias, half, one def c0, 2.0f, -1.0f, 0.5f, 1.0f. texld r1, t1, s1 ;normal map mov r1.x, r1.a ; restore x from alpha mad r1, r1, c0.x, c0.y ; scale/bias to -1,1 ; Compute third component dp2add r1.z, r1, -r1, c0.w ; 1-x*x-y*y rsq r1.z, r1.z ; 1 / sqrt(1-x*x-y*y) rcp r1.z, r1.z ; sqrt(1-x*x-y*y)

32. Normal Map Compression with ATI 3Dc™ DXT5 Normal Map Compression RenderMonkey Example

33. Normal Map Compression with ATI 3Dc™ Summary Normal Mapping great feature Directly using DXT compression not great for normal textures Compression with 3Dc™ lowers the cost normal mapping 3Dc™ available in D3D and OpenGL Swizzled DXT5 compression reasonable alternative for older hardware

34. Normal Map Compression with ATI 3Dc™ QUESTIONS? Email me jzarge@ati.com or devrel@ati.comjzarge@ati.comdevrel@ati.com More information available at ati.com/developerati.com/developer Please go see Chris Oat’s talk on “Advanced Character Rendering” Saturday, 10:25am