1. Real-Time Rendering COMS 6998-3, Lecture 9

2. Real-Time Rendering Demo Motivation: Interactive rendering with complex natural illumination and realistic, measured BRDFs

3. Complex Light Transport

4. Challenges Illumination complexity Material (BRDF)/view complexity Transport complexity (shadows, interreflection)

5. Reflection Maps Blinn and Newell, 1976

6. Environment Maps Miller and Hoffman, 1984

7. Environment Maps Interface, Chou and Williams (ca. 1985)

8. Environment Maps Cubical Environment Map 180 degree fisheye Photo by R. Packo Cylindrical Panoramas

9. Reflectance Maps Reflectance Maps (Index by N) Horn, 1977 Irradiance (N) and Phong (R) Reflection Maps Hoffman and Miller, 1984 Mirror Sphere Chrome SphereMatte Sphere

10. Irradiance Environment Maps Incident Radiance (Illumination Environment Map) Irradiance Environment Map R N

11. Assumptions Diffuse surfaces Distant illumination No shadowing, interreflection Hence, Irradiance is a function of surface normal

12. Diffuse Reflection Radiosity (image intensity) Reflectance (albedo/texture) Irradiance (incoming light) × = quake light map

13. Analytic Irradiance Formula Lambertian surface acts like low-pass filter 0 01 2

14. 9 Parameter Approximation -201 2 0 1 2 Exact image Order 0 1 term RMS error = 25 %

15. 9 Parameter Approximation -201 2 0 1 2 Exact image Order 1 4 terms RMS Error = 8%

16. 9 Parameter Approximation -201 2 0 1 2 Exact image Order 2 9 terms RMS Error = 1% For any illumination, average error < 3% [Basri Jacobs 01]

17. Comparison Incident illumination 300x300 Irradiance map Texture: 256x256 Hemispherical Integration 2Hrs Irradiance map Texture: 256x256 Spherical Harmonic Coefficients 1sec

18. Rendering Irradiance approximated by quadratic polynomial Surface Normal vector column 4-vector 4x4 matrix (depends linearly on coefficients L lm )

19. Hardware Implementation Simple procedural rendering method (no textures) Requires only matrix-vector multiply and dot-product In software or NVIDIA vertex programming hardware

20. Reflectance Space Shading Cabral, Olano, Nemec 1999

21. OLF Parameterization N L N V

22. N V N V R Reparameterize by reflection vector

23. OLF Parameterization Captures structure of BRDF (and hence OLF) better Reflective BRDFs become low-dimensional N V N V R Reparameterize by reflection vector

24. OLF Structure: Lafortune View y Single 2D reflection map no longer sufficient But variation with viewing direction is slow View x

25. OLF Structure: Lafortune View maps vary slowly Reflection x Reflection y View y View x

26. A Simple Factorization View x View y Reflection y Reflection x *

27. Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments Peter-Pike Sloan, Microsoft Research Jan Kautz, MPI Informatik John Snyder, Microsoft Research

28. Previous Work – Where We Fit Transport Complexity Lighting simple shadows point lights full env. map inter- reflections single area light ??[Moeller02] [Blinn76] [Heidrich00] [Crow77] [Ashikhmin02] our technique Non-Interactive[Matusik02]

29. Motivation Better light integration and transport dynamic, area lights self-shadowing interreflections For diffuse and glossy surfaces At real-time rates point light area light area lighting, no shadows area lighting, shadows

30. light 2D example, piecewise constant basis, shadows only Diffuse Self-Transfer Preprocess Project Light light Rendering = = =

31. Precomputation Basis 16 Basis 17 Basis 18 illuminateresult......

32. Diffuse Transfer Results No Shadows/Inter Shadows Shadows+Inter No Shadows/Inter Shadows Shadows+Inter

33. Arbitrary BRDF Results Other BRDFs Spatially Varying Anisotropic BRDFs

34. Shadow Mapping with Today’s OpenGL Hardware CEDEC 2001 Tokyo, Japan Mark J. Kilgard Graphics Software Engineer NVIDIA Corporation

35. Motivation Shadows increase scene realism Real world has shadows More control of the game’s feel dramatic effects spooky effects Other art forms recognize the value of shadows But yet most games lack realistic shadows

36. Common Real-time Shadow Techniques Shadow volumes Light maps Projected planar shadows Hybrid approaches

37. Problems Mostly tricks with lots of limitations Projected planar shadows well works only on flat surfaces Stenciled shadow volumes determining the shadow volume is hard work Light maps totally unsuited for dynamic shadows In general, hard to get everything shadowing everything

38. Shadow Mapping Image-space shadow determination Lance Williams published the basic idea in 1978 By coincidence, same year Jim Blinn invented bump mapping (a great vintage year for graphics) Completely image-space algorithm means no knowledge of scene’s geometry is required must deal with aliasing artifacts Well known software rendering technique Pixar’s RenderMan uses the algorithm Basic shadowing technique for Toy Story, etc.

39. References Important SIGGRAPH papers Lance Williams, “Casting Curved Shadows on Curved Surfaces,” SIGGRAPH 78 William Reeves, David Salesin, and Robert Cook (Pixar), “Rendering antialiased shadows with depth maps,” SIGGRAPH 87 Mark Segal, et. al. (SGI), “Fast Shadows and Lighting Effects Using Texture Mapping,” SIGGRAPH 92

40. Shadow Mapping Concept (1) Depth testing from the light’s point-of-view Two pass algorithm First, render depth buffer from the light’s point-of-view the result is a “depth map” or “shadow map” essentially a 2D function indicating the depth of the closest pixels to the light This depth map is used in the second pass

41. Shadow Mapping Concept (2) Shadow determination with the depth map Second, render scene from the eye’s point-of-view For each rasterized fragment determine fragment’s XYZ position relative to the light this light position should be setup to match the frustum used to create the depth map compare the depth value at light position XY in the depth map to fragment’s light position Z

42. Shadow Mapping Concept (3) The Shadow Map Comparison Two values A = Z value from depth map at fragment’s light XY position B = Z value of fragment’s XYZ light position If B is greater than A, then there must be something closer to the light than the fragment then the fragment is shadowed If A and B are approximately equal, the fragment is lit

43. Visualizing Shadow Mapping A fairly complex scene with shadows the point light source

44. Visualizing Shadow Mapping Compare with and without shadows with shadows without shadows

45. Visualizing Shadow Mapping The scene from the light’s point-of-view FYI: from the eye’s point-of-view again

46. Visualizing Shadow Mapping The depth buffer from the light’s point-of-view FYI: from the light’s point-of-view again

47. Visualizing Shadow Mapping Projecting the depth map onto the eye’s view FYI: depth map for light’s point-of-view again

48. Visualizing Shadow Mapping Comparing light distance to light depth map Green is where the light planar distance and the light depth map are approximatel y equal Non-green is where shadows should be

49. Visualizing Shadow Mapping Scene with shadows Notice how specular highlights never appear in shadows Notice how curved surfaces cast shadows on each other

50. Hardware Shadow Map Filtering “Percentage Closer” filtering Normal texture filtering just averages color components Averaging depth values does NOT work Solution [Reeves, SIGGARPH 87] Hardware performs comparison for each sample Then, averages results of comparisons Provides anti-aliasing at shadow map edges Not soft shadows in the umbra/penumbra sense

51. Hardware Shadow Map Filtering GL_NEAREST: blocky GL_LINEAR: antialiased edges Low shadow map resolution used to heightens filtering artifacts

52. Soft Shadows?

53. Conclusion Variety of techniques Shadows Complex illumination Complex materials Limited manipulability in most cases Cannot change lighting, geometry, viewpoint etc. Is it possible to have everything simultaneously?