1. Microflakes Radiation Transport and Volume Rendering Nelson Max, Tom Duff, and Ben Mildenhall Pixar Animation Studios 1

2. Outline Context at Pixar Prior related work Microflake theory for volume rendering of single and multiple light scattering Three methods of specifying the distribution of microflake normals; two are new Application to rendering leaves in trees Computer animations 2

3. Problem Render scenes defined by polygons with a lot of compexity in the distance, without storing every polygon, by summarizing the data. The polygon data for this forest required 257 Gb, while our summarized data required only 2.4 GB. 3

4. Sparse Voxel Octree (SVO) Heitz and Neyret, “Representing appearance and prefiltering subpixel data in Sparse Voxel Octrees”, High Performance Graphics (2012). We used this SVO method to store the sufaces of the trunk and branches of the tree, and I developed an enhancement using microflakes for the leaves. 4

5. Sparse Voxel Octree (SVO) 5

6. Recursively divide volumes that contain data into eight children (only four shown here). 6

7. 7

8. 8

9. 9

10. Find and save area-weighted average plane of the polygons clipped to each cell. Also average their colors. 10

11. Further average these average planes into the parent cells. For ray tracing, these planes are assumed to extend across the whole cell, so the object appears larger. 11

12. Further average these average planes into the parent cells. For ray tracing, these planes are assumed to extend across the whole cell, so the object appears larger. 12

13. Use ray differential (distance to neighboring rays) to decide which octree cells to use for rendering, so that when the object is farther away, larger cells are used. 13

14. If many small leaves are present, they usually do not have a closely fitting average plane, and one small leaf may grow to fill a volume cell. So it is better to represent the leaves with infinitesimal microflakes. 14

15. Basic microflake idea Convert polygons clipped to a cell into a volume cloud of infinitesimal microflakes with reflection and transmission properties similar to the polygon object, i. e. with the same total area and the same distribution of normals. 15

16. Participating media of microflakes The basic reference is Jacob, Arbee, Moon, Bala, and Marschner, “A radiative transfer framework for rendering materials with anisotropic structure” (2010). 16

17. Microflake Normal Distribution microflake area per unit volume at position with normals within solid angle We will find formulas for the extinction coefficient for how much the cloud of microflakes occludes the light passing through it, and the diffuse and specular inscattering coefficients, measuring how much light reflects from microflakes into a viewing direction. 17

18. These optical properties are then used for volume rendering the participating meduim along viewing rays, until the first opaque surface is hit. 18

19. Microflake projected area Subscripts on :o for “out”, direction to viewpoint i for “in”, direction to light source n for “normal” to microflake A microflake of area and normal projects to an area normal to the viewpoint direction. 19

20. Extinction coefficient = fraction of a unit area near x in a plane perpendicular to occluded by micro- flakes in a slab of a small thickness ds is the unit sphere is the fraction not occluded, that is, the transparency. 20

21. Transparency transparency betweenand 21

22. Transparency Differential Equation 22

23. Definition of radiance Radiance in direction is the flux of photons per unit time per unit area normal to per solid angle. 23

24. Definition of radiance Radiance in direction is the flux of photons per unit time per unit area normal to per solid angle. 24

25. Diffuse inscattering If the microflake has diffuse reflectivity and is illuminated from solid angle near direction with radiance, its radiance is where 25

26. Diffuse inscattering If the microflake has diffuse reflectivity and is illuminated from solid angle near direction with radiance, its radiance is where 26

27. Diffuse inscattering If the microflake has diffuse reflectivity and is illuminated from solid angle near direction with radiance, its contribution to the radiance at the front of the slab is. 27

28. Diffuse inscattering radiance diffusely inscattered from an infinitesimal slab of thickness 28

29. Specular inscattering A perfect mirror microflake with normal will reflect light from a source in direction to a viewpoint in direction when, where is the “halfway” unit vector. 29

30. Specular inscattering A microflake of area and normal projects to an area normal to viewing direction. So the reflected flux per unit area normal to direction is 30

31. Angular spread of from If varies near in the plane of and by an angle, and the illumination direction is fixed, then the relection direction will vary by. Accounting for the 3D solid angle,. 31

32. Specular inscattering The reflected flux per unit area normal to direction is But radiance in a direction is flux per unit area normal to per solid angle. So the specular inscattered radiance is The total inscattered radiance per length is 32

33. Differential equation for radiance Volume rendering integral 33

34. In the integral the factor so depends on the radiance at other points and in all other directions, making a solution difficult. Some solution methods are volume path tracing, volume photon mapping, finite elements, and the diffusion approximation. 34

35. Single scattering approximation Assume that the light from a single external source scatters only once in the participating medium, but account for the attenuation of the medium’s partial transparency from the light source to the scattering point, and from that point to the eye. Compute at each sample point on the viewing ray segment, or precompute it at each cell in the octree. Use it to compute 35

36. Computing the volume integral Assume the optical properties and are constant in each cell of a voxel grid or octree. Trace viewing rays incrementally through the grid, determinging these properties and the ray length in each intersected cell, and adding their effects to the volume rendering integral. 36

37. Approximating the volume integral T = 1; // global transparency L = 0; // global radiance for ( j = 0; j < n; ++j ) { t = exp( - ); // local transparency L = L + T ; T = T t ; } L = L + T ; * * * ** 37

38. Diffuse inscattering radiance diffusely inscattered from an infinitesimal slab of thickness 38

39. Excess inscattering Radiance diffusely inscattered from a slab of finite thickness 39

40. Correct transparency and opacity Rather than using 1 -, the correct transparency is. This is the area of the front slab face that is not covered. Therefore the opacity, or area of the front face that is covered, is 40

41. Average radiance or color The average color of this covered area is 41

42. Incorrect volume integral T = 1; L = 0; for ( j = 0; j < n; ++j ) { t = exp( - ); L = L + T ; T = T t ; } L = L + T ; * * * ** 42

43. Corrected volume integral code T = 1; L = 0; for ( j = 0; j < n; ++j ) { t = exp( - ); L = L + T (1 – t) / ; T = T t ; } L = L + T ; * * * * * 43

44. Three methods for approximating the distribution of microflake normals: A: Weighted mixture of a small number of representative directions, with variances, B: Spherical harmonics (SH) expansion, and C: The distribution of normals of an ellipsoid. Method C is described in detail in Heitz, Dupuy, Crassin, and Dachsbacher, “The SGGX Microflake Distribution” (ACM SIGGRAPH 2015). Flattened ellipsoids are used for surfaces, and long thin ones for fur, hair, or pine needles. 44

45. Sub-outline For each of the two methods A and B, I will describe: 1.How to convert polygon data to microflakes 2.How to compute extinction coefficient 3.How to compute diffuse inscattering 4.How to compute specular inscattering 45

46. Method A: weighted mixture The distribution of normals is approximated by an area-weighted sum of representative unit normals : Clip polygons to octree leaves, and add new normal representatives that are far from existing ones. Then do several weighted k-means passes through the polygon data. In each pass, group each normal with the closest representative, and redefine the representative to be the area-weighted mean of its group. 46

47. Estimate the variance of each representative from the amount that its length is less than one, by the method from Toksvig “Mipmapping normal maps” (2005). Use the same k-means method to find weighed representatives and variances for the internal octree nodes, from the representatives of their children. 47

48. Estimate the variance of each representative from the amount that its length is less than one, by the method from Toksvig “Mipmapping normal maps” (2005). Use the same k-means method to find weighed representatives and variances for the internal octree nodes, from the representatives of their children. 48

49. Extinction coefficient 49

50. Diffuse inscattering 50

51. Specular inscattering where is the Beckmann specular function, and is the microfacet RMS slope from the group variance associated with representative. Then where is halfway between and. 51

52. Sampling scattering direction Sample a representative direction with probability proportional to its area weight. Choose a random between 0 and 1, and choose the representative by the interval with. 52

53. Method B: Spherical Harmonics The spherical harmonics functions form an orthonormal basis for functions on the unit sphere, and are polynomials in x, y, and z of degree at most L. where A polygon fragment in an octree leaf cell with area and normal corresponds to so increment by 53

54. Extinction coefficient Expand in spherical harmonics: Since the are orthonormal so 54

55. Diffuse inscattering Expand and in spherical harmonics: so where is the tabulated triple product integral 55

56. Specular inscattering 56

57. Results with cottonwood trees Polygonsk-means, k = 3 RMS.086 SH, L = 4 RMS.063 Ellipsoids RMS.067 57 RMS value is the root mean square color difference from an image ray traced from all original polygons.

58. Results with lodgepole pine trees Polygonsk-means, k = 3 RMS.056 SH, L = 4 RMS.050 Ellipsoids RMS.057 58

59. Results with hair Polygonsk-means, k = 3 RMS.117 SH, L = 4 RMS.105 Ellipsoids RMS.101 59

60. Shadows 60

61. Shadows Ray traced shadows, 61

62. Shadows Ray traced shadows, 62

63. Shadows Ray traced shadows, Precomputed shadows, 63

64. Shadows Ray traced shadows, Precomputed shadows, Ray casting, 64

65. Shadow results, using k-means, k = 3 Ray traced, 346.2 sec. Precomputation, 13.6 sec. Ray tracing, 60.1 sec. 65

66. Path tracing result, using SH, L = 4 Diffuse reflection and diffuse transmission Diffuse transmission only 66