2. Physically Based Rendering Rendering that accurately reproduces the physical behavior of light –Laws of physics that represent the behavior of light (electromagnetic waves) Geometrical Optics Wave Optics –Wave Equation »Maxwell’s equations –Scattering Equation »Compton scattering »Rayleigh scattering »Mie scattering »Raman scattering »Etc…

3. Equation covered in this talk The Rendering Equation –Equation that represents the light behavior in CG Rendering was introduced by [Kajiya 86] takes into account elements of geometrical optics Ignores elements of wave optics –Volumetric rendering –Sub-surface scattering –Fluorescent effects –Interference effects –Etc…

4. Motivation Achieve physically correct rendering by solving the rendering equation –Apologies in advance for the ton of equations The goal of this talk is not about vaguely implementing functions and equations in the rendering field It is about physically understanding them For ease of comprehension, in the slides, the  (outgoing direction) and  '(incident direction) are not written with positive or negative signs. When implementing these formulas, replace them by the right ones. Caution!

5. The Rendering Equation The equation that calculates L o ( outgoing light at direction  from a given point x Different functions can be defined for each wavelength λ with different physical properties What does this equation represent?

6. Radiance? Outgoing light Radiance outgoing to the direction  at given position x

7. Radiance  (radiant flux), the quantity of light that passes through an area on a unit sphere d  (solid angle) and a unit surface area dA

8. Irradiance  (radiant flux) incident on a unit area dA

9. Radiant flux Time derivative of the light energy –W (watt) = J/s – Q represents the light energy (radiant energy; J)

10. Rendering To calculate the incoming light energy received by a pixel of a screen (or sensor) –or received by a retina cell

11. Assume that radiance L ( x,  ) on a pixel is given –Rendering at frame t (time) at 60Hz becomes… Try to calculate L(x,)L(x,)

12. Solution for radiant energy from radiant flux Solution for irradiance from radiance Try to calculate Solution for radiant flux from irradiance Integrate the incident light (irradiance) over the pixel domain Integrate the incident light (radiance) over the hemisphere

13. Aperture Since the result is the sum of incident light from entire hemisphere, the image gets blurred –Introduce “aperture” as a solution 1 (where direction  is inside the aperture at x ) 0 (where direction  is outside the aperture at x ) Substitute

14. Without considering the aperture Generally, real-time rendering doesn’t take into account the aperture is a Dirac’s delta function where the value becomes 1 when direction  equals to direction from x to the center of the virtual aperture Effects of aperture… (with the same intensity of light source) Smaller aperture size for shaper image (but the image gets darker) Larger aperture size for brighter image (but the image gets blurrier)

15. Introduction of a Lens f refraction is a function that represents a thin lens bending light from direction  to direction  lens

16. Solution for outgoing radiance Emitted radiance in direction  from given position x Emitted light amount in direction  which is just one part of the total self emitted light from position x

17. Solution for outgoing radiance Reflected radiance in direction  from given position x Reflected light amount in direction  from the total incident light amount at position x

18. Then… Outgoing light (radiance) in direction  at given position x Light amount in direction  for a total light amount of self emission from position x Reflected light amount in direction  for a total incident light amount at position x = +

19. Calculate L e (Self emission) As you like –Use an arbitrary function Texture? Animation curve? Illumination model? –It represents a light source

20. Solution for L r (reflection) The sum of reflected light for direction  from incident light at position x –Incident light? –Behavior of reflection?

21. L r =… Integrate the formula over hemisphere around the normal  ’

22. L r =… Incident light amount (radiance) from direction  ’ at position x Geometric effect for incident light at position x ( n is a normal at x )

23. L r =… A function that represents the ratio between the amount of reflected light for direction  and the amount of incident light from direction  ’ at position x = Bi-directional Reflectance Distribution Function (BRDF)

24. Derive BRDF(1) Differentiate both side by  ’ Transpose

25. Derive BRDF(2) Substitute the definition of radiance for L i Simplify Where

26. As a result… The function that represents the ratio between the reflected light amount (radiance) for direction  and the incident light amount (irradiance) from direction  ’ at a given position x Reflected radiance for  Incident irradiance from  ’

27. Utilize BRDF at a given position x –(at a given wavelength ) from a given direction  ’ When light comes for a given direction  Reflected light amount can be obtained

28. Physical property of BRDF (1) For all  ’ The law of conservation of energy The sum of reflected light must not exceed the amount of incoming light "Practical Implementation of physically based shading models at tri-Ace" SIGGRAPH 2010

29. Physical property of BRDF (2) Helmholtz reciprocity The reflectance stays same when incident direction and reflected direction are swapped

30. Swap L and E Verify reciprocity (1) With ad-hoc Blinn (1) Describe as a shading model

31. Swap L and E Verify reciprocity (2) With ad-hoc Blinn (2) Describe as a shading model

32. Solve the rendering equation Since radiance value is recursively included on both side, to find an analytic solution, it is necessary to look for a solution to the infinite series. Algorithms to find an approximated solution Path tracing, MLT and so on…

33. Solve it with Lambert(Assumptions) BRDF (Lambert) x : Shading position x p : Sensor pixel position n x : Surface normal at x n p : Sensor normal at x p A p : Area of sensor pixel A x : Shading area at x (eye vector) (light vector) L a : Distance from sensor to aperture(focal length) A a : Area of apperture x a : Aperture position

34. Solve it with Lambert(1) Substitution BRDF in the rendering equation Self radiation is not considered Substitute the definition of radiance for L i Acquisition reflected radiance from x Simplify …Eq. (1)

35. Solve it with Lambert (2) Acquisition incident radiant flux to a pixel sensor …Eq.(2)

36. Solve it with Lambert (3) Furthermore… dA x represents the region where x is projected from dA p, the pixel surface …Eq.(3)

37. Solve it with Lambert (4) Moreover… dE becomes the solid angle that represents the area of dA x seen from x …Eq.(4)

38. Solve it with Lambert (5) Eq.2 Eq.1Eq.3 Eq.4 Substitute the equations in Eq.2 Simplify

39. Solve it with Lambert (6) Simplified equation Since L a 2 is a constant … Eq.(5)

40. Solve it with Lambert (7) Furthermore… dA a, area of the aperture viewed from pixel in the view direction, becomes a domain orthogonal to the view vector

41. Solve it with Lambert (8) Eq.(5) Substitute for the innermost integral in Eq.5 Simplify

42. Solve it with Lambert (9) Simplified equation Simplify more

43. Solve it with Lambert (10) Final result What does this equation mean?

44. Meaning of the equation Light Intensity is proportional to the dot product of the view vector and pixel normal elevated at the power of 4 ( cosine fourth law ) For a given aperture size, brightness is inversely proportional to the squared distance from sensor to aperture. (The relationship between focal length and F-stop) Light intensity is proportional to the size of aperture (The relation between F-stop and brightness) Amount of received light is proportional to the pixel area Light Intensity is 1/  independently of the view point (Lambertian). (It's doesn't mean that it is physically correct. Note that Lambert is rough approximation of the diffuse component) (The cosine fourth law or other view dependent components are not included in this factor) Pixel brightness is proportional to incident irradiance to x(A x )

45. Solve with shader Real-time ray-traced base algorithms are not realistic on current hardware –Partially available Only diffuse component Point-Based Global Illumination Screen-Space Photon Mapping Precomputed Radiance Transfer and derived ones And so on… How to solve the equation using shader based on rasterization rendering?

46. Radiance in shader How to deal with incident radiance in shader? –A function that represents incident radiance values from hemisphere –Texture? What is a radiance texture?

47. Radiance Texture A texture that stores radiance in each texel Because x (position) is 3 dimensions,  (orientation) is 2, if the equation is used as it is, a 5 dimensional texture is necessary If a light source is assumed to exist in infinite distance…?

48. Radiance Environment Map (REM) Environment map storing radiance Perform rendering using the radiance environment map and the rendering equation No need to consider x, because incident light doesn’t depend on the shading point Emission surface( dA ) always faces to the center of the (environmental) sphere, cos  always becomes 1 ∞

49. Generate REM (1) Generate REM as a cubemap –Ideally, REM would be rendered to a spherical render target –However for ease of implementation, render to a cubemap and compensate for the difference Conventional real-time rasterization rendering doesn’t store radiant energy or radiant flux in the render target –Directly store radiance computed at the shading pixel in the render target

50. Generate REM (2) Generate an REM as a cubemap –Make the general render target a cube map if it stores radiance (typically it does) Discretize

51. Generate REM (3) Since an environment map always illuminates the center of the sphere, the normal (n) of each emission surface (  A ) and the orientation of  becomes identical Solid angle (   ) is equivalent to  A  A represents the area of a texel in the cubemap texture

52. Calculate  A Incident light in the rendering equation is transported from the hemisphere centered around the shading point ( x ) –Each texel in the cubemap corresponds to a texel over the (hemi-)sphere –  A is equivalent to the area of the spherical quadrilateral that is the cubemap texel projected onto the sphere

53. Rendering w/REM(assumptions) Incident light BRDF (Lambert) No self-emission Following assumptions are used  ’ is calculated by projecting the coordinate ( face,s,t ) of the cubemap onto the sphere surface

54. Rendering using REM(1) Nothing Substitute in the rendering equation

55. Rendering using REM(2) Discretize Substitute the definition of texREM

56. Rendering using REM(3) Recall that when lighting using a REM Simplify

57. Rendering using REM(4) Using this radiance, –(ideal) Compute incident radiant flux for each pixel with the rendering equation –(typical) Store it directly in each pixel and use it similar to conventional rendering

58. Irradiance Environment Map Does this equation calculate irradiance? …similar?

59. Generate IEM The rendering equation of a Lambertian diffuse surface for one iteration can be calculated using an Irradiance Environment Map and multiplying by 1/ 

60. Optimization Solving the rendering equation is expensive –Need to solve high dimensional differential equation Lots of loops Image resolution is high Giga or Tera order of computation –Need to drastically reduce computation for real-time rendering Very rough approximation –A little sacrifice of physically correctness (not physically perfect) –Physically plausible rather than physically perfection

61. Integral decomposition Can this integral decompose like this? ≠

62. Optimize integral Use Fourier transform (Convolution Theorem) –For convolution (integral of product of two functions), –Apply Fourier transform to each function –Fourier transform of convolution becomes a product of Fourier transformed functions

63. Convolution theorem Advantage of convolution theorem –Drastic optimization using Fast Fourier Transform (FFT) Spherical Harmonics can be regarded as spherical Fourier transform Spherical Harmonics are easy to use for rendering FFT and IFFT could have bottleneck in some cases However, solving the rendering equation in real-time is still too expensive on current generation hardware

64. Basis transformation Use basis transformation –More optimization using orthogonal transformation –Reduce unnecessary coefficients for more optimization Still low frequency? Using PCA-related algorithms? Eventually, is it PRT? “Introduction of PRT for game programmers” [CEDEC ‘06]

65. More aggresively After all, –Does this really not work? –Evaluate what case this approximation works?

66. Verification (1) ? Verify the problem with discretization ? Verify

67. Verification (2) Suppose each function… Evaluate this

68. Verification (3) First termSecond term = ?

69. Verification (4) Doesn’t look so good How about dividing the decomposed first term by n ? Simplify

70. Conclusion Since this formula introduces errors, the decomposition becomes a good approximation as the error is close to zero Good approximation if dealing well with errors –Understand the introduced errors –Visually compare the approximated result to the ground truth (a.k.a. real world)

71. Wrap up It is naturally possible to achieve physically correct rendering by using the rendering equation –Because the integration is computed in the code, It is fair enough to correctly write down the required conditions in mathematical form –Neccesary calculations are not difficult Drastic optimization according to requirements by hardware specification –Compromised and rough approximation for some cases

72. Acknowledgements R&D department, tri-Ace, Inc. –Tatsuya Shoji –Satoshi Ishii –Takafumi Ohshima –Elliott Davis Thanks for the English version –Sébastien Lagarde, Marc Heng and Naty Hoffman

73. Questions? http://research.tri-ace.com