1. Frequency Domain Normal Map Filtering Charles Han Bo Sun Ravi Ramamoorthi Eitan Grinspun Columbia University

2. Normal Mapping (Blinn 78)

3. Normal Mapping (Blinn 78) Specify surface normals

4. Normal Mapping

5. A Problem… Multiple normals per pixel Undersampling Filtering needed ?

6. Supersampling Correct results Too slow

7. MIP mapping Pre-filter Normals do not interpolate linearly Blurring of details

8. Comparison supersampled MIP mapped

9. Representation a single vector is not enough how do we represent multiple surface normals?

10. Previous Work Gaussian Distributions –(Olano and North 97) –(Schilling 97) –(Toksvig 05) Mixture Models –(Fournier 92) –(Tan, et.al. 05) 3D Gaussian 2D covariance matrix 1D Gaussian mixture of Phong lobes mixture of 2D Gaussians no general solution

11. Our Contributions Theoretical Framework –Normal Distribution Function (NDF) –Linear averaging for filtering –Convolution for rendering –Unifies previous works New normal map representations –Spherical harmonics –von Mises-Fisher Distribution Simple, efficient rendering algorithms

12. Normal Distribution Function (NDF) Describes normals within region Defined on the unit sphere Integrates to one Extended Gaussian Image (Horn 84)

13. Normal Distribution Function NDF normal map

14. Normal Distribution Function NDF normal map

15. Normal Distribution Function NDF normal map

16. Normal Distribution Function NDF normal map

17. NDF Filtering normal map

18. NDF Filtering normal map

19. NDF Filtering NDF averaging is linear Store NDFs in MIP map

20. Rendering rendered image normal, pixel value lightsBRDF Radially symmetric BRDFs Lambertian: Blinn-Phong: Torrance-Sparrow: Factored:

21. Supersampling supersampled image samples Effective BRDF

22. samples Effective BRDF NDF,

23. Spherical Convolution Form studied in lighting –(Basri and Jacobs 01) –(Ramamoorthi and Hanrahan 01) Effective BRDF = convolution of NDF & BRDF

24. NDF Spherical Convolution Effective BRDF BRDF

25. Previous Work Gaussian Distributions –Olano and North (97) –Schilling (97) –Toksvig (05) Mixture Models –Fournier (92) –Tan, et.al. (05) Our Work 3D Gaussian 2D covariance matrix 1D Gaussian mixture of Phong lobes mixture of 2D Gaussians NDF representations spherical harmonics von Mises-Fisher mixtures

26. Spherical Harmonics Analogous to Fourier basis Convolution formula:

27. BRDF Coefficients Arbitrary BRDFs Cheaply represented –Analytic: compute in shader –Measured: store on GPU Easily changed at runtime

28. NDF Coefficients Store in MIP mapped textures Finest-level NDFs are delta functions, so: Use standard linear filtering

29. Effective BRDF Coefficients Product of NDF, BRDF coefficients Proceed as usual

31. Limitations Storage cost of NDF –One texture per coefficient –O( ) cost Limited to low frequencies

32. von Mises-Fisher Distribution (vMF) Spherical analogue to Gaussian Desirable properties –Spherical domain –Distribution function –Radially symmetric more concentratedless concentrated

33. Mixtures of vMFs NDF number of vMFs 1 2 3456

34. Expectation Maximization (EM) From machine learning Used in (Tan et.al. 05) Fit model parameters to data data NDF model vMF Mixture EM

35. Rendering Convolution –Spherical harmonic coefficients –Analytic convolution formula Extensions to EM –Aligned lobes (Tan et.al. 05) –Colored lobes NDFrendered image

40. Conclusion Summary –Theoretical Framework –New NDF representations –Practical rendering algorithms Future directions –Offline rendering, PRT –Further applications for vMFs –Shadows, parallax, inter-reflections, etc.

41. Thanks! Tony Jebara, Aner Ben-Artzi, Peter Belhumeur, Pat Hanrahan, Shree Nayar, Evgueni Parilov, Makiko Yasui, Denis Zorin, and nVidia. http://www.cs.columbia.edu/cg/normalmap