1. Surface Signals for Graphics John Snyder Researcher 3D Graphics Group Microsoft Research

2. Why Surface Signals? l Many useful types of surface signals: n texture map [Catmull74, Blinn&Newell76] (color) n bump map[Max81] (normal) n displacement map [Cook 84] (geometric offset) n geometry image (geometry) n bidirectional texture function (precomputed shading) n self-transfer texture (spherical harmonic coefs) n … l Simplicity of regular 2D image l Support on current graphics hardware (e.g. pixel shaders) l Research questions: n How to generate and manipulate signals? n What new graphics architectures? l Many useful types of surface signals: n texture map [Catmull74, Blinn&Newell76] (color) n bump map[Max81] (normal) n displacement map [Cook 84] (geometric offset) n geometry image (geometry) n bidirectional texture function (precomputed shading) n self-transfer texture (spherical harmonic coefs) n … l Simplicity of regular 2D image l Support on current graphics hardware (e.g. pixel shaders) l Research questions: n How to generate and manipulate signals? n What new graphics architectures?

3. Surface Signal Research Projects Creation precomputed radiance transfer Parameterization signal-specialized param. Representation geometry images Renderingsignal-based graphics architecture

4. Motivation for Precomputed Transfer l better light integration and light transport n dynamic, area lights n shadowing n interreflections l in real-time point light area light area lighting, no shadows area lighting, shadows

5. Self-Transfer Signal (25D) Basis 16 Basis 17 Basis 18 illuminateresult...... Reduces shading to a 25D dot product (low-frequency lighting)

6. Self-Transfer Results (Diffuse) No Shadows/Inter Shadows Shadows+Inter No Shadows/Inter Shadows Shadows+Inter

7. Self-Transfer Results (Glossy) Self-Transfer Results (Glossy) No Shadows/Inter Shadows Shadows+Inter No Shadows/Inter Shadows Shadows+Inter

8. Self-Transfer Demo

9. Geometry-based (know geometry only) Signal-specialized (know geometry+signal) Parameterization of Surface Signals

10. Measuring Parameterization Quality 2D texture domain surface in 3D linear map singular values: γ, Γ

11. Geometric Stretch Metric 2D texture domain surface in 3D linear map TT singular values: γ, Γ geometric stretch = γ 2 + Γ 2 Parameterize = minimize surface integral of geometric stretch

12. n geometric stretch: γ f 2 + Γ f 2 n signal stretch: γ h 2 + Γ h 2 n geometric stretch: γ f 2 + Γ f 2 n signal stretch: γ h 2 + Γ h 2 Parameterize = minimize surface integral of signal stretch Signal Stretch Metric f g domainsurface signal h = g f

13. Geometric stretch [Sander01] Conformal [Floater97] Signal stretch [Sander02]

14. Geometric stretch Signal stretch (64x64 texture) Results: Scanned Color

15. Results: Normal Map Geometric stretch Signal stretch 128x128 texture - multichart

16. Results: Precomputed Radiance Transfer 25D signal – 256x256 texture Geometric stretch Signal stretch

17. 3D graphics = 2D image processing? not quite use images but of surface signals not views synthesize images from 3D surface descriptions synthesize images from 3D surface descriptions run-time flexibility – change view, lighting, rendering params run-time flexibility – change view, lighting, rendering params compactness – single surface parameterization, not multiple views compactness – single surface parameterization, not multiple views high quality (global illumination) – resolution independence high quality (global illumination) – resolution independence cheap creation – no costly rigs & operator, easy to edit cheap creation – no costly rigs & operator, easy to edit as preprocess, convert 3D descriptions to 2D image reps (surface signals) to accelerate run-time as preprocess, convert 3D descriptions to 2D image reps (surface signals) to accelerate run-time signals can be represented as regular 2D images signals can be represented as regular 2D images rendering via general, programmable image processing ops rendering via general, programmable image processing ops

18. Rendering Factorization Preprocess (slow) Run-Time (fast) 3D surfaces (meshes) 3D surfaces (meshes) 3D graphics: 3D graphics: ray tracing, Monte Carlo integration, dynamics simulation, encoding 2D images, streams 2D images, streams 2D image processing 2D image processing decoding, interpolation / decimation, programmable pixel shaders, sample gather global illumination computation is too expensive from scratch surface signals

19. EndEnd

20. PeoplePeople Microsoft Research 3D Graphics Group: Jim Blinn, Conal Elliot, Brian Guenter, Hugues Hoppe, Charles Loop, Don Mitchell, Kirk Olynyk, Peter-Pike Sloan, John Snyder, Turner Whitted Collaborators: Collaborators: Steven Gortler, Xianfeng Gu, Ziyad Hakura, Jesse Hall, Jan Kautz, Leonard McMillan, Pedro Sander, Zoe Wood