Presentation is loading. Please wait.

Presentation is loading. Please wait.

Spherical Convolution in Computer Graphics and Vision Ravi Ramamoorthi Columbia Vision and Graphics Center Columbia University SIAM Imaging Science Conference:

Similar presentations


Presentation on theme: "Spherical Convolution in Computer Graphics and Vision Ravi Ramamoorthi Columbia Vision and Graphics Center Columbia University SIAM Imaging Science Conference:"— Presentation transcript:

1 Spherical Convolution in Computer Graphics and Vision Ravi Ramamoorthi Columbia Vision and Graphics Center Columbia University SIAM Imaging Science Conference: May 17, 2006

2 Outline  Motivation and Practical Problems  Spherical Convolution and Applications  A Theory of Spherical Harmonic Identities  Signal Processing for Visual Appearance

3 Real-Time Rendering Motivation: Interactive rendering with natural illumination and realistic, measured materials

4 Inverse Rendering Photographs Geometric model Inverse Rendering Algorithm BRDF

5 Inverse Rendering Geometric model Forward Rendering Algorithm BRDF Novel lighting Rendering Photographs

6 Direct Object Relighting ? Unknown Lighting Unknown BRDF

7 Lighting-Insensitive Recognition Illuminate subject from many incident directions Space of images as lighting is varied

8 Checking Image Consistency  Easy to tamper / splice images  Image processing software widely available  In news reporting and other applications  Need to detect tampering or photomontage  Verify image consistency  Try to check consistency of lighting, shading

9 Checking Image Consistency

10 Without knowing illumination and reflectance:  How do we know if two objects / people in a photograph are lit consistently?  How do we detect inconsistencies or tampering?

11 Outline  Motivation and Practical Problems  Spherical Convolution and Applications  A Theory of Spherical Harmonic Identities  Signal Processing for Visual Appearance

12 Environment Maps Miller and Hoffman, 1984 Later, Greene 86, Cabral 87, 99,…

13 Irradiance Environment Maps Incident Radiance (Illumination Environment Map) Irradiance Environment Map R N

14 Computing Irradiance  Classically, hemispherical integral for each pixel  Lambertian surface is like a low pass filter  Frequency-space analysis (spherical harmonics) Incident Radiance Irradiance

15 Spherical Harmonics

16 Analytic Irradiance Formula Lambertian surface is low-pass filter Basri & Jacobs 01 Ramamoorthi & Hanrahan 01a

17 9 Parameter Approximation Exact image Order 2 9 terms RMS Error = 1% For any illumination, average error < 2% [Basri, Jacobs 01] Ramamoorthi and Hanrahan 01b

18 Real-Time Rendering Simple procedural rendering method (no textures)  Requires only matrix-vector multiply and dot-product  In software or NVIDIA vertex programming hardware Widely used in Games (AMPED for Microsoft Xbox), Movies (Pixar, Framestore CFC, …)

19 Computer Vision Complex Illumination Low Dimensional Subspace  Lighting Insensitive Recognition (Basri and Jacobs 01, Lee et al. 01, Ramamoorthi 02, …)  Photometric stereo, shape acquisition

20 Convolution for General Materials Ramamoorthi and Hanrahan 01 Frequency: product Spatial: integral Spherical Harmonics

21 Related Theoretical Work  Qualitative observation of reflection as convolution: Miller & Hoffman 84, Greene 86, Cabral et al. 87,99  Reflection as frequency-space operator: D’Zmura 91  Lambertian reflection is convolution: Basri Jacobs 01 Our Contributions  Explicitly derive frequency-space convolution formula  Formal quantitative analysis in general 3D case  Apply to real-time, inverse rendering, computer vision

22 Natural Lighting, Realistic Materials Ramamoorthi and Hanrahan 02

23 Inverse Rendering 3 photographs of cat sculpture Complex unknown illumination Geometry known Estimate microfacet BRDF and distant lighting

24 New View, Lighting Photograph Rendering Ramamoorthi and Hanrahan, 01c

25 Outline  Motivation and Practical Problems  Spherical Convolution and Applications  A Theory of Spherical Harmonic Identities  Signal Processing for Visual Appearance Mahajan, Ramamoorthi, Curless ECCV 06

26 Lighting 1 Lighting 2 Material 1Material 2 Two Objects – Two Lightings

27 Lighting 1 Lighting 2 Material 1Material 2 Two Objects – Two Lightings

28 Lighting 1 Lighting 2 Material 1Material 2 Two Objects – Two Lightings

29 Lighting 1 Lighting 2 Material 1Material 2 Two Objects – Two Lightings Independent of Lighting and BRDF

30 Lighting 1 Lighting 2 Material 1Material 2 Image Estimation Framework ?

31 Lighting 1 Lighting 2 Material 1Material 2 Image Estimation Framework ?

32 Lighting 1 Lighting 2 Material 1Material 2 Image Estimation Framework ?

33 Lighting 1 Lighting 2 Material 1Material 2 Image Estimation – BRDF Transfer ? BRDF Transfer Function

34 Lighting 1 Lighting 2 Material 1Material 2 Image Estimation – Lighting Transfer ?

35 Lighting 1 Lighting 2 Material 1Material 2 Image Estimation – Lighting Transfer ?

36 Lighting 1 Lighting 2 Material 1Material 2 Image Estimation – Lighting Transfer ? Light Transfer Function

37 Image Estimation Object 1 Object 2 Lighting 1 Our MethodActual Lighting 2 ? No BRDF and lighting known or estimated

38 Image Consistency Checking Spliced Image

39 Image Consistency Checking Spliced Image

40 Image Consistency Checking Single Image Identity diffuse + specular case Two Lightings – Same Reflectance Identity Two Materials – Two Lightings identity Tampered Cat Tampered CatUntampered Cat

41 Outline  Motivation and Practical Problems  Spherical Convolution and Applications  A Theory of Spherical Harmonic Identities  Signal Processing for Visual Appearance

42 Signal Processing for Appearance Signal Processing widely applicable visual appearance  Convolution relation for cast shadows [Soler and Sillion 98, Ramamoorthi et al. 04]  Convolution with glows for participating media (mist, fog, haze) [Sun et al. 05]  Signal-Processing analysis of light field and reflectance [Chai et al. 00, Zickler et al. 06]  Triple Product Integrals [Ng et al. 04]  First Order Analysis [Ramamoorthi et al. 06]

43 Video clip 1 Real Time Rendering of Scattering

44 Video clip 1 Reflectance Sharing

45 Acknowledgements  Collaborators  Dhruv Mahajan  Brian Curless  Pat Hanrahan  Sameer Agarwal (helpful discussions)  Funding: NSF and Sloan Foundation

46 Questions


Download ppt "Spherical Convolution in Computer Graphics and Vision Ravi Ramamoorthi Columbia Vision and Graphics Center Columbia University SIAM Imaging Science Conference:"

Similar presentations


Ads by Google