Download presentation

Presentation is loading. Please wait.

Published byEricka Dempster Modified about 1 year ago

1
Spherical Convolution in Computer Graphics and Vision Ravi Ramamoorthi Columbia Vision and Graphics Center Columbia University SIAM Imaging Science Conference: May 17, 2006

2
Outline Motivation and Practical Problems Spherical Convolution and Applications A Theory of Spherical Harmonic Identities Signal Processing for Visual Appearance

3
Real-Time Rendering Motivation: Interactive rendering with natural illumination and realistic, measured materials

4
Inverse Rendering Photographs Geometric model Inverse Rendering Algorithm BRDF

5
Inverse Rendering Geometric model Forward Rendering Algorithm BRDF Novel lighting Rendering Photographs

6
Direct Object Relighting ? Unknown Lighting Unknown BRDF

7
Lighting-Insensitive Recognition Illuminate subject from many incident directions Space of images as lighting is varied

8
Checking Image Consistency Easy to tamper / splice images Image processing software widely available In news reporting and other applications Need to detect tampering or photomontage Verify image consistency Try to check consistency of lighting, shading

9
Checking Image Consistency

10
Without knowing illumination and reflectance: How do we know if two objects / people in a photograph are lit consistently? How do we detect inconsistencies or tampering?

11
Outline Motivation and Practical Problems Spherical Convolution and Applications A Theory of Spherical Harmonic Identities Signal Processing for Visual Appearance

12
Environment Maps Miller and Hoffman, 1984 Later, Greene 86, Cabral 87, 99,…

13
Irradiance Environment Maps Incident Radiance (Illumination Environment Map) Irradiance Environment Map R N

14
Computing Irradiance Classically, hemispherical integral for each pixel Lambertian surface is like a low pass filter Frequency-space analysis (spherical harmonics) Incident Radiance Irradiance

15
Spherical Harmonics

16
Analytic Irradiance Formula Lambertian surface is low-pass filter Basri & Jacobs 01 Ramamoorthi & Hanrahan 01a

17
9 Parameter Approximation Exact image Order 2 9 terms RMS Error = 1% For any illumination, average error < 2% [Basri, Jacobs 01] Ramamoorthi and Hanrahan 01b

18
Real-Time Rendering Simple procedural rendering method (no textures) Requires only matrix-vector multiply and dot-product In software or NVIDIA vertex programming hardware Widely used in Games (AMPED for Microsoft Xbox), Movies (Pixar, Framestore CFC, …)

19
Computer Vision Complex Illumination Low Dimensional Subspace Lighting Insensitive Recognition (Basri and Jacobs 01, Lee et al. 01, Ramamoorthi 02, …) Photometric stereo, shape acquisition

20
Convolution for General Materials Ramamoorthi and Hanrahan 01 Frequency: product Spatial: integral Spherical Harmonics

21
Related Theoretical Work Qualitative observation of reflection as convolution: Miller & Hoffman 84, Greene 86, Cabral et al. 87,99 Reflection as frequency-space operator: D’Zmura 91 Lambertian reflection is convolution: Basri Jacobs 01 Our Contributions Explicitly derive frequency-space convolution formula Formal quantitative analysis in general 3D case Apply to real-time, inverse rendering, computer vision

22
Natural Lighting, Realistic Materials Ramamoorthi and Hanrahan 02

23
Inverse Rendering 3 photographs of cat sculpture Complex unknown illumination Geometry known Estimate microfacet BRDF and distant lighting

24
New View, Lighting Photograph Rendering Ramamoorthi and Hanrahan, 01c

25
Outline Motivation and Practical Problems Spherical Convolution and Applications A Theory of Spherical Harmonic Identities Signal Processing for Visual Appearance Mahajan, Ramamoorthi, Curless ECCV 06

26
Lighting 1 Lighting 2 Material 1Material 2 Two Objects – Two Lightings

27
Lighting 1 Lighting 2 Material 1Material 2 Two Objects – Two Lightings

28
Lighting 1 Lighting 2 Material 1Material 2 Two Objects – Two Lightings

29
Lighting 1 Lighting 2 Material 1Material 2 Two Objects – Two Lightings Independent of Lighting and BRDF

30
Lighting 1 Lighting 2 Material 1Material 2 Image Estimation Framework ?

31
Lighting 1 Lighting 2 Material 1Material 2 Image Estimation Framework ?

32
Lighting 1 Lighting 2 Material 1Material 2 Image Estimation Framework ?

33
Lighting 1 Lighting 2 Material 1Material 2 Image Estimation – BRDF Transfer ? BRDF Transfer Function

34
Lighting 1 Lighting 2 Material 1Material 2 Image Estimation – Lighting Transfer ?

35
Lighting 1 Lighting 2 Material 1Material 2 Image Estimation – Lighting Transfer ?

36
Lighting 1 Lighting 2 Material 1Material 2 Image Estimation – Lighting Transfer ? Light Transfer Function

37
Image Estimation Object 1 Object 2 Lighting 1 Our MethodActual Lighting 2 ? No BRDF and lighting known or estimated

38
Image Consistency Checking Spliced Image

39
Image Consistency Checking Spliced Image

40
Image Consistency Checking Single Image Identity diffuse + specular case Two Lightings – Same Reflectance Identity Two Materials – Two Lightings identity Tampered Cat Tampered CatUntampered Cat

41
Outline Motivation and Practical Problems Spherical Convolution and Applications A Theory of Spherical Harmonic Identities Signal Processing for Visual Appearance

42
Signal Processing for Appearance Signal Processing widely applicable visual appearance Convolution relation for cast shadows [Soler and Sillion 98, Ramamoorthi et al. 04] Convolution with glows for participating media (mist, fog, haze) [Sun et al. 05] Signal-Processing analysis of light field and reflectance [Chai et al. 00, Zickler et al. 06] Triple Product Integrals [Ng et al. 04] First Order Analysis [Ramamoorthi et al. 06]

43
Video clip 1 Real Time Rendering of Scattering

44
Video clip 1 Reflectance Sharing

45
Acknowledgements Collaborators Dhruv Mahajan Brian Curless Pat Hanrahan Sameer Agarwal (helpful discussions) Funding: NSF and Sloan Foundation

46
Questions

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google