Download presentation

Presentation is loading. Please wait.

Published byRuby Parks Modified about 1 year ago

1
Light Fields PROPERTIES AND APPLICATIONS

2
Outline What are light fields Acquisition of light fields from a 3D scene from a real world scene Image rendering from light fields Changing viewing angle Changing the focal plane Sampling and reconstruction Depth vs spectral support Optimal reconstruction Analysis of light transport

3
Outline What are light fields Acquisition of light fields from a 3D scene from a real world scene Image rendering from light fields Changing viewing angle Changing the focal plane Sampling and reconstruction Depth vs spectral support Optimal reconstruction Analysis of light transport

4
The Plenoptic Function Plenus – Complete, full. Optic - appearance, look. The set of things one can ever see Light intensity as a function of ◦Viewpoint – orientation and position ◦Time ◦Wavelength 7D function!

5
The 5D Plenoptic Function Ignoring wavelength and time We need a 5D function to describe light rays across occlusions ◦2D orientation ◦3D position

6
The Light Field (4D Assuming no occlusions ◦Light is constant across rays ◦Need only 4D to represent the space of Rays Is this assumption reasonable? In free space, i.e outside the convex hull of the scene occluders

7
The Light Field Parameterizations ◦Point on a Plane or curved Surface (2D) and Direction on a Hemisphere (2D) ◦Two Points on a Sphere ◦Two Points on two different Planes

8
Two Plane Parameterization Convenient parameterization for computational photography Why? Similar to camera geometry (i.e. film plane vs lens plane) Linear parameterization - easy computations, no trigonometric functions, etc.

9
2D light field Used for visualization. Assume the world is flat (2D)

10
The image a pinhole at (u,v) captures All views of a pixel (s,t) Intuition Light Field Rendering, Levoy Hanrahan '96.

11
Outline What are light fields Acquisition of light fields from a 3D scene from a real world scene Image rendering from light fields Changing viewing angle Changing the focal plane Sampling and reconstruction Depth vs spectral support Optimal reconstruction Analysis of light transport

12
Acquisition of Light Fields Synthetic 3D Scene ◦Discretize s,t,u,v and capture all rays intersecting the objects using a standard Ray Tracer

13
Acquisition of Light Fields Real world scenes Will be explained in more detail next week…

14
Outline What are light fields Acquisition of light fields from a 3D scene from a real world scene Image rendering from light fields Changing viewing angle Changing the focal plane Sampling and reconstruction Depth vs spectral support Optimal reconstruction Analysis of light transport

15
Changing the View Point Problem: Computer Graphics ◦Render a novel view point without expensive ray tracing Solution: ◦Sample a Synthetic light field using Ray Tracing ◦Use the Light Field to generate any point of view, no need to Ray Trace Light Field Rendering, Levoy Hanrahan '96.

16
Changing the View Point Conceptually: Use Ray Trace from all pixels in image plane Actually: Use Homographic mapping from XY plane to the VU and TS, and lookup resulting ray radiance. pinhole

17
Light Field Interpolation NN NN + Linear Linear

18
Changing the focal plane Fourier Slice Photography, Ng, 05

19
In-Camera Light Field Parameterization

20
The camera operator Can define a camera as an operator on the Light Field. ◦The conventional camera operator: y x [Stroebel et al. 1986]

21
Reminder - Thin lens formula D D’ 1 D 1 const += To focus closer - increase the sensor-to-lens distance.

22
Refocusing - Reparameterization

23
Reparametrization - 4D

24
Refocusing - Reparameterization Refocus Reparameterization of the light field Shearing of the Light field

25
Refocusing camera operator Shear and Integrate the original light field *(cos term from conventional camera model is absorbed into L)

26
Computation of Refocusing Operator Naïve Approach For every X,Y go over all U,V and calculate the sum after reparameterization => O(n^4) Can we do better ???? y x

27
Fourier Analysis of the Camera Operator Projection in the spatial domain Slicing in the fourier domain Given that: Then:

28
Fourier Slice Theorem F – Fourier Transform Operator I – Integral Projection Operator S – Slicing Operator

29
Fourier Analysis of the Camera Operator

30
Fourier Slice Photography, Ng, 05

31
Fourier Slice Photography Thm – More corollaries Two important results that are worth mentioning: 1. Filtered Light Field Photography Thm 2. The light field dimensionality gap

32
Filtered Light Field Photography Thm Theorem: Filtered Light Field Photography

33
The light field dimensionality gap ◦The light field is 4D ◦In the frequency domain – The support of all the images with different focus depth is a 3D manifold This observation was used in order to generate new views of the scene from a focal stack (Levin et al. 2010)

34
Outline What are light fields Acquisition of light fields from a 3D scene from a real world scene Image rendering from light fields Changing viewing angle Changing the focal plane Sampling and reconstruction Depth vs spectral support Optimal reconstruction Analysis of light transport

35
Sampling and reconstruction of light fields In many cases the sampling rate is bounded due to camera limitations. We want to understand the spectrum of the light fields better in order to reconstruct better Two main papers go in this direction Plenoptic sampling (SIGGRAPH 2000) – light field spectrum VS. scene’s depth Frequency analysis of light transport (SIGGRAPH 2005) - light field spectrum VS. Path of the light

36
Light Field Sampling Light Field Acquisition – Discretization Light Field Sampling is Limited Example – Camera Array: u,v t,s

37
Sampling in frequency domain Aliasing in the frequency domain Need to analyze Light Field Spectrum * = ALIASING! No Aliasing!

38
Scene Depth and Light Field Light Field Spectrum is related to Scene Depth From Lambertian property each point in the scene corresponds to a line in the Light Field Line slope is a function of the depth (z) of the point. Plenoptic Sampling, Chai et al., 00.

39
Spectral Support of Light Field Constant Depth SceneLight FieldLF Spectrum Plenoptic Sampling, Chai et al., 00.

40
Spectral Support of Light Field Varying Depth SceneLF Spectrum Plenoptic Sampling, Chai et al., 00.

41
Spectral Support of Light Field Plenoptic Sampling, Chai et al., 00.

42
Reconstruction Filters Optimal Slope for filter: Plenoptic Sampling, Chai et al., 00.

43
Limitations Assumptions ◦Lambertian surfaces ◦Free Space – No occlusions

44
Frequency Analysis of Light Transport Informally: Different features of lighting and scene causes different effects in the Frequency Content Blurry Reflections Shadow Boundries Low frequencyHigh frequency A Frequency analysis of Light Transport, Durand et al. 05.

45
Not Wave Optics!!!

46
Frequency Analysis of Light Transport Look at light transport as a signal processing system. ◦Light source is the input signal ◦Interaction are filters / transforms SourceTransportOcclusionTransportReflection (BRDF)

47
Local Light Field We study the local 4D Light Field around a central Ray during transport ◦In Spatial Domain ◦In Frequency Domain * Local light field offers us the ability to talk about the Spectrum In a local setting

48
Local Light Field (2D) Parameterization The analysis is in flatland, an extension to 4D light field is available x-v parameterizationx-Θ parameterization A Frequency analysis of Light Transport, Durand et al. 05.

49
Example Scenario Reflection A Frequency analysis of Light Transport, Durand et al. 05.

50
Light Transport – Spatial Domain Light Propagation Shear of the local Light Field ◦No change in slope (v) ◦Linear change in displacement (X)

51
Light Transport – Frequency Domain Shear in spatial domain is also a shear in Frequency domain

52
Occlusion Spatial domain: Occlusion pointwise multiplication in the spatial domain The incoming light field is multiplied by the binary occlusion function of the occluders. Frequency domain convolution in the frequency domain:

53
Occlusion – example

54
Reflection * Similar analysis for curved surfaces is also presented in the paper

55
Reflection - cosine term Spatial domain - multiplication:: Frequency domain:

56
cosine term example Incoming Light field After cosine term

57
Reflection – Mirror reparameterization

58
Reparameterization example Incoming Light field After reparameterization

59
Reflection - BRDF

60
BRDF Intuition * = direction x (space)

61
Reflection - BRDF

62
BRDF example Incoming Light field After BRDF change - ×

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google