Download presentation

Presentation is loading. Please wait.

Published byDeborah Thornton Modified about 1 year ago

1
Fourier Slice Photography Ren Ng Stanford University

2
Conventional Photograph

3
Light Field Photography Capture the light field inside the camera body

4
Hand-Held Light Field Camera Medium format digital cameraCamera in-use 16 megapixel sensorMicrolens array

5
Light Field in a Single Exposure

6

7
Light Field Inside the Camera Body

8
Digital Refocusing

9

10
Questions About Digital Refocusing What is the computational complexity? Are there efficient algorithms? What are the limits on refocusing? How far can we move the focal plane?

11
Overview Fourier Slice Photography Theorem Fourier Refocusing Algorithm Theoretical Limits of Refocusing

12
Previous Work Integral photography Lippmann 1908, Ives 1930 Lots of variants, especially in 3D TV Okoshi 1976, Javidi & Okano 2002 Closest variant is plenoptic camera Adelson & Wang 1992 Fourier analysis of light fields Chai et al Refocusing from light fields Isaksen et al. 2000, Stewart et al. 2003

13
Fourier Slice Photography Theorem In the Fourier domain, a photograph is a 2D slice in the 4D light field. Photographs focused at different depths correspond to 2D slices at different trajectories.

14
Digital Refocusing by Ray-Tracing Lens Sensor u x

15
Digital Refocusing by Ray-Tracing Lens Sensor u Imaginary film x

16
Digital Refocusing by Ray-Tracing Lens Sensor u x Imaginary film

17
Digital Refocusing by Ray-Tracing Lens Sensor u Imaginary film x

18
Digital Refocusing by Ray-Tracing Lens Sensor u x Imaginary film

19
Refocusing as Integral Projection Lens Sensor u x Imaginary film x u

20
Refocusing as Integral Projection Lens Sensor u x Imaginary film x u

21
Refocusing as Integral Projection Lens Sensor u x x u Imaginary film

22
Refocusing as Integral Projection Lens Sensor u x x u Imaginary film

23
Classical Fourier Slice Theorem 2D Fourier Transform 1D Fourier Transform Integral Projection Slicing

24
2D Fourier Transform Classical Fourier Slice Theorem 1D Fourier Transform Integral Projection Slicing

25
Classical Fourier Slice Theorem 2D Fourier Transform 1D Fourier Transform Integral Projection Slicing

26
Fourier Domain Classical Fourier Slice Theorem Integral Projection Slicing Spatial Domain

27
Classical Fourier Slice Theorem Integral Projection Slicing Fourier Domain

28
Fourier Slice Photography Theorem Integral Projection Slicing Fourier Domain Spatial Domain

29
Fourier Slice Photography Theorem 4D Fourier Transform Integral Projection Slicing

30
Fourier Slice Photography Theorem 4D Fourier Transform 2D Fourier Transform Integral Projection Slicing

31
Fourier Slice Photography Theorem 4D Fourier Transform 2D Fourier Transform Integral Projection Slicing

32
Fourier Slice Photography Theorem 4D Fourier Transform 2D Fourier Transform Integral Projection Slicing

33
Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing

34
Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing

35
Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing

36
Theorem Limitations Film parallel to lens Everyday camera, not view camera Aperture fully open Closing aperture requires spatial mask

37
Overview Fourier Slice Photography Theorem Fourier Refocusing Algorithm Theoretical Limits of Refocusing

38
Existing Refocusing Algorithms Existing refocusing algorithms are expensive O(N 4 ) where light field has N samples in each dimension All are variants on integral projection Isaksen et al Vaish et al.2004 Levoy et al Ng et al. 2005

39
Refocusing in Spatial Domain 4D Fourier Transform 2D Fourier Transform Integral Projection Slicing

40
Refocusing in Fourier Domain 4D Fourier Transform Inverse 2D Fourier Transform Integral Projection Slicing

41
Refocusing in Fourier Domain 4D Fourier Transform Inverse 2D Fourier Transform Integral Projection Slicing

42
Asymptotic Performance Fourier-domain slicing algorithm Pre-process: O(N 4 log N) Refocusing: O(N 2 log N) Spatial-domain integration algorithm Refocusing: O(N 4 )

43
Resampling Filter Choice Triangle filter (quadrilinear) Kaiser-Bessel filter (width 2.5) Gold standard (spatial integration)

44
Overview Fourier Slice Photography Theorem Fourier Refocusing Algorithm Theoretical Limits of Refocusing

45
Problem Statement Assume a light field camera with An f /A lens N x N pixels under each microlens If we compute refocused photographs from these light fields, over what range can we move the focal plane? Analytical assumption Assume band-limited light fields

46
Band-Limited Analysis

47
Light field shot with camera Band-width of measured light field

48
Band-Limited Analysis

49

50

51
Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing

52
Results of Band-Limited Analysis Assume a light field camera with An f /A lens N x N pixels under each microlens From its light fields we can Refocus exactly within depth of field of an f /(A N) lens In our prototype camera Lens is f /4 12 x 12 pixels under each microlens Theoretically refocus within depth of field of an f/48 lens

53
Light Field Photo Gallery

54
Stanford Quad

55
Rodin’s Burghers of Calais

56
Palace of Fine Arts, San Francisco

57

58
Waiting to Race

59
Start of the Race

60
Summary of Main Contributions Formal theorem about relationship between light fields and photographs Computational application gives asymptotically fast refocusing algorithm Theoretical application gives analytic solution for limits of refocusing

61
Future Work Apply general signal-processing techniques Cross-fertilization with medical imaging

62
Thanks and Acknowledgments Collaborators on camera tech report Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz and Pat Hanrahan Readers and listeners Ravi Ramamoorthi, Brian Curless, Kayvon Fatahalian, Dwight Nishimura, Brad Osgood, Mike Cammarano, Vaibhav Vaish, Billy Chen, Gaurav Garg, Jeff Klingner Anonymous SIGGRAPH reviewers Funding sources NSF, Microsoft Research Fellowship, Stanford Birdseed Grant

63
Questions? “Start of the race”, Stanford University Avery Pool, July 2005

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google