Presentation is loading. Please wait.

Presentation is loading. Please wait.

Fourier Slice Photography Ren Ng Stanford University.

Similar presentations


Presentation on theme: "Fourier Slice Photography Ren Ng Stanford University."— Presentation transcript:

1 Fourier Slice Photography Ren Ng Stanford University

2 Conventional Photograph

3 Light Field Photography Capture the light field inside the camera body

4 Hand-Held Light Field Camera Medium format digital cameraCamera in-use 16 megapixel sensorMicrolens array

5 Light Field in a Single Exposure

6

7 Light Field Inside the Camera Body

8 Digital Refocusing

9

10 Questions About Digital Refocusing What is the computational complexity? Are there efficient algorithms? What are the limits on refocusing? How far can we move the focal plane?

11 Overview Fourier Slice Photography Theorem Fourier Refocusing Algorithm Theoretical Limits of Refocusing

12 Previous Work Integral photography Lippmann 1908, Ives 1930 Lots of variants, especially in 3D TV Okoshi 1976, Javidi & Okano 2002 Closest variant is plenoptic camera Adelson & Wang 1992 Fourier analysis of light fields Chai et al Refocusing from light fields Isaksen et al. 2000, Stewart et al. 2003

13 Fourier Slice Photography Theorem In the Fourier domain, a photograph is a 2D slice in the 4D light field. Photographs focused at different depths correspond to 2D slices at different trajectories.

14 Digital Refocusing by Ray-Tracing Lens Sensor u x

15 Digital Refocusing by Ray-Tracing Lens Sensor u Imaginary film x

16 Digital Refocusing by Ray-Tracing Lens Sensor u x Imaginary film

17 Digital Refocusing by Ray-Tracing Lens Sensor u Imaginary film x

18 Digital Refocusing by Ray-Tracing Lens Sensor u x Imaginary film

19 Refocusing as Integral Projection Lens Sensor u x Imaginary film x u

20 Refocusing as Integral Projection Lens Sensor u x Imaginary film x u

21 Refocusing as Integral Projection Lens Sensor u x x u Imaginary film

22 Refocusing as Integral Projection Lens Sensor u x x u Imaginary film

23 Classical Fourier Slice Theorem 2D Fourier Transform 1D Fourier Transform Integral Projection Slicing

24 2D Fourier Transform Classical Fourier Slice Theorem 1D Fourier Transform Integral Projection Slicing

25 Classical Fourier Slice Theorem 2D Fourier Transform 1D Fourier Transform Integral Projection Slicing

26 Fourier Domain Classical Fourier Slice Theorem Integral Projection Slicing Spatial Domain

27 Classical Fourier Slice Theorem Integral Projection Slicing Fourier Domain

28 Fourier Slice Photography Theorem Integral Projection Slicing Fourier Domain Spatial Domain

29 Fourier Slice Photography Theorem 4D Fourier Transform Integral Projection Slicing

30 Fourier Slice Photography Theorem 4D Fourier Transform 2D Fourier Transform Integral Projection Slicing

31 Fourier Slice Photography Theorem 4D Fourier Transform 2D Fourier Transform Integral Projection Slicing

32 Fourier Slice Photography Theorem 4D Fourier Transform 2D Fourier Transform Integral Projection Slicing

33 Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing

34 Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing

35 Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing

36 Theorem Limitations Film parallel to lens Everyday camera, not view camera Aperture fully open Closing aperture requires spatial mask

37 Overview Fourier Slice Photography Theorem Fourier Refocusing Algorithm Theoretical Limits of Refocusing

38 Existing Refocusing Algorithms Existing refocusing algorithms are expensive O(N 4 ) where light field has N samples in each dimension All are variants on integral projection Isaksen et al Vaish et al.2004 Levoy et al Ng et al. 2005

39 Refocusing in Spatial Domain 4D Fourier Transform 2D Fourier Transform Integral Projection Slicing

40 Refocusing in Fourier Domain 4D Fourier Transform Inverse 2D Fourier Transform Integral Projection Slicing

41 Refocusing in Fourier Domain 4D Fourier Transform Inverse 2D Fourier Transform Integral Projection Slicing

42 Asymptotic Performance Fourier-domain slicing algorithm Pre-process: O(N 4 log N) Refocusing: O(N 2 log N) Spatial-domain integration algorithm Refocusing: O(N 4 )

43 Resampling Filter Choice Triangle filter (quadrilinear) Kaiser-Bessel filter (width 2.5) Gold standard (spatial integration)

44 Overview Fourier Slice Photography Theorem Fourier Refocusing Algorithm Theoretical Limits of Refocusing

45 Problem Statement Assume a light field camera with An f /A lens N x N pixels under each microlens If we compute refocused photographs from these light fields, over what range can we move the focal plane? Analytical assumption Assume band-limited light fields

46 Band-Limited Analysis

47 Light field shot with camera Band-width of measured light field

48 Band-Limited Analysis

49

50

51 Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing

52 Results of Band-Limited Analysis Assume a light field camera with An f /A lens N x N pixels under each microlens From its light fields we can Refocus exactly within depth of field of an f /(A N) lens In our prototype camera Lens is f /4 12 x 12 pixels under each microlens Theoretically refocus within depth of field of an f/48 lens

53 Light Field Photo Gallery

54 Stanford Quad

55 Rodin’s Burghers of Calais

56 Palace of Fine Arts, San Francisco

57

58 Waiting to Race

59 Start of the Race

60 Summary of Main Contributions Formal theorem about relationship between light fields and photographs Computational application gives asymptotically fast refocusing algorithm Theoretical application gives analytic solution for limits of refocusing

61 Future Work Apply general signal-processing techniques Cross-fertilization with medical imaging

62 Thanks and Acknowledgments Collaborators on camera tech report Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz and Pat Hanrahan Readers and listeners Ravi Ramamoorthi, Brian Curless, Kayvon Fatahalian, Dwight Nishimura, Brad Osgood, Mike Cammarano, Vaibhav Vaish, Billy Chen, Gaurav Garg, Jeff Klingner Anonymous SIGGRAPH reviewers Funding sources NSF, Microsoft Research Fellowship, Stanford Birdseed Grant

63 Questions? “Start of the race”, Stanford University Avery Pool, July 2005


Download ppt "Fourier Slice Photography Ren Ng Stanford University."

Similar presentations


Ads by Google