1. Light Field Stitching with a Plenoptic Camera Zhou Xue LCAV - École Polytechnique Fédérale de Lausanne Dec. 2 2013

2. 2 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

3. 3 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

4. 4 The Light Field of an Oil Painting Camera location (x, y, z) View direction (θ, Φ) Wavelength and time (θ, Φ) RGB or Multi-spectral image Video or image ( x, y, z )

5. 5 Light Field of the Oil Painting Capture the oil painting at different camera plane Camera plane x image plane v

6. 6 Capture the Light Field with Plenoptic Camera

7. 7 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

8. 8 Light Field Stitching Framework: Set up a X-Y table with accurate translations Registering the light field from each sample Stitching and interpolating a new light field Challenges: Sampling pattern of POC Registration problem How to create the virtual light field

9. 9 Understand Image Stitching in the Light Field Image Stitching with camera rotations Image Stitching with camera translations

10. 10 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

11. 11 Light Field Sample of a Plenoptic Camera A simplified model of the plenoptic camera: Main lens + moving pinhole camera aLaL bLbL tMtM vMvM

12. 12 Light Field Sample of a Plenoptic Camera Samples from the moving pinhole camera Image on the conjugate plane Light rays propagate to the actual camera plane t = t + v·a L v

13. 13 Light Field Sample of a Plenoptic Camera Light field image example

14. 14 Light Field Sample of a Plenoptic Camera Light field image example

15. 15 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

16. 16 Light Field Stitching by Camera Translations Comparison between image stitching and light field stitching by camera translations

17. 17 Light Field Stitching by Camera Rotations Comparison between image stitching and light field stitching by camera rotations

18. 18 Light Field Stitching by Camera Rotations Comparison between image stitching and light field stitching by camera rotations

19. 19 Light Field Stitching for Increasing Angular Range Light field stitching by combining translation and rotation Enlarge the main lens of the plenoptic camera Directly stitch the pinhole camera image

20. 20 Light Field Stitching for Increasing Angular Range One example for acquisition of surface light field Main lens focus on the flat surface Each pinhole image relates the BRDF of a pigment

21. 21 Revisit the Light Field of an Oil Painting Vertical line represents the image taken on plane t Horizontal line represents the image taken at infinity Green line represents the image taken at plane a L - 1/a L

22. 22 Light Field Stitching for Increasing Spatial Range Light field re-sample to generate panorama image from a virtual camera behind the camera plane

23. 23 Preliminary Results of Light Field Stitching Panorama image from multiple light field images Main lens: (unit: meters) Lens radiance = 10*1.0e-2; Focal length = 10*1.0e-3; Micro lens array (MLA): Radiance of MLA= 60*1.0e-3; Samples on MLA=15; Sensor of the moving pinhole camera: Sensor radiance =1*1.0e-4; Samples on the sensor=11; Scene: Distance of the center= 0.2; Slanted angle = 3 degree; Stitching parameters: LF image samples =41×41; LF image size(u, v, x, y)=11×11×15×15 Stitched image size =451×451

24. 24 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

25. 25 Following Work Work with plenoptic camera (Lytro and Raytrix) Light field stitching experiments Stitching with camera translation or rotation Extend angular range by combining camera motions Extend spatial range by re-sampling the light field Derive the theoretical boundary for the stitched light field Maximum angular and spatial range Maximum angular and spatial resolution

26. Thank you for your attention! Questions?