1. © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

2. © 2010 Adobe Systems Incorporated. All Rights Reserved. Capturing Memorable Moments 2

3. © 2010 Adobe Systems Incorporated. All Rights Reserved. What’s Missing with this Picture? 3

4. © 2010 Adobe Systems Incorporated. All Rights Reserved. The Lippmann Sensor  Can we request that Photography renders the full variety offered by the direct observation of objects? Is it possible to create a photographic print in such a manner that it represents the exterior world framed, in appearance, between the boundaries of the print, as if those boundaries were that of a window opened on reality. Gabriel Lippmann, 1908.

5. © 2010 Adobe Systems Incorporated. All Rights Reserved. Pixels and Cores: What do we do with that power?  Moore’s Law: Megapixels keep growing  7.2 MP = 8 by 10 at 300dpi  Available on cell phones  60MP sensors available now  Larger available soon (can a use be found?)  Use pixels to capture richer information about a scene  Computationally process captured data  GPU power also riding Moore’s Law curve 5

6. © 2010 Adobe Systems Incorporated. All Rights Reserved. How do we take advantage of these trends?  Plenoptic camera – High resolution; computationally intensive rendering  The pixels are samples of the radiance in the 4D ray space  Optical elements (lenses, space) transform the ray space  We take a picture by rendering (computationally)  We adjust the picture by transforming the ray space (computationally) 6

7. © 2010 Adobe Systems Incorporated. All Rights Reserved. The Full Variety: Focusing

8. © 2010 Adobe Systems Incorporated. All Rights Reserved. Focusing 8

9. © 2010 Adobe Systems Incorporated. All Rights Reserved. Focusing 9

10. © 2010 Adobe Systems Incorporated. All Rights Reserved. The Full Variety: Different Views 10

11. © 2010 Adobe Systems Incorporated. All Rights Reserved. Different Views 11

12. © 2010 Adobe Systems Incorporated. All Rights Reserved. Different Views 12

13. © 2010 Adobe Systems Incorporated. All Rights Reserved. Radiance (Plenoptic Function, Lightfield)  Instead of integrating rays coming from all directions into a given point (traditional camera), capture the rays individually (capture the radiance) using the Lippmann Sensor  Record all the information about the scene 13

14. © 2010 Adobe Systems Incorporated. All Rights Reserved. Plenoptic lens array outside the camera 14

15. © 2010 Adobe Systems Incorporated. All Rights Reserved. Microlenses inside the camera 15

16. © 2010 Adobe Systems Incorporated. All Rights Reserved. Problem - Solution  Problem: Limited depth of field  Solution: Microlenses of different focal lengths (This is similar to HDR where we use microlenses of different apertures) 16

17. © 2010 Adobe Systems Incorporated. All Rights Reserved. Multifocus  Lytro and Raytrix already selling plenoptic cameras  Variable focal lengths: Better focusing. Our work on that started 2009.  Raytrix actually use microlenses of different focal lengths in their camera.  Lytro use single focal length (Plenoptic 1.0) 17

18. © 2010 Adobe Systems Incorporated. All Rights Reserved. Variable Focal Lengths Microlenses 18

19. © 2010 Adobe Systems Incorporated. All Rights Reserved. 19 Variable Focal Lengths Microlenses

20. © 2010 Adobe Systems Incorporated. All Rights Reserved. 20 Variable Focal Lengths Microlenses

21. © 2010 Adobe Systems Incorporated. All Rights Reserved. 21 Variable Focal Lengths Microlenses

22. © 2010 Adobe Systems Incorporated. All Rights Reserved. 22 Variable Focal Lengths Microlenses

23. © 2010 Adobe Systems Incorporated. All Rights Reserved. 23 Variable Focal Lengths Microlenses: Compare Focus Quality

24. © 2010 Adobe Systems Incorporated. All Rights Reserved. 24 Phase Space analysis

25. © 2010 Adobe Systems Incorporated. All Rights Reserved. 25 Phase Space analysis

26. © 2010 Adobe Systems Incorporated. All Rights Reserved. 26 Phase Space analysis: Miocrolenses of different focal lengths

27. © 2010 Adobe Systems Incorporated. All Rights Reserved. To Find Out More  This work was first presented at Asilomar2010/ This work was first presented at Asilomar2010/ 27

28. © 2010 Adobe Systems Incorporated. All Rights Reserved. Live Demonstrations 28

29. © 2010 Adobe Systems Incorporated. All Rights Reserved.