1. Course 3: Computational Photography Ramesh Raskar Mitsubishi Electric Research Labs Jack Tumblin Northwestern University Course WebPage : http://www.merl.com/people/raskar/photo

2. Course 3: Computational Photography Course WebPage http://www.merl.com/people/raskar/photo Course Evaluation http://www.siggraph.org /courses_evaluation

3. Welcome Understanding Film-like PhotographyUnderstanding Film-like Photography –Parameters, Nonlinearities, Ray-based concepts Image Processing and Reconstruction ToolsImage Processing and Reconstruction Tools –Multi-image Fusion, Gradient domain, Graph Cuts Improving Camera PerformanceImproving Camera Performance –Better dynamic range, focus, frame rate, resolution Future DirectionsFuture Directions –HDR cameras, Gradient sensing, Smart optics/lighting

4. Goals –Review of 30+ recent papers –Understand computational aspects of cameras Discuss issues in traditional camerasDiscuss issues in traditional cameras Explore alternative imaging methodsExplore alternative imaging methods Learn vision and optics techniquesLearn vision and optics techniques –Discuss image processing and reconstruction tools –What we will not cover Film Cameras, Novel view rendering (IBR), Color issues, Traditional image processing/editingFilm Cameras, Novel view rendering (IBR), Color issues, Traditional image processing/editing

5. Speaker: Jack Tumblin Jack Tumblin is an Assistant Professor of Computer Science at Northwestern University. His interests include novel photographic sensors to assist museum curators in historical preservation, computer graphics and visual appearance, and image-based modeling and rendering. During his doctoral studies at Georgia Tech and post-doc at Cornell, he investigated tone-mapping methods to depict high-contrast scenes. His MS in Electrical Engineering (December 1990) and BSEE (1978), also from Georgia Tech, bracketed his work as co-founder of IVEX Corp., (>45 people as of 1990) where his flight simulator design work was granted 5 US Patents. He is an Associate Editor of ACM Transactions on Graphics, was a member of the SIGGRAPH Papers Committee (2003, 2004), and in 2001 was a Guest Editor of IEEE Computer Graphics and Applications. http://www.cs.northwestern.edu/~jet

6. Speaker: Ramesh Raskar Ramesh Raskar is a Senior Research Scientist at MERL. His research interests include projector-based graphics, computational photography and non-photorealistic rendering. He has published several articles on imaging and photography including multi-flash photography for depth edge detection, image fusion, gradient-domain imaging and projector-camera systems. His papers have appeared in SIGGRAPH, EuroGraphics, IEEE Visualization, CVPR and many other graphics and vision conferences. He was a course organizer at Siggraph 2002, 2003 and 2004. He is a panel organizer at the Symposium on Computational Photography and Video in Cambridge, MA in May 2005. He is a member of the ACM and IEEE. http://www.merl.com/people/raskar/raskar.html

7. Opportunities –Unlocking Photography How to expand camera capabilitiesHow to expand camera capabilities Digital photography that goes beyond film-like photographyDigital photography that goes beyond film-like photography –Opportunities Computing corrects for lens, sensor and lighting limitationsComputing corrects for lens, sensor and lighting limitations Computing merges results from multiple imagesComputing merges results from multiple images Computing reconstructs from coded image samplesComputing reconstructs from coded image samples Cameras benefit from computerized light sourcesCameras benefit from computerized light sources –Think beyond post-capture image processing Computation well before image processing and editingComputation well before image processing and editing –Learn how to build your own camera-toys

8. Traditional Photography Lens Detector Pixels Image

9. Computational Photography: Optics, Sensors and Computations Generalized Sensor Generalized Optics Computations Picture 4D Ray Bender Upto 4D Ray Sampler Ray Reconstruction

10. Computational Photography Novel Cameras Generalized Sensor Generalized Optics Processing

11. Computational Photography Novel Illumination Novel Cameras Generalized Sensor Generalized Optics Processing Light Sources

12. Computational Photography Novel Illumination Novel Cameras Scene : 8D Ray Modulator Generalized Sensor Generalized Optics Processing Light Sources

13. Computational Photography Novel Illumination Novel Cameras Scene : 8D Ray Modulator Display Generalized Sensor Generalized Optics Processing Recreate 4D Lightfield Light Sources

14. Computational Photography Novel Illumination Novel Cameras Scene : 8D Ray Modulator Display Generalized Sensor Generalized Optics Processing 4D Ray Bender Upto 4D Ray Sampler Ray Reconstruction Generalized Optics Recreate 4D Lightfield Light Sources Modulators 4D Incident Lighting 4D Light Field

15. A Teaser: Dual Photography Scene Photocell Projector

16. A Teaser: Dual Photography Scene Photocell Projector

17. A Teaser: Dual Photography Scene Photocell Projector

18. A Teaser: Dual Photography Scene Photocell Projector Camera

19. camera The 4D transport matrix: Contribution of each projector pixel to each camera pixel scene projector

20. camera The 4D transport matrix: Contribution of each projector pixel to each camera pixel scene projector Sen et al, Siggraph 2005

21. camera The 4D transport matrix: Which projector pixel contribute to each camera pixel scene projector Sen et al, Siggraph 2005 ?

22. Dual photography from diffuse reflections the camera’s view Sen et al, Siggraph 2005

23. Camera Obscura, Gemma Frisius, 1558 1558 A Brief History of Images

24. Lens Based Camera Obscura, 1568 1558 1568 A Brief History of Images

25. Still Life, Louis Jaques Mande Daguerre, 1837 1558 1837 1568 A Brief History of Images

26. Silicon Image Detector, 1970 1558 1837 1568 1970 A Brief History of Images

27. 1558 1837 1568 1970 1994 A Brief History of Images Digital Cameras

28. Dream of A New Photography Old New People and Time ~Cheap Precious Each photo Precious Free Lighting Critical Automated* External Sensors No Yes ‘Stills / Video’ Disjoint Merged Exposure Settings Pre-select Post-Process Exposure Time Pre-select Post-Process Resolution/noise Pre-select Post-Process ‘HDR’ range Pre-select Post-Process

29. Survey How many of you are photographers ?How many of you are photographers ? How many of you are photo-artists ?How many of you are photo-artists ? How many of you are involved in camera design/development ?How many of you are involved in camera design/development ? How many do active programming ?How many do active programming ? Field of work: Academics? Industry ? Research ? Art ?Field of work: Academics? Industry ? Research ? Art ?

30. 8:30 Introduction (Raskar) 8:40 Photographic Signal & Film-like Photography (Tumblin) 9:10 Image Processing Tools (Raskar) 9:40 Improving Film-like Photography (Tumblin) 10:15 Break 10:30 Image Reconstruction Techniques (Raskar) 11:15 Smart Lights and Beyond Photography (Tumblin) 11:45 Smart Optics and Sensors (Raskar) 12:05 Discussion Schedule Course Page : http://www.merl.com/people/raskar/photo