Light Field Stitching with a Plenoptic Camera Zhou Xue LCAV - École Polytechnique Fédérale de Lausanne Dec. 2 2013.

Slides:

Advertisements

Similar presentations

Advertisements

Lytro The first light field camera for the consumer market

Chapter 23:Mirrors and Lenses Flat Mirrors Homework assignment : 20,24,42,45,51  Image of a point source P P’ The reflected rays entering eyes look as.

Light Fields PROPERTIES AND APPLICATIONS. Outline  What are light fields  Acquisition of light fields  from a 3D scene  from a real world scene 

© 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera.

Convex Lenses Thicker in the center than edges. – Lens that converges (brings together) light rays. – Forms real images and virtual images depending on.

EVENTS: INRIA Work Review Nov 18 th, Madrid.

Light Field Rendering Shijin Kong Lijie Heng.

Intelligent Systems Lab. Extrinsic Self Calibration of a Camera and a 3D Laser Range Finder from Natural Scenes Davide Scaramuzza, Ahad Harati, and Roland.

Mirrors Ch. 20. Mirrors Mirror – any smooth object that reflects light to form an image.

CSCE 641 Computer Graphics: Image Mosaicing Jinxiang Chai.

Computational Photography: Image Mosaicing Jinxiang Chai.

Representations of Visual Appearance COMS 6160 [Fall 2006], Lecture 2 Ravi Ramamoorthi

CS485/685 Computer Vision Prof. George Bebis

CSCE641: Computer Graphics Image Formation Jinxiang Chai.

CSCE 641 Computer Graphics: Image-based Rendering (cont.) Jinxiang Chai.

CSCE 641 Computer Graphics: Image Mosaicing Jinxiang Chai.

Linear View Synthesis Using a Dimensionality Gap Light Field Prior

Computer Graphics (Fall 2004) COMS 4160, Lecture 16: Illumination and Shading 2 Lecture includes number of slides from.

CS223b, Jana Kosecka Rigid Body Motion and Image Formation.

Rendering with Concentric Mosaics Heung-Yeung Shum Li-Wei he Microsoft Research.

CSCE 641: Computer Graphics Image-based Rendering Jinxiang Chai.

Camera: optical system d  21 thin lens small angles: Y Z  2 1 curvature radius.

Scenes, Cameras & Lighting. Outline  Constructing a scene  Using hierarchy  Camera models  Light models.

Stereo Ranging with verging Cameras Based on the paper by E. Krotkov, K.Henriksen and R. Kories.

The Story So Far The algorithms presented so far exploit: –Sparse sets of images (some data may not be available) –User help with correspondences (time.

Basic Principles of Imaging and Photometry Lecture #2 Thanks to Shree Nayar, Ravi Ramamoorthi, Pat Hanrahan.

Multi-Aperture Photography Paul Green – MIT CSAIL Wenyang Sun – MERL Wojciech Matusik – MERL Frédo Durand – MIT CSAIL.

Light Field. Modeling a desktop Image Based Rendering  Fast Realistic Rendering without 3D models.

Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL.

A plane-plus-parallax algorithm Basic Model: When FOV is not very large and the camera motion has a small rotation, the 2D displacement (u,v) of an image.

Capturing, Modeling, Rendering 3D Structures

Image Formation. Input - Digital Images Intensity Images – encoding of light intensity Range Images – encoding of shape and distance They are both a 2-D.

Chapter 18: Ray Optics Lisa & Becky. Ray Model of Light  Light rays travel in straight lines  Light rays cross but do not interact  Light rays travel.

1 CS6825: Image Formation How are images created. How are images created.

Photography Lesson 2 Pinhole Camera Lenses. The Pinhole Camera.

Chapter 18-1 Mirrors. Plane Mirror a flat, smooth surface light is reflected by regular reflection rather than by diffuse reflection Light rays are reflected.

Integral University EC-024 Digital Image Processing.

Light wave Recall: Light must reflect off of an object before it can be used to see the object. A flat mirror is called a plane mirror. A plane mirror.

Plain Mirror Style SNC2D. Angles – What’s It All Mean?

Optics Real-time Rendering of Physically Based Optical Effects in Theory and Practice Masanori KAKIMOTO Tokyo University of Technology.

High-Resolution Interactive Panoramas with MPEG-4 발표자 : 김영백 임베디드시스템연구실.

How to create great Panoramic Images © Alan Moore 2013.

1 Finding depth. 2 Overview Depth from stereo Depth from structured light Depth from focus / defocus Laser rangefinders.

03/24/03© 2003 University of Wisconsin Last Time Image Based Rendering from Sparse Data.

: Chapter 11: Three Dimensional Image Processing 1 Montri Karnjanadecha ac.th/~montri Image.

03/09/05© 2005 University of Wisconsin Last Time HDR Image Capture Image Based Rendering –Improved textures –Quicktime VR –View Morphing NPR Papers: Just.

0 Assignment 1 (Due: 3/9) The projections of two parallel lines, l 1 and l 2, which lie on the ground plane G, onto the image plane I converge at a point.

Refraction of light through a Convex Lens. Lens diagram a cross section through the centre plane.

CSE 681 DISTRIBUTED RAY TRACING some implementation notes.

112/5/ :54 Graphics II Image Based Rendering Session 11.

Image formation.

Computer vision: models, learning and inference M Ahad Multiple Cameras

CSCE 641 Computer Graphics: Image-based Rendering (cont.) Jinxiang Chai.

Chapter 2: The Lens. Focal Length is the distance between the center of a lens and the film plane when focused at infinity.

Image-Based Rendering Geometry and light interaction may be difficult and expensive to model –Think of how hard radiosity is –Imagine the complexity of.

Radiometry of Image Formation Jitendra Malik. A camera creates an image … The image I(x,y) measures how much light is captured at pixel (x,y) We want.

Camera surface reference images desired ray ‘closest’ ray focal surface ‘closest’ camera Light Field Parameterization We take a non-traditional approach.

Mirrors. Types of mirror There are two types of mirror Plane (flat) Curved Concave (curves in) Convex (curves out)

Auto-stereoscopic Light-Field Display By: Jesus Caban George Landon.

Radiometry of Image Formation Jitendra Malik. What is in an image? The image is an array of brightness values (three arrays for RGB images)

Spherical Aberration. Rays emanating from an object point that are incident on a spherical mirror or lens at different distances from the optical axis,

Light & Optics Chapters Electromagnetic Wave.

3D Rendering 2016, Fall.

Rendering Pipeline Fall, 2015.

CS262 – Computer Vision Lect 4 - Image Formation

Factors that Influence the Geometric Detection Pattern of Vehicle-based Licence Plate Recognition Systems Martin Rademeyer Thinus Booysen, Arno Barnard.

Lesson 14 Key Concepts and Notes

Part One: Acquisition of 3-D Data 2019/1/2 3DVIP-01.

Presentation transcript:

Light Field Stitching with a Plenoptic Camera Zhou Xue LCAV - École Polytechnique Fédérale de Lausanne Dec

2 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

3 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

4 The Light Field of an Oil Painting Camera location (x, y, z) View direction (θ, Φ) Wavelength and time (θ, Φ) RGB or Multi-spectral image Video or image ( x, y, z )

5 Light Field of the Oil Painting Capture the oil painting at different camera plane Camera plane x image plane v

6 Capture the Light Field with Plenoptic Camera

7 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

8 Light Field Stitching Framework: Set up a X-Y table with accurate translations Registering the light field from each sample Stitching and interpolating a new light field Challenges: Sampling pattern of POC Registration problem How to create the virtual light field

9 Understand Image Stitching in the Light Field Image Stitching with camera rotations Image Stitching with camera translations

10 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

11 Light Field Sample of a Plenoptic Camera A simplified model of the plenoptic camera: Main lens + moving pinhole camera aLaL bLbL tMtM vMvM

12 Light Field Sample of a Plenoptic Camera Samples from the moving pinhole camera Image on the conjugate plane Light rays propagate to the actual camera plane t = t + v·a L v

13 Light Field Sample of a Plenoptic Camera Light field image example

14 Light Field Sample of a Plenoptic Camera Light field image example

15 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

16 Light Field Stitching by Camera Translations Comparison between image stitching and light field stitching by camera translations

17 Light Field Stitching by Camera Rotations Comparison between image stitching and light field stitching by camera rotations

18 Light Field Stitching by Camera Rotations Comparison between image stitching and light field stitching by camera rotations

19 Light Field Stitching for Increasing Angular Range Light field stitching by combining translation and rotation Enlarge the main lens of the plenoptic camera Directly stitch the pinhole camera image

20 Light Field Stitching for Increasing Angular Range One example for acquisition of surface light field Main lens focus on the flat surface Each pinhole image relates the BRDF of a pigment

21 Revisit the Light Field of an Oil Painting Vertical line represents the image taken on plane t Horizontal line represents the image taken at infinity Green line represents the image taken at plane a L - 1/a L

22 Light Field Stitching for Increasing Spatial Range Light field re-sample to generate panorama image from a virtual camera behind the camera plane

23 Preliminary Results of Light Field Stitching Panorama image from multiple light field images Main lens: (unit: meters) Lens radiance = 10*1.0e-2; Focal length = 10*1.0e-3; Micro lens array (MLA): Radiance of MLA= 60*1.0e-3; Samples on MLA=15; Sensor of the moving pinhole camera: Sensor radiance =1*1.0e-4; Samples on the sensor=11; Scene: Distance of the center= 0.2; Slanted angle = 3 degree; Stitching parameters: LF image samples =41×41; LF image size(u, v, x, y)=11×11×15×15 Stitched image size =451×451

24 Overview What is Plenoptic Function Light field stitching for artwork acquisition Revisit the image stitching Light field sample by a plenoptic camera Light field stitching by camera translations and rotations Following work

25 Following Work Work with plenoptic camera (Lytro and Raytrix) Light field stitching experiments Stitching with camera translation or rotation Extend angular range by combining camera motions Extend spatial range by re-sampling the light field Derive the theoretical boundary for the stitched light field Maximum angular and spatial range Maximum angular and spatial resolution

Thank you for your attention! Questions?