The Plenoptic Function Lázaro Hermoso Beltrán

2 Previous Concepts “The body of the air is full of an infinite number of radiant pyramids caused by the objects located in it” Leonardo Da Vinci Pencil of rays: The set of light rays passing through any point in space Plenoptic function Plenus : complete or full Optic

3 Definition (I) “Radiance received along any direction V arriving at any point E in space, at any time ‘t’ and over any range of wavelength” 7-Dimensional function

4 Geometric Components of the pencil of ray lights ANGULAR COORDINATES –E = (Ex, Ey, Ez) Viewpoint –. Direction of the ray light passin trough the Viewpoint CARTESIAN COORDINATES –Comonly used in machine vision

5 How would obtain plenoptic function 1.Placing an imaginary eye at every position 2.Record the intensity of light at every angle 3.For every wavelength 4.At every time t It does not need specify the direction of gaze!!

6 The Holographic Movie Example Black and white photo Color photograph Color movie Color holographic movie

7 Plenoptic Measurements in Human Vision PF  potential iformation available Observer takes samples from PF Not all the information is usefull. Enviroment has determined what is usefull (movements  predators). Enviroment constraints

8 P.M. in Human Vision Measurements has limited resolution and number of Samples Space and time the more importants

9 P.M. in Human Vision An example of what we can not measure teaching/flash/koffka-movie.swf

10 P.M. in Human Vision

11 Plenoptic Measurements: Red Bar Example (Vx, Vy, Vz)

12 P.M. in Machine Vision Develop a taxonomy of derivative types “Periodic Table” of visual elements Constraints  Available information (PF)

13 Aplications: Image Based Rendering IBR : Sampling and Rendering Problem, PF is 7D function.We need reduce it! RGB representation Plenoptic modelling Light Field/Lumigraph P(s, t, u, v) 2D-Image Mosaicing

14 RGB – Representation Three 5-D functions

15 Plenoptic Modelling Without time and wavelength Taking samples with cameras in enough close positions We can reconstruct PF interpolating the samples

16 Light Field Assumption that that rays does not change intensity along direction Parametrization rays with 2 parallel planes

17 2D-Image Mosacing To paste serial 2D images to obtain a wider image. The images share the same proyection point Panoramic mosaic or Panorama We reconstruct a 2D Plenoptic function

18 Interpolate Views

19 References E.H. Adelson and J. Bergen, “The plenoptic function andthe elements of early vision” In Computational Models ofVisual Processing, pages MIT Press, 1991 The Plenoptic Illumination Function Tien-TsinWong1 Chi-Wing Fu2 Pheng-Ann Heng1 Chi-Sing Leung3 McMillan, Bishop. “Plenotic Modeling: An Image-Based Rendering System” Shum, Kang. “A Review of Image-based Rendering Techniques”