General Imaging Model Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Conference Vancouver, July 2001 Partially funded by NSF ITR Award, DARPA/ONR MURI

Imaging What is a general imaging model ? How do we Compute its Parameters ? SceneImaging SystemImages

Perspective Imaging Model Camera Obscura rays selected rays become image points

Systems that are not perspective multiple camera system catadioptric system fisheye lens compound eyes

General Imaging Model Essential components: – Photosensitive elements – optics i PiPi Maps incoming pixels to rays

Raxel = Ray + Pixel Small perspective camera – Simple lens – One pixel photo-detector Raxel symbol IndexGeometryRadiometry PositionDirectionPoint SpreadFall-offResponse Most general model is a list of raxels

Ray Surfaces (pX, pY, pZ)(pX, pY, pZ) (q , q  ) imaging optics virtual detectors (raxels) physical detectors (pixels) ray surface Position: (p X, p Y, p Z ) Direction: (q , q  )

perspective Rays in 2D Singularity of rays called a caustic position-direction space position space X Y  non-perspective caustic

Computing Caustics Change coordinates –(x,y,d) (X,Y,Z) Solve for d

Caustic Ray Surface Caustic is a singularity or envelope of incoming rays Caustic represents loci of view-points raxels Caustic curve imaging optics

Simple Examples perspectivesingle viewpointmulti-viewpoint

Raxel Radiometry Non-linear response of photosensitive element Linear fall-off of optical elements Raxel index Normalized Fall-off h(x) Normalized Exposure (e) Normalized Response g(e)

Point Spread Elliptical gaussian model of point spread. – Major and minor deviation lengths,  a (d),  b (d) – Angle of axis  (when  a (d),  b (d) are different) Impulse at Scene point d, Scene depth Chief ray  aa bb Image plane

Finding the Parameters Known optical components: Compute Unknown optical components: Calibration Environment

Calibration Apparatus Structured light at two planes – Geometry from binary patterns – Radiometry from uniform patterns z pfpf pnpn qfqf i

Finding the parameters: Perspective System laptop LCD video camera with perspective lens translating stage sample image

Computed Raxel Model: Geometry X in mm Y in mm Z in mm

Computed Raxel Model: Radiometry Radiometric response g(e) normalized exposure normalized response Pointwise fall-off h(x,y) radius in pixels normalized fall-off

Finding the parameters: Non-single Viewpoint System laptop LCD video camera with perspective lens translating stage parabolic Mirror sample image

Computed Raxel Model: Geometry Rotationally symmetric mm from caustic max mm from axis of symmetry

Computed Raxel Model: Radiometry Fall-off toward edge as resolution increases: – less light collected radius in pixels normalized fall-off

Summary Most general model simply list of raxels Caustics summarize geometry Simple procedure for obtaining parameters from a black box system IndexGeometryRadiometry PositionDirectionPoint SpreadFall-offResponse x, yp X, p Y, p Z q , q   a,  b,  hg(e)