1. General Imaging Model Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Conference Vancouver, July 2001 Partially funded by NSF ITR Award, DARPA/ONR MURI

2. Imaging What is a general imaging model ? How do we Compute its Parameters ? SceneImaging SystemImages

3. Perspective Imaging Model Camera Obscura rays selected rays become image points

4. Systems that are not perspective multiple camera system catadioptric system fisheye lens compound eyes

5. General Imaging Model Essential components: – Photosensitive elements – optics i PiPi Maps incoming pixels to rays

6. Raxel = Ray + Pixel Small perspective camera – Simple lens – One pixel photo-detector Raxel symbol IndexGeometryRadiometry PositionDirectionPoint SpreadFall-offResponse Most general model is a list of raxels

7. Ray Surfaces (pX, pY, pZ)(pX, pY, pZ) (q , q  ) imaging optics virtual detectors (raxels) physical detectors (pixels) ray surface Position: (p X, p Y, p Z ) Direction: (q , q  )

8. perspective Rays in 2D Singularity of rays called a caustic position-direction space position space X Y  non-perspective caustic

9. Computing Caustics Change coordinates –(x,y,d) (X,Y,Z) Solve for d

10. Caustic Ray Surface Caustic is a singularity or envelope of incoming rays Caustic represents loci of view-points raxels Caustic curve imaging optics

11. Simple Examples perspectivesingle viewpointmulti-viewpoint

12. Raxel Radiometry Non-linear response of photosensitive element Linear fall-off of optical elements Raxel index Normalized Fall-off h(x) Normalized Exposure (e) Normalized Response g(e)

13. Point Spread Elliptical gaussian model of point spread. – Major and minor deviation lengths,  a (d),  b (d) – Angle of axis  (when  a (d),  b (d) are different) Impulse at Scene point d, Scene depth Chief ray  aa bb Image plane

14. Finding the Parameters Known optical components: Compute Unknown optical components: Calibration Environment

15. Calibration Apparatus Structured light at two planes – Geometry from binary patterns – Radiometry from uniform patterns z pfpf pnpn qfqf i

16. Finding the parameters: Perspective System laptop LCD video camera with perspective lens translating stage sample image

17. Computed Raxel Model: Geometry 180 160 360 140 120 100 80 60 180 160 140 120 100 80 340 320 300 280 260 X in mm Y in mm Z in mm

18. Computed Raxel Model: Radiometry Radiometric response g(e) normalized exposure normalized response Pointwise fall-off h(x,y) radius in pixels normalized fall-off 1.0 0.8 0.6 0.4 0.2 0.0 1.0 0.8 0.6 0.4 0.2 0.0 1.00.90.80.70.60.50.40.30.20.10.01.00.90.80.70.60.50.40.30.20.10.0 050100150200250300 0.1 0.0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2

19. Finding the parameters: Non-single Viewpoint System laptop LCD video camera with perspective lens translating stage parabolic Mirror sample image

20. Computed Raxel Model: Geometry Rotationally symmetric 10 5 -35 0 -5 -10 -15 -20 -25 -30 -60 -40 -20 0 60 40 20 -60 -40 -20 0 60 40 20 mm from caustic max mm from axis of symmetry

21. Computed Raxel Model: Radiometry Fall-off toward edge as resolution increases: – less light collected radius in pixels normalized fall-off

22. Summary Most general model simply list of raxels Caustics summarize geometry Simple procedure for obtaining parameters from a black box system IndexGeometryRadiometry PositionDirectionPoint SpreadFall-offResponse x, yp X, p Y, p Z q , q   a,  b,  hg(e)