Precise Omnidirectional Camera Calibration Dennis Strelow, Jeffrey Mishler, David Koes, and Sanjiv Singh

Overview (1) Projection model for omnidirectional cameras that accounts for the full rotation and translation between camera and mirror Projection model handles noncentral omnidirectional cameras Calibration algorithm determines relative position from one omnidirectional image of known 3D targets

Overview (2) One image sufficient for accurate calibration of transformation Full calibration allows shape-from-motion and epipolar matching even if camera- mirror misalignment is severe Full model improves shape-from-motion and epipolar geometry results even if the camera and mirror are closely aligned

Omnidirectional cameras

Omnidirectional projections (1) The mirror point m determines the projection

Omnidirectional projections (2) Finding the mirror point is… one-dimensional (z only) if the mirror and camera are assumed aligned closed form for aligned single viewpoint cameras

Omnidirectional projections (3) If the camera and mirror are not aligned, then two constraints determine m

Equiangular Cameras (1)

Equiangular cameras (2) Relative rotation, translation between axes distorts projections

Calibration (1)

Calibration (2) Least squares error to be minimized: Known: 2D projections x i, 3D points p i Unknown: Camera position R c, t c ; mirror-to- camera transformation (implicit in ∏ )

Experiments (1) Basic questions about calibration: 1. Does the calibration produce the correct mirror-to-camera transformation? 2. Is the model correct, e.g., is it possible to perform SFM with misaligned a mirror? 3. Is the full model worthwhile if the mirror is nearly aligned?

Experiments (2) Three lab sequences Mirror and camera axes: 1. Closely aligned 2. Moderate misalignment 3. Severe misalignment

Experiments (3)

Experiments (4) Performed shape-from-motion on each sequence using each of three calibrations: A. Calibrate nothing B. Calibrate mirror-camera distance C. Calibrate rotation and translation Calibration B is interesting because computing the projection in this case is a one-dimensional problem

Experiments (5): residuals Difference between observed target image location and reprojected location (pixels) Cal. ACal. BCal. C Seq Seq Seq

Experiments (6): values Sequences differ mainly in t x Standard deviations are small t x (cm) Seq ± Seq ± Seq ±

Experiments (7): apex reproj. Difference between observed screw center and reprojected mirror apex Difference Seq pixels Seq pixels Seq pixels

Experiments (8): SFM Shape-from-motion average reprojection errors (pixels) and depth errors (cm) Cal. ACal. BCal. C Seq / / / 2.0 Seq / / / 1.9 Seq / / / 1.9

Experiments (9): epipolar error Average distance in pixels from epipolar line to correct match Cal. ACal. BCal. C Seq Seq Seq