1. Precise Omnidirectional Camera Calibration Dennis Strelow, Jeffrey Mishler, David Koes, and Sanjiv Singh

2. Overview (1) Projection model for omnidirectional cameras that accounts for the full rotation and translation between camera and mirror Projection model handles noncentral omnidirectional cameras Calibration algorithm determines relative position from one omnidirectional image of known 3D targets

3. Overview (2) One image sufficient for accurate calibration of transformation Full calibration allows shape-from-motion and epipolar matching even if camera- mirror misalignment is severe Full model improves shape-from-motion and epipolar geometry results even if the camera and mirror are closely aligned

4. Omnidirectional cameras

5. Omnidirectional projections (1) The mirror point m determines the projection

6. Omnidirectional projections (2) Finding the mirror point is… one-dimensional (z only) if the mirror and camera are assumed aligned closed form for aligned single viewpoint cameras

7. Omnidirectional projections (3) If the camera and mirror are not aligned, then two constraints determine m

8. Equiangular Cameras (1)

9. Equiangular cameras (2) Relative rotation, translation between axes distorts projections

10. Calibration (1)

11. Calibration (2) Least squares error to be minimized: Known: 2D projections x i, 3D points p i Unknown: Camera position R c, t c ; mirror-to- camera transformation (implicit in ∏ )

12. Experiments (1) Basic questions about calibration: 1. Does the calibration produce the correct mirror-to-camera transformation? 2. Is the model correct, e.g., is it possible to perform SFM with misaligned a mirror? 3. Is the full model worthwhile if the mirror is nearly aligned?

13. Experiments (2) Three lab sequences Mirror and camera axes: 1. Closely aligned 2. Moderate misalignment 3. Severe misalignment

14. Experiments (3)

15. Experiments (4) Performed shape-from-motion on each sequence using each of three calibrations: A. Calibrate nothing B. Calibrate mirror-camera distance C. Calibrate rotation and translation Calibration B is interesting because computing the projection in this case is a one-dimensional problem

16. Experiments (5): residuals Difference between observed target image location and reprojected location (pixels) Cal. ACal. BCal. C Seq. 11.921.911.34 Seq. 24.013.851.40 Seq. 36.506.301.36

17. Experiments (6): values Sequences differ mainly in t x Standard deviations are small t x (cm) Seq. 10.0052 ± 0.0035 Seq. 20.21 ± 0.0053 Seq. 30.37 ± 0.0056

18. Experiments (7): apex reproj. Difference between observed screw center and reprojected mirror apex Difference Seq. 15.0 pixels Seq. 26.0 pixels Seq. 34.7 pixels

19. Experiments (8): SFM Shape-from-motion average reprojection errors (pixels) and depth errors (cm) Cal. ACal. BCal. C Seq. 10.40 / 3.30.41 / 3.40.37 / 2.0 Seq. 21.1 / 9.91.1 / 9.60.43 / 1.9 Seq. 31.9 / 15.91.92 / 15.20.38 / 1.9

20. Experiments (9): epipolar error Average distance in pixels from epipolar line to correct match Cal. ACal. BCal. C Seq. 10.680.690.64 Seq. 21.31.40.71 Seq. 32.1 0.64