1. Uncalibrated reconstruction Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial scene knowledge S. Soatto, J. Kosecka

2. ICRA 20032 Uncalibrated Camera Image Plane Coordinates Perspective projection Calibrated camera Pixel coordinates Camera intrinsic parameters Projection Matrix Uncalibrated camera

3. ICRA 20033 Taxonomy of uncalibrated reconstruction K is known, back to calibrated case K is unknown Calibration (with a rig) – estimate K Uncalibrated reconstruction despite the lack of knowledge of K Autocalibration (recover K from images) Use complete scene knowledge (a rig) Use partial knowledge Parallel lines, vanishing points, planar motion, constant intrinsic Ambiguities, stratification (multiple views)

4. ICRA 20034 Calibration with a rig Given 3-D coordinates on known object Eliminate unknown scales Recover Projection Matrix Factor the into and using QR decomposition Solve for translation

5. ICRA 20035 Uncalibrated camera vs. distorted space Inner product in Euclidean space : compute distances and angles Calibration K transforming spatial coordinates Transformation induced on the inner product S (the metric of the space) and K are equivalent

6. ICRA 20036 Calibrated vs. Uncalibrated Space

7. ICRA 20037 Calibrated vs. Uncalibrated Space Distances and angles are modified by S

8. ICRA 20038 Motion in the distorted space Uncalibrated coordinates are related by Conjugate of the Euclidean group Calibrated spaceUncalibrated space

9. ICRA 20039 Uncalibrated Camera or Distorted Space Uncalibrated camera with Calibration matrix K viewing points in Euclidean space and moving with (R,T) is equivalent to calibrated camera viewing points in distorted space governed by S and moving with motion conjugate to (R,T)

10. Uncalibrated Epipolar Geometry Fundamental matrix equivalent forms of F Epipolar Constraint

11. ICRA 200311 Image correspondences Epipolar lines Epipoles Properties of the fundamental matrix

12. ICRA 200312 Characterization of the fundamental matrix A nonzero matrix is a fundamental matrix if has A singular value decomposition (SVD) with For some. There is little structure in the matrix except that

13. ICRA 200313 Decomposing the fundamental matrix Decomposition of the Fundamental matrix into skew symmetric matrix and nonsingular matrix Decomposition of F is not unique Unknown parameters - ambiguity Corresponding projection matrix

14. ICRA 200314 What does F tell us? F can be inferred from point matches (8-point algorithm) Cannot extract motion, structure and calibration from one fundamental matrix (2 views) F allows reconstruction up to a projective transformation (as we will see soon) F encodes all the geometric information among two views when no additional information is available

15. ICRA 200315 Ambiguities in the image formation Potential Ambiguities Ambiguity in K (K can be recovered uniquely – Cholesky or QR) Structure of the motion parameters Just an arbitrary choice of reference frame

16. ICRA 200316 Ambiguities in the image formation Structure of the Motion Parameters

17. ICRA 200317 Ambiguities in the image formation For any invertible 4 x 4 matrix H In the uncalibrated case we cannot distinguish between camera imaging word from camera imaging distorted world In general, H is of the following form In order to preserve the choice of the first reference frame we can restrict some DOF of H Structure of the projection matrix

18. ICRA 200318 Structure of the projective ambiguity H can be further decomposed as For i-th frame 1 st frame as reference Choose the projective reference frame then ambiguity is

19. ICRA 200319 Geometric stratification (contd.)

20. ICRA 200320 Projective reconstruction From points, extract F; from F extract  X Canonical decomposition Projection matrices

21. ICRA 200321 Projective reconstruction Given projection matrices recover projective structure Given 2 images and no prior information, the scene can be recovered up a a 4-parameter family of solutions. This is the best one can do!

22. ICRA 200322 Affine upgrade Upgrade projective structure to an affine structure Exploit partial scene knowledge Vanishing points, no skew, known principal point Special motions Pure rotation, pure translation, planar motion, rectilinear motion Constant camera parameters (multi-view)

23. ICRA 200323 Affine upgrade using vanishing points Maps the points To points with affine coordinates

24. ICRA 200324 Vanishing point estimation

25. ICRA 200325 Euclidean upgrade Exploit special motions (e.g. pure rotation) If Euclidean is the goal, perform Euclidean reconstruction directly (no stratification) Multi-view reconstruction (tomorrow) Autocalibration (Kruppa)

26. ICRA 200326 Overview of the methods

27. ICRA 200327 Direct stratification for multiple views Absolute Quadric Constraint

28. ICRA 200328 Direct methods - Summary

29. ICRA 200329 Summary