1. Uncalibrated Geometry & Stratification Sastry and Yang
Uncalibrated Camera
Uncalibrated Geometry & Stratification Sastry and Yang
MASKS © 2004
Invitation to 3D vision

2. Uncalibrated Camera calibrated coordinates Linear transformation pixel
MASKS © 2004
Invitation to 3D vision

3. Overview Calibration with a rig Uncalibrated epipolar geometry
Ambiguities in image formation
Stratified reconstruction
Autocalibration with partial scene knowledge
MASKS © 2004
Invitation to 3D vision

4. Uncalibrated Camera Calibrated camera Uncalibrated camera
Image plane coordinates
Camera extrinsic parameters
Perspective projection
Calibrated camera
Pixel coordinates
Projection matrix
Uncalibrated camera
MASKS © 2004
Invitation to 3D vision

5. Taxonomy on Uncalibrated Reconstruction
is known, back to calibrated case
is unknown
Calibration with complete scene knowledge (a rig) – estimate
Uncalibrated reconstruction despite the lack of knowledge of
Autocalibration (recover from uncalibrated images)
Use partial knowledge
Parallel lines, vanishing points, planar motion, constant intrinsic
Ambiguities, stratification (multiple views)
MASKS © 2004
Invitation to 3D vision

6. Calibration with a Rig
Use the fact that both 3-D and 2-D coordinates of feature
points on a pre-fabricated object (e.g., a cube) are known.
MASKS © 2004
Invitation to 3D vision

7. Calibration with a Rig Given 3-D coordinates on known object
Eliminate unknown scales
Recover projection matrix
Factor the into and using QR decomposition
Solve for translation
MASKS © 2004
Invitation to 3D vision

8. Uncalibrated Epipolar Geometry
Epipolar constraint
Fundamental matrix
Equivalent forms of
MASKS © 2004
Invitation to 3D vision

9. Properties of the Fundamental Matrix
Epipolar lines
Epipoles
Image
correspondences
MASKS © 2004
Invitation to 3D vision

10. Properties of the Fundamental Matrix
A nonzero matrix is a fundamental matrix if has
a singular value decomposition (SVD) with
for some
There is little structure in the matrix except that
MASKS © 2004
Invitation to 3D vision

11. Estimating Fundamental Matrix
Find such F that the epipolar error is minimized
Pixel coordinates
Fundamental matrix can be estimated up to scale
Denote
Rewrite
Collect constraints from all points
MASKS © 2004
Invitation to 3D vision

12. Two view linear algorithm – 8-point algorithm
Solve the LLSE problem:
Solution eigenvector associated with
smallest eigenvalue of
Compute SVD of F recovered from data
Project onto the essential manifold:
cannot be unambiguously decomposed into pose
and calibration
MASKS © 2004
Invitation to 3D vision

13. Calibrated vs. Uncalibrated Space
MASKS © 2004
Invitation to 3D vision

14. Calibrated vs. Uncalibrated Space
Distances and angles are modified by S
MASKS © 2004
Invitation to 3D vision

15. Motion in the distorted space
Calibrated space
Uncalibrated space
Uncalibrated coordinates are related by
Conjugate of the Euclidean group
MASKS © 2004
Invitation to 3D vision

16. What Does F Tell Us?
can be inferred from point matches (eight-point algorithm)
Cannot extract motion, structure and calibration from one fundamental matrix (two views)
allows reconstruction up to a projective transformation (as we will see soon)
encodes all the geometric information among two views when no additional information is available
MASKS © 2004
Invitation to 3D vision

17. Decomposing the Fundamental Matrix
Decomposition of the fundamental matrix into a skew
symmetric matrix and a nonsingular matrix
Decomposition of is not unique
Unknown parameters - ambiguity
Corresponding projection matrix
MASKS © 2004
Invitation to 3D vision

18. Projective Reconstruction
From points, extract , followed by computation
of projection matrices and structure
Canonical decomposition
Given projection matrices – recover structure
Projective ambiguity – non-singular 4x4 matrix
Both and are consistent with the epipolar geometry – give the same fundamental matrix
MASKS © 2004
Invitation to 3D vision

19. Projective Reconstruction
Given projection matrices recover projective structure
This is a linear problem and can be solve using linear least-squares
Projective reconstruction – projective camera matrices and
projective structure
MASKS © 2004
Invitation to 3D vision
Euclidean Structure
Projective Structure

20. Euclidean vs Projective reconstruction
Euclidean reconstruction – true metric properties of objects
lenghts (distances), angles, parallelism are preserved
Unchanged under rigid body transformations
=> Euclidean Geometry – properties of rigid bodies under
rigid body transformations, similarity transformation
Projective reconstruction – lengths, angles, parallelism are NOT
preserved – we get distorted images of objects – their distorted 3D counterparts --> 3D projective reconstruction
=> Projective Geometry
MASKS © 2004
Invitation to 3D vision

21. Homogeneous Coordinates (RBM)
3-D coordinates are related by:
Homogeneous coordinates:
Homogeneous coordinates are related by:
MASKS © 2004
Invitation to 3D vision

22. Homogenous and Projective Coordinates
Homogenous coordinates in 3D before – attach 1 as the last
coordinate – render the transformation as linear transformation
Projective coordinates – all points are equivalent up to a scale
2D- projective plane
3D- projective space
Each point on the plane is represented by a ray in projective space
Ideal points – last coordinate is 0 – ray parallel to the image plane
points at infinity – never intersects the image plane
MASKS © 2004
Invitation to 3D vision

23. Vanishing points – points at infinity
Representation of a 3-D line – in homogeneous coordinates
When  -> 1 - vanishing points – last coordinate -> 0
Similarly in the image plane
MASKS © 2004
Invitation to 3D vision

24. Ambiguities in the image formation
Potential Ambiguities
Ambiguity in K (K can be recovered uniquely – Cholesky or QR)
Structure of the motion parameters
Just an arbitrary choice of reference frame
MASKS © 2004
Invitation to 3D vision

25. Ambiguities in Image Formation
Structure of the (uncalibrated) projection matrix
For any invertible 4 x 4 matrix
In the uncalibrated case we cannot distinguish between camera imaging word from camera imaging distorted world
In general, is of the following form
In order to preserve the choice of the first reference frame we can restrict some DOF of
MASKS © 2004
Invitation to 3D vision

26. Structure of the Projective Ambiguity
1st frame as reference
Choose the projective reference frame
then ambiguity is
can be further decomposed as
MASKS © 2004
Invitation to 3D vision

27. Stratified (Euclidean) Reconstruction
General ambiguity – while preserving choice of first reference
frame
Decomposing the ambiguity into affine and projective one
Note the different effect of the 4-th homogeneous coordinate
MASKS © 2004
Invitation to 3D vision

28. Affine upgrade Upgrade projective structure to an affine structure
Exploit partial scene knowledge
Vanishing points, no skew, known principal point
Special motions
Pure rotation, pure translation, planar motion, rectilinear motion
Constant camera parameters (multi-view)
MASKS © 2004
Invitation to 3D vision

29. Affine upgrade using vanishing points
How to compute
Maps the points
To points with affine coordinates
Vanishing points – last homogeneous affine coordinate is 0
MASKS © 2004
Invitation to 3D vision

30. Affine Upgrade Need at least three vanishing points
3 equations, 4 unknowns (-1 scale)
Solve for
Set up and update the projective
structure
MASKS © 2004
Invitation to 3D vision

31. Euclidean upgrade We need to estimate remaining affine ambiguity
In the case of special motions (e.g. pure rotation) – no projective ambiguity – cannot do projective reconstruction
Estimate directly (special case of rotating camera – follows)
Multi-view case – estimate projective and affine ambiguity together
Use additional constraints of the scene structure (next)
Autocalibration (Kruppa equations)
Alternatives:
MASKS © 2004
Invitation to 3D vision

32. Euclidean Upgrade using vanishing points
Vanishing points – intersections of the parallel lines
Vanishing points of three orthogonal directions
Orthogonal directions – inner product is zero
Provide directly constraints on matrix
S – has 5 degrees of freedom, 3 vanishing points – 3 constraints (need additional assumption about K)
Assume zero skew and aspect ratio = 1
MASKS © 2004
Invitation to 3D vision

33. Geometric Stratification
MASKS © 2004
Invitation to 3D vision

34. Special Motions – Pure Rotation
Calibrated Two views related by rotation only
Mapping to a reference view – rotation can be estimated
Mapping to a cylindrical surface
MASKS © 2004
Invitation to 3D vision

35. Pure Rotation - Uncalibrated Case
Calibrated Two views related by rotation only
Conjugate rotation can be estimated
Given C, how much does it tell us about K (or S) ?
Given three rotations around linearly independent
axes – S, K can be estimated using linear techniques
Applications – image mosaics
MASKS © 2004
Invitation to 3D vision

36. Overview of the methods
MASKS © 2004
Invitation to 3D vision

37. Summary
MASKS © 2004
Invitation to 3D vision

38. Direct Autocalibration Methods
The fundamental matrix
satisfies the Kruppa’s equations
If the fundamental matrix is known up to scale
Under special motions, Kruppa’s equations become linear.
Solution to Kruppa’s equations is sensitive to noises.
MASKS © 2004
Invitation to 3D vision

39. Direct Stratification from Multiple Views
From the recovered projective projection matrix
we obtain the absolute quadric contraints
Partial knowledge in (e.g. zero skew, square pixel) renders
the above constraints linear and easier to solve.
The projection matrices can be recovered from the multiple-view
rank method to be introduced later.
MASKS © 2004
Invitation to 3D vision

40. Summary of (Auto)calibration Methods
MASKS © 2004
Invitation to 3D vision