Camera Models CMPUT 498/613 Richard Hartley and Andrew Zisserman, Multiple View Geometry, Cambridge University Publishers, 2000 Readings: HZ Ch 6, 7

Hierarchy of cameras Camera center Image plane Object plane X 0 (origin) x persp Perspective: x parap Para-perspective: First order approximation of perspective x orth Orthographic: Weak perspective: x wp

Examples of camera projections perspectiveOrthographic (parallel)

Animal eye: a looonnng time ago. Pinhole perspective projection: Brunelleschi, XV th Century. Camera obscura: XVI th Century. Photographic camera: Niepce, 1816.

Pinhole camera model

Principal point offset principal point

Principal point offset calibration matrix

Camera rotation and translation

CCD camera

Finite projective camera non-singular 11 dof (5+3+3) decompose P in K,R,C? {finite cameras}={P 4x3 | det M≠0} If rank P=3, but rank M<3, then cam at infinity

Camera anatomy Camera center Column points Principal plane Axis plane Principal point Principal ray

Column vectors Image points corresponding to X,Y,Z directions and origin

Row vectors note: p 1,p 2 dependent on image reparametrization Camera center null-space camera projection matrix, (3 plane intersection)

The principal point principal point

Action of projective camera on point Forward projection Back-projection (pseudo-inverse)

Camera matrix decomposition Finding the camera center (use SVD to find null-space) Finding the camera orientation and internal parameters (use RQ decomposition ~QR) Q R =( ) -1 = Q R (if only QR, invert)

When is skew non-zero? 1  arctan(1/s) for CCD/CMOS, almost always s=0 Image from image, s≠0 possible (non coinciding principal axis) resulting camera:

Euclidean vs. projective general projective interpretation Meaningfull decomposition in K,R,t requires Euclidean image and space Camera center is still valid in projective space Principal plane requires affine image and space Principal ray requires affine image and Euclidean space

Cameras at infinity Camera center at infinity Affine and non-affine cameras Definition: affine camera has P 3T =(0,0,0,1)

Affine cameras

modifying p 34 corresponds to moving along principal ray

Affine cameras now adjust zoom to compensate

Error in employing affine cameras point on plane parallel with principal plane and through origin, then general points

Affine imaging conditions Approximation should only cause small error  much smaller than d 0 2.Points close to principal point (i.e. small field of view)

Decomposition of P ∞ absorb d 0 in K 2x2 alternatives, because 8dof (3+3+2), not more

Summary parallel projection canonical representation calibration matrix principal point is not defined

A hierarchy of affine cameras Orthographic projection Scaled orthographic projection (5dof) (6dof)

A hierarchy of affine cameras Weak perspective projection (useful infinite appr. of unknown CCD camera) (7dof)

1.Affine camera=camera with principal plane coinciding with  ∞ 2.Affine camera maps parallel lines to parallel lines 3.No center of projection, but direction of projection P A D=0 (point on  ∞ ) A hierarchy of affine cameras Affine camera (8dof)

Hierarchy of cameras Camera center Image plane Object plane X 0 (origin) x persp Perspective: x parap Para-perspective: First order approximation of perspective x orth Orthographic: Weak perspective: x wp

Other Cameras Pushbroom cameras Straight lines are not mapped to straight lines! (otherwise it would be a projective camera) (11dof) Line cameras (5dof) Null-space PC=0 yields camera center Also decomposition

Projection equationProjection equation x i =P i X x i =P i X Resection:Resection: –x i,X P i A 3D Vision Problem: Multi-view geometry - resection Given image points and 3D points calculate camera projection matrix.

Estimating camera matrix P Given a number of correspondences between 3- D points and their 2-D image projections X i  x i, we would like to determine the camera projection matrix P such that x i = PX i for all iGiven a number of correspondences between 3- D points and their 2-D image projections X i  x i, we would like to determine the camera projection matrix P such that x i = PX i for all i

A Calibration Target courtesy of B. Wilburn XZ Y XiXi xixi

Estimating P : The Direct Linear Transformation (DLT) Algorithm x i = PX i is an equation involving homogeneous vectors, so PX i and x i need only be in the same direction, not strictly equalx i = PX i is an equation involving homogeneous vectors, so PX i and x i need only be in the same direction, not strictly equal We can specify “same directionality” by using a cross product formulation:We can specify “same directionality” by using a cross product formulation:

DLT Camera Matrix Estimation: Preliminaries Let the image point x i = (x i, y i, w i ) T (remember that X i has 4 elements)Let the image point x i = (x i, y i, w i ) T (remember that X i has 4 elements) Denoting the j th row of P by p jT (a 4-element row vector), we have:Denoting the j th row of P by p jT (a 4-element row vector), we have:

DLT Camera Matrix Estimation: Step 1 Then by the definition of the cross product,Then by the definition of the cross product, x i  PX i is: x i  PX i is:

DLT Camera Matrix Estimation: Step 2 The dot product commutes, so p jT X i = X T i p j, and we can rewrite the preceding as:The dot product commutes, so p jT X i = X T i p j, and we can rewrite the preceding as:

DLT Camera Matrix Estimation: Step 3 Collecting terms, this can be rewritten as a matrix product:Collecting terms, this can be rewritten as a matrix product: where 0 T = (0, 0, 0, 0). This is a 3 x 12 matrix times a 12-element column vector p = (p 1T, p 2T, p 3T ) T

What We Just Did

DLT Camera Matrix Estimation: Step 4 There are only two linearly independent rows hereThere are only two linearly independent rows here –The third row is obtained by adding x i times the first row to y i times the second and scaling the sum by -1/w i

DLT Camera Matrix Estimation: Step 4 So we can eliminate one row to obtain the following linear matrix equation for the i th pair of corresponding points:So we can eliminate one row to obtain the following linear matrix equation for the i th pair of corresponding points: Write this as A i p = 0Write this as A i p = 0

DLT Camera Matrix Estimation: Step 5 Remember that there are 11 unknowns which generate the 3 x 4 homogeneous matrix P (represented in vector form by p )Remember that there are 11 unknowns which generate the 3 x 4 homogeneous matrix P (represented in vector form by p ) Each point correspondence yields 2 equations (the two row of A i )Each point correspondence yields 2 equations (the two row of A i )  We need at least 5 ½ point correspondences to solve for p Stack A i to get homogeneous linear system A p = 0Stack A i to get homogeneous linear system A p = 0

Experiment: