Camera from F Computing K from 1 image HZ8.8 IAC and K HZ8.5 Camera matrix from F HZ9.5 IAC HZ , 8.5 Extracting camera parameters HZ6.2 Camera matrix HZ6.1 Calibration using Q* HZ19.3 Hartley 92

F does not uniquely identify a camera pair a fundamental matrix F relates two images, and therefore two cameras F already gives indirect information about the camera through the epipole (the null vector of F) we will now see that much fuller information about both cameras can be extracted from F, in particular the two camera matrices but a fundamental matrix does not uniquely identify two camera matrices Theorem: Let H be a homography of 3-space. The fundamental matrix associated with camera pair (P,P’) is the same as the fundamental matrix associated with camera pair (PH, P’H). Proof: if (x = PX, x’ = P’X) is a point correspondence in the 1 st camera system arising from 3d point X, then (x = (PH)(H -1 X), x’ = (P’H)(H -1 X)) is a point correspondence in the 2 nd camera system arising from 3d point H -1 X thus, upon finding F, we only know the camera matrix up to a homography analogous to affine rectification: extra work is required to remove this projective freedom and get to metric structure of the camera (stay tuned!) luckily this is the only degree of freedom: if two camera pairs have the same F, then they must differ by a homography (Thm 9.10) HZ254

F does not depend on world frame camera matrix depends on image frame and world frame, since x = PX is dependent on the image coordinates x and the world coordinates X but the above result is telling us that the fundamental matrix is independent of the world frame we are happy to relax to within a similarity of the truth, but we are relaxing to within a projectivity of the truth, which pleases us less –we will need calibration techniques –(these calibration techniques may be interpreted as ways of protecting the absolute conic) HZ253

Camera matrix from F let’s see what we can get from F we assume that the first camera matrix is always [I 0] –we can always normalize so that it is, using some homography we are solving for the “relative offset” of the 2 nd camera from the 1 st Lemma: F is fundamental matrix of camera pair (P,P’) iff P’ t FP is skew-symmetric. Proof: P’ t FP is skew-symmetric  X t (P’ t FP) X = 0 for all X  x’t F x = 0 where x’ = P’X and x = PX, the images of X  F is fundamental matrix of (P,P’) Theorem: The camera matrices associated with F may be chosen to be [I 0] and [SF e’] where –e’^t F = 0 (2 nd image epipole) –S = any skew-symmetric matrix –proof: just check that [SF e’]^t F [I 0] is skew-symmetric Corollary: The camera matrices associated with F may be chosen to be [I 0] and [[e’]x F e’]. –choose S = [e’]x to guarantee a rank 3 camera matrix P’ –see proof on 256 –ironic: to guarantee rank 3 P’ (necessary, so good), we force rank 2 M’ (a bit bad) HZ255-6

Pseudoinverse P + the camera matrix P is rectangular (3x4) so it does not have an inverse but we want to use the notion of inverse to talk about the preimage of an image point a rectangular matrix P has a pseudoinverse P + P + = (P t P) -1 P t –note that P t P is square –note that P + degenerates to P -1 when P is square and nonsingular x = A + b is the least-squares solution of Ax=b if A is full rank (Trefethen 81)

F from P two camera matrices do uniquely specify a fundamental matrix let P and P’ be two camera matrices, and e’ the epipole in the 2 nd image –note: e’ can be calculated as P’C (image of other camera center) the associated fundamental matrix of these 2 cameras: –F = [e’] x P’ P + –note: P’P + is the homography H in the earlier definition of F = [e’] x H –proof idea: epipolar line of x is built from the epipole e’ and the camera-2- image of a typical point of the camera-1-preimage of x (or P’ P + x) –proof: Fx = L’ = e’ x P’ (P + x) so F = [e’] x P’ P + note how the fundamental and camera matrices are independent of the actual contents of the image; only dependent on the camera setup HZ244