Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, , 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see

Step 1. Extract features Step 2. Compute a set of potential matches Step 3. do Step 3.1 select minimal sample (i.e. 7 matches) Step 3.2 compute solution(s) for F Step 3.3 determine inliers until  (#inliers,#samples)<95% #inliers90%80%70%60%50% #samples Step 4. Compute F based on all inliers Step 5. Look for additional matches Step 6. Refine F based on all correct matches (generate hypothesis) (verify hypothesis) Automatic computation of F

Abort verification early Given n samples and an expected proportion of inliers p, how likely is it that I have observed less than T inliers? abort if P<0.02 (initial sample most probably contained outliers) (inspired from Chum and Matas BMVC2002) OOOOOIOOIOOOOOIOOOOOOOIOOOOOIOIOOOOOOOO OIOIIIIOIIIOIOIIIIOOIOIIIIOIOIOIIIIIIII (use normal approximation to binomial) To avoid problems this requires to also verify at random! (but we already have a random sampler anyway)

restrict search range to neighborhood of epipolar line (e.g.  1.5 pixels) relax disparity restriction (along epipolar line) Finding more matches

Degenerate cases Planar scene Pure rotation No unique solution Remaining DOF filled by noise Use simpler model (e.g. homography) Solution 1: Model selection (Torr et al., ICCV´98, Kanatani, Akaike) Compare H and F according to expected residual error (compensate for model complexity) Solution 2: RANSAC Compare H and F according to inlier count (see next slide) Degenerate cases:

RANSAC for (quasi-)degenerate cases Full model (8pts, 1D solution) Sample for out of plane points among outliers closest rank-6 of A nx9 for all plane inliers (accept inliers to solution F) (accept inliers to solution F 1,F 2 &F 3 ) Planar model (6pts, 3D solution) Accept if large number of remaining inliers Plane+parallax model (plane+2pts) 80% in plane 2% out plane 18% outlier

Absence of sufficient features (no texture) Repeated structure ambiguity (Schaffalitzky and Zisserman, BMVC‘98) Robust matcher also finds Robust matcher also finds support for wrong hypothesis support for wrong hypothesis solution: detect repetition solution: detect repetition More problems:

RANSAC for ambiguous matching Include multiple candidate matches in set of potential matches Select according to matching probability (~ matching score) Helps for repeated structures or scenes with similar features as it avoids an early commitment, but also useful in general (Tordoff and Murray ECCV02)

geometric relations between two views is fully described by recovered 3x3 matrix F two-view geometry

Triangulation (finally!) C1C1 x1x1 L1L1 x2x2 L2L2 X C2C2 Triangulation - calibration - correspondences

Triangulation Backprojection Triangulation Iterative least-squares Maximum Likelihood Triangulation C1C1 x1x1 L1L1 x2x2 L2L2 X C2C2

Optimal 3D point in epipolar plane Given an epipolar plane, find best 3D point for (m 1,m 2 ) m1m1 m2m2 l1l1 l2l2 l1l1 m1m1 m2m2 l2l2 m1´m1´ m2´m2´ Select closest points (m 1 ´,m 2 ´) on epipolar lines Obtain 3D point through exact triangulation Guarantees minimal reprojection error (given this epipolar plane)

Non-iterative optimal solution Reconstruct matches in projective frame by minimizing the reprojection error Non-iterative method Determine the epipolar plane for reconstruction Reconstruct optimal point from selected epipolar plane Note: only works for two views (Hartley and Sturm, CVIU´97) (polynomial of degree 6) m1m1 m2m2 l 1  l 2  3DOF 1DOF

Backprojection Represent point as intersection of row and column Useful presentation for deriving and understanding multiple view geometry (notice 3D planes are linear in 2D point coordinates) Condition for solution?

Multi-view geometry (intersection constraint) (multi-linearity of determinants) (= epipolar constraint!) (counting argument: 11x2-15=7)

Multi-view geometry (multi-linearity of determinants) (= trifocal constraint!) (3x3x3=27 coefficients) (counting argument: 11x3-15=18)

Multi-view geometry (multi-linearity of determinants) (= quadrifocal constraint!) (3x3x3x3=81 coefficients) (counting argument: 11x4-15=29)

Next class: rectification and stereo image I(x,y) image I´(x´,y´) Disparity map D(x,y) (x´,y´)=(x+D(x,y),y)