# The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.

## Presentation on theme: "The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views."— Presentation transcript:

The Trifocal Tensor Multiple View Geometry

Scene planes and homographies plane induces homography between two views

6-point algorithm x 1,x 2,x 3,x 4 in plane, x 5,x 6 out of plane Compute H from x 1,x 2,x 3,x 4

Three-view geometry

The trifocal tensor Three back-projected lines have to meet in a single line Incidence relation provides constraint on lines Let us derive the corresponding algebraic constraint…

Notations

Incidence e.g.  is part of bundle formed by  ’ and  ”

Incidence relation

The Trifocal Tensor Trifocal Tensor = {T 1,T 2,T 3 } Only depends on image coordinates and is thus independent of 3D projective basis Also and but no simple relation General expression not as simple as DOF T: 3x3x3=27 elements, 26 up to scale 3-view relations: 11x3-15=18 dof 8(=26-18) independent algebraic constraints on T (compare to 1 for F, i.e. rank-2)

Homographies induced by a plane

Line-line-line relation Eliminate scale factor: (up to scale)

Point-line-line relation

Point-line-point relation note: valid for any line through x”, e.g. l”=[x”] x x” arbitrary

Point-point-point relation note: valid for any line through x’, e.g. l’=[x’] x x’ arbitrary

Overview incidence relations

Non-incident configuration incidence in image does not guarantee incidence in space

Epipolar lines if l’ is epipolar line, then satisfied for arbitrary l” inversely, epipolar lines are right and left null-space of

Epipoles With points becomes respectively Epipoles are intersection of right resp. left null-space of (e=P’C and e”=P”C)

Algebraic properties of T i matrices

Extracting F good choice for l” is e” (V 3 T e”=0)

Computing P,P‘,P“ ? ok, but not specifically, (no derivation)

matrix notation is impractical Use tensor notation instead

Definition affine tensor Collection of numbers, related to coordinate choice, indexed by one or more indices Valency = ( n+m ) Indices can be any value between 1 and the dimension of space ( d (n+m) coefficients)

Conventions Einstein’s summation: (once above, once below) Index rule: Contravariant indices Covariant indices

More on tensors Transformations (covariant) (contravariant)

Some special tensors Kronecker delta Levi-Cevita epsilon (valency 2 tensor) (valency 3 tensor)

Trilinearities

Compute F and P from T

matrix notation is impractical Use tensor notation instead

Definition affine tensor Collection of numbers, related to coordinate choice, indexed by one or more indices Valency = ( n+m ) Indices can be any value between 1 and the dimension of space ( d (n+m) coefficients)

Conventions Contraction: (once above, once below) Index rule:

More on tensors Transformations (covariant) (contravariant)

Some special tensors Kronecker delta Levi-Cevita epsilon (valency 2 tensor) (valency 3 tensor)

Trilinearities

Transfer: epipolar transfer

Transfer: trifocal transfer Avoid l’=epipolar line

Transfer: trifocal transfer point transfer line transfer degenerate when known lines are corresponding epipolar lines

Computation of Trifocal Tensor Linear method (7-point) Minimal method (6-point) Geometric error minimization method RANSAC method

Basic equations Three points Correspondence Relation #lin. indep.Eq. 4 Two points, one line One points, two line 2 1 2 Three lines At=0 (26 equations) (more equations) min||At|| with ||t||=1

Normalized linear algorithm At=0 Points Lines or Normalization: normalize image coordinates to ~1

Normalized linear algorithm Objective Given n  7 image point correspondences across 3 images, or a least 13 lines, or a mixture of point and line corresp., compute the trifocal tensor. Algorithm (i)Find transformation matrices H,H’,H” to normalize 3 images (ii)Transform points with H and lines with H -1 (iii)Compute trifocal tensor T from At=0 (using SVD) (iv)Denormalize trifocal tensor

Internal constraints 27coefficients 1 free scale 18 parameters 8 internal consistency constraints (not every 3x3x3 tensor is a valid trifocal tensor!) (constraints not easily expressed explicitly) Trifocal Tensor satisfies all intrinsic constraints if it corresponds to three cameras {P,P’,P”}

Maximum Likelihood Estimation data cost function parameterization (24 parameters+3N) also possibility to use Sampson error (24 parameters)

Objective Compute the trifocal tensor between two images Algorithm (i)Interest points: Compute interest points in each image (ii)Putative correspondences: Compute interest correspondences (and F) between 1&2 and 2&3 (iii)RANSAC robust estimation: Repeat for N samples (a) Select at random 6 correspondences and compute T (b) Calculate the distance d  for each putative match (c) Compute the number of inliers consistent with T (d  { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/14/4258419/slides/slide_46.jpg", "name": "Objective Compute the trifocal tensor between two images Algorithm (i)Interest points: Compute interest points in each image (ii)Putative correspondences: Compute interest correspondences (and F) between 1&2 and 2&3 (iii)RANSAC robust estimation: Repeat for N samples (a) Select at random 6 correspondences and compute T (b) Calculate the distance d  for each putative match (c) Compute the number of inliers consistent with T (d 

108 putative matches 18 outliers 88 inliers 95 final inliers (26 samples) (0.43) (0.23) (0.19)

Matrix formulation for m-View Consider one object point X and its m images: i x i =P i X i, i=1, ….,m: i.e. rank(M) < m+4.

Laplace expansions The rank condition on M implies that all (m+4)*(m+4) minors of M are equal to 0. These can be written as sums of products of camera matrix parameters and image coordinates.

Matrix formulation for non-trivially zero minors, one row has to be taken from each image (m). 4 additional rows left to choose

only interesting if 2 or 3 rows from view

The three different types 1.Take the 2 remaining rows from one image block and the other two from another image block, gives the 2-view constraints. 2.Take the 2 remaining rows from one image block 1 from another and 1 from a third, gives the 3-view constraints. 3.Take 1 row from each of four different image blocks, gives the 4-view constraints.

Download ppt "The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views."

Similar presentations