Computing the Fundamental matrix Peter Praženica FMFI UK May 5, 2008.

Slides:



Advertisements
Similar presentations
The fundamental matrix F
Advertisements

Lecture 11: Two-view geometry
3D reconstruction.
MASKS © 2004 Invitation to 3D vision Lecture 7 Step-by-Step Model Buidling.
Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer.
Two-View Geometry CS Sastry and Yang
Two-view geometry.
Camera Calibration. Issues: what are intrinsic parameters of the camera? what is the camera matrix? (intrinsic+extrinsic) General strategy: view calibration.
Lecture 8: Stereo.
Camera calibration and epipolar geometry
Structure from motion.
Multiple View Geometry
Parameter estimation class 5 Multiple View Geometry Comp Marc Pollefeys.
Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.
Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.
Uncalibrated Geometry & Stratification Sastry and Yang
Lecture 21: Multiple-view geometry and structure from motion
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Multiple-view Reconstruction from Points and Lines
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Previously Two view geometry: epipolar geometry Stereo vision: 3D reconstruction epipolar lines Baseline O O’ epipolar plane.
Projected image of a cube. Classical Calibration.
May 2004Stereo1 Introduction to Computer Vision CS / ECE 181B Tuesday, May 11, 2004  Multiple view geometry and stereo  Handout #6 available (check with.
ECE 530 – Analysis Techniques for Large-Scale Electrical Systems
Multiple View Geometry
Multiple View Geometry. THE GEOMETRY OF MULTIPLE VIEWS Reading: Chapter 10. Epipolar Geometry The Essential Matrix The Fundamental Matrix The Trifocal.
Structure Computation. How to compute the position of a point in 3- space given its image in two views and the camera matrices of those two views Use.
3-D Scene u u’u’ Study the mathematical relations between corresponding image points. “Corresponding” means originated from the same 3D point. Objective.
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
Automatic Camera Calibration
Computer vision: models, learning and inference
Camera Geometry and Calibration Thanks to Martial Hebert.
Multi-view geometry.
Epipolar geometry The fundamental matrix and the tensor
1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained.
Projective cameras Motivation Elements of Projective Geometry Projective structure from motion Planches : –
© 2005 Yusuf Akgul Gebze Institute of Technology Department of Computer Engineering Computer Vision Geometric Camera Calibration.
Course 12 Calibration. 1.Introduction In theoretic discussions, we have assumed: Camera is located at the origin of coordinate system of scene.
CSCE 643 Computer Vision: Structure from Motion
3D Reconstruction Jeff Boody. Goals ● Reconstruct 3D models from a sequence of at least two images ● No prior knowledge of the camera or scene ● Use the.
Multiview Geometry and Stereopsis. Inputs: two images of a scene (taken from 2 viewpoints). Output: Depth map. Inputs: multiple images of a scene. Output:
Vector Norms and the related Matrix Norms. Properties of a Vector Norm: Euclidean Vector Norm: Riemannian metric:
Computing F. Content Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. Single View: Camera model, Calibration, Single.
Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning.
Parameter estimation. 2D homography Given a set of (x i,x i ’), compute H (x i ’=Hx i ) 3D to 2D camera projection Given a set of (X i,x i ), compute.
Geometry of Multiple Views
Raquel A. Romano 1 Scientific Computing Seminar May 12, 2004 Projective Geometry for Computer Vision Projective Geometry for Computer Vision Raquel A.
A Flexible New Technique for Camera Calibration Zhengyou Zhang Sung Huh CSPS 643 Individual Presentation 1 February 25,
Two-view geometry. Epipolar Plane – plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections of the.
1 Comparative Survey on Fundamental Matrix Estimation Computer Vision and Robotics Group Institute of Informatics and Applications University of Girona.
Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : –
Parameter estimation class 5 Multiple View Geometry CPSC 689 Slides modified from Marc Pollefeys’ Comp
Camera Calibration Course web page: vision.cis.udel.edu/cv March 24, 2003  Lecture 17.
Lec 26: Fundamental Matrix CS4670 / 5670: Computer Vision Kavita Bala.
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
Lecture 16: Image alignment
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
René Vidal and Xiaodong Fan Center for Imaging Science
Parameter estimation class 5
Two-view geometry Computer Vision Spring 2018, Lecture 10
Epipolar geometry.
3D Photography: Epipolar geometry
Epipolar geometry continued
Estimating 2-view relationships
Uncalibrated Geometry & Stratification
Reconstruction.
Two-view geometry.
Two-view geometry.
Multi-view geometry.
Calibration and homographies
Presentation transcript:

Computing the Fundamental matrix Peter Praženica FMFI UK May 5, 2008

Contents Fundamental matrix Linear methods Basic algorithm Normalized Nonlinear methods The distance to epipolar lines The Gradient criterion and an interpretation as a distance The “optimal” (reprojected) method M-Estimator with the Tukey function Monte Carlo - Least-median-of-squares method Example of estimation with comparison

Fundamental matrix -The basic description of the projective geometry of two views - The computation of the Fundamental matrix is of utmost theoretical and practical importance (self-calibration method, recovering the 3D projective geometry) Stage for this chapter - Two views of a scene taken by two (perspective projective) cameras - n (respectively n’) points of interest have been detected in image 1 (respectively in image 2) - Goal: to establish a set of correspondences and to compute the estimate of the Fundamental matrix F that best explains those correspondences

The key idea: Each point correspondence (m, m') yields one equation, therefore with a sufficient number of point correspondences in general position we can determine F. No knowledge about the cameras or scene structure is necessary. Principle: Let and be two corresponding points. If the Fundamental matrix is, then the epipolar constraint can be rearranged as Where Equation 6.1 is linear and homogeneous in the 9 unknown coefficients of matrix F. Thus if we are given 8 matches, then we will be able, in general, to determine a unique solution for F, defined up to a scale factor. This approach is known as the eight-point algorithm. Linear methods (6.1)

In this case, we have more linear equations than unknowns, and, in the presence of noise, we look for the minimum of the error criterion which is equivalent to finding the minimum of (6.3) Where 2 solutions 1) closed-form solution via the linear equations 2) solution of subject to LM: Least-squares solution

Closed-form solution via the linear equations: One of the coefficients of F is set to 1, yielding a parameterization of F by eight values, which are the ratios of the eight other coefficients to the normalizing one. Let us assume for the sake of discussion that the normalizing coefficient is F33. We can rewrite (6.3) as where is formed with the first 8 columns of, and b is the ninth column of. The solution is known to be if the matrix is invertible. The second method solves the classical problem with The solution is the eigenvector associated with the smallest eigenvalue of.

Linear methods: Discussion The advantage of the eight-point algorithm is that it leads to a noniterative computation method which is easy to implement using any linear algebra numerical package; however, we have found that it is quite sensitive to noise, even with numerous data points. The constraint det(F) = 0 is not satisfied The quantity that is minimized in (6.2) does not have a good geometric interpretation, one could say that one might be trying to minimize an irrelevant quantity In all of the linear estimation techniques that will be described in this section, we deal with projective coordinates of the pixels. In a typical image of dimensions 512 x 512, a typical image point will have projective coordinates whose orders of magnitude are given componentwise by the triple (250,250,1). The fact that the first two coordinates are more than one order of magnitude larger than the third one has the effect of causing poor numerical conditioning. Solution: normalizing the points in the two images so that their three projective coordinates are of roughly the same order.

LM: Enforcing the rank constraint Two-step approximate linear determination of parameters: The idea is to note that in Equation 5.10 of Proposition 5.12, if the epipoles are known, then the Fundamental matrix is a linear function of the coefficients (a, b, c, d) of the epipolar homography. 1. We determine the epipole e by solving the constrained minimization problem subject to, which yields e as the unit norm eigenvector of matrix corresponding to the smallest eigenvalue 2. Once the epipoles are known, we can use Equation 5.10 or, if the epipoles are not at infinity, Equation 5.14, and write the expression as a linear expression of the coefficients of the epipolar homography: where (respectively ) are the affine coordinates of (respectively ) This equation can be used to solve for the coefficients of the epipolar homography in a least-squares formulation, thereby completing the determination of the parameters of F.

LM: Enforcing the rank constraint Using the closest rank 2 matrix: The matrix F found by the eight-point algorithm is replaced by the singular matrix that minimizes the Frobenius norm llF - F'll. This step can be done using a singular value decomposition. If, with, then. Solving a cubic equation: One selects two components of F, say, without loss of generality, F32 and F33 and rewrites (6.3) as The vector h is made of the first 7 components of the vector f, the matrix is made of the first 7 columns of matrix, and the vectors W 8 and W 9 are the eighth and ninth column vectors of that matrix, respectively. The solution h is We have a linear family of solutions parametrized by F32 and F33. In that family of solutions, there are some that satisfy the condition det(F) = 0. They are obtained by expressing the determinant of the solutions as a function of F32 and F33. One obtains a homogeneous polynomial of degree 3 which must be zero. Solving for one of the two ratios F32 : F33 or F33 : F32 one obtains at least one real solution for the ratio and therefore at least one solution for the Fundamental matrix.

Nonlinear methods The distance to epipolar lines The idea is to use a non-quadratic error function and to minimize However, two images should play the symmetric roles It can be written using the fact that. It can be observed that this criterion does not depend on the scale factor used to compute F. The Gradient criterion and an interpretation as a distance Uses criterion

Nonlinear methods The “optimal” method The nonlinear method based on the minimization of distances between observed points and reprojected ones where f is a vector of parameters for F, {Mi} are the 3D coordinates of the points in space, and P(f), P’(f) are the perspective projection matrices in the first and second images, given the Fundamental matrix F(f). That is, we estimate not only F, but also the most likely true positions. Because of the epipolar constraint, there are degrees of freedom for each point correspondence, so the total number of parameters is 3p + 7. The estimation of the Fundamental matrix parameters f can be decoupled from the estimation of the parameters Mi : Since the optimal parameters can be determined from the parameters f, we have to minimize only over f. At each iteration, we have to perform the projective reconstruction of p points. The problem can be rewritten as

M-Estimator Let be the residual of the i-th datum. For example, in the case of the linear methods, in the case of the nonlinear methods The standard least-squares method tries to minimize, which is unstable if there are outliers present in the data. The M-estimators try to reduce the effect of outliers by replacing the squared residuals by another function of the residuals, yielding the error function where p is a symmetric, positive function with a unique minimum at zero and is chosen to be growing slower than the square function. The M-estimator of f based on the function is a vector f which is a solution of the system of q equations (6.16) where the derivative is called the influence function. If we now define define a weight function then Equation 6.16 becomes

M-Estimator with the Tukey function otherwise where is some estimated standard deviation of errors and c = is the tuning constant. The corresponding weight function is otherwise

Monte Carlo - Least-median-of-squares method It estimates the parameters by solving the nonlinear minimization problem Given p point correspondences,we proceed through the following steps 1) A Monte Carlo type technique is used to draw m random subsamples of q = 7 different point correspondences 2) For each subsample, indexed by J, we use the technique described in Proposition 5.14 to compute the Fundamental matrix F J. We may have at most 3 solutions 3) Compute the median, denoted by M J, of the squared residuals 4) Retain the estimate F J for which M J is minimal among all m M J ’s

Example of estimation with comparison - Pair of calibrated stereo images - There are 241 point matches which are established automatically - Outliers have been discarded - The calibrated parameters of the cameras are not used, but the Fundamental matrix computed from these parameters serves as a ground truth Methodee’RMSCPU Calibrated Linear5.85% s Normalization7.20% s Nonlinear0.92% s Gradient0.92% s M-estimator0.12% s Reprojected0.92% s LMedS0.13% s (+ each F is normalized by its Frobenius norm) e, e’ positions of 2 epipoles RMS = squared distances between points and their epipolar lines CPU = approximate CPU time in seconds when the program is run on a Sparc 20 workstation

Better comparison method 1)Choose randomly a point m in the first image 2)Draw epipolar line of m in the second image using F 1 3)If the epipolar line does not intersect the second image, then go to Step 1 4)Choose randomly a point m’ on the epipolar line. Note that m and m’ correspond to each other exactly with respect to F 1 5)Draw the epipolar line of m in the second image using F 2, and compute the distance, denoted by d’ 1, between point m’ and line F 2 m 6)Draw the epipolar line of m’ in the first image using F 2, and compute the distance, denoted by d 1, between point m and line F 2 Tm’ 7)Conduct the same procedure from Step 2 through Step 6, but reversing the roles of F 1 and F 2, and compute d 2 and d’ 2 8)Repeat Step 1 through Step 7 N times 9)Compute the average distance of d’s, which is the measure of difference between the two Fundamental matrices

New comparason LinearNormalizedNonlinearGradientM-EstimatorReprojectedLMeDS Calibrated Linear Normalized Nonlinear Gradient M-Estimator Reprojected1.88 Linear method is very bad The linear method with prior data normalization gives quite a reasonable result The nonlinear method based on point-line distances and that based on gradient- weighted epipolar errors give very similar result to those obtained based on minimization of distances between observed points and reprojected ones. The latter should be avoided because it is too time consuming. M-Estimators or the LMedS method give still better results because they try to limit or eliminate the effect of poorly localized points. The epipolar geometry estimated by LMedS is closer to the one computed through stereo calibration.