1. Estimating 2-view relationships
CMPUT 499/615

2. Fundamental matrix Algebraic representation of epipolar geometry
[Faugeras ’92, Hartley ’92 ]
Algebraic representation of epipolar geometry
Step 1: X on a plane 
Step 2: epipolar line l’ F
3x3, Rank 2, det(F)=0
Linear sol. – 8 corr. Points (unique)
Nonlinear sol. – 7 corr. points (3sol.)
Very sensitive to noise & outliers
Epipolar lines:
Epipoles:
Projection matrices:

3. The projective reconstruction theorem
If a set of point correspondences in two views determine the fundamental matrix uniquely, then the scene and cameras may be reconstructed from these correspondences alone, and any two such reconstructions from these correspondences are projectively equivalent
allows reconstruction from pair of uncalibrated images!

4. Upcoming: 2view Reconstuction
Objective
Given two uncalibrated images compute (PM,P‘M,{XMi})
(i.e. within similarity of original scene and cameras)
Algorithm
Compute projective reconstruction (P,P‘,{Xi})
Compute F from xi↔x‘i
Compute P,P‘ from F
Triangulate Xi from xi↔x‘i
Rectify reconstruction from projective to metric
Direct method: compute H from control points
Stratified method:
Affine reconstruction: compute p∞
Metric reconstruction: compute IAC w

5. separate known from unknown
Epipolar geometry: basic equation
separate known from unknown
(data)
(unknowns)
(linear)

6. 8-point algorithm
Solve for nontrivial solution using SVD:
Var subst:
Now Min
Hence x = last vector in V

7. ! the NOT normalized 8-point algorithm Orders of magnitude difference
~10000
~100 1 !
Orders of magnitude difference
Between column of data matrix
 least-squares yields poor results

8. Transform image to ~[-1,1]x[-1,1]
the normalized 8-point algorithm
Transform image to ~[-1,1]x[-1,1]
(0,0)
(700,500)
(700,0)
(0,500)
(1,-1)
(0,0)
(1,1)
(-1,1)
(-1,-1)
normalized least squares yields good results (Hartley, PAMI´97) 8

9. the singularity constraint
Non-singular F Singular F

10. the singularity constraint
SVD from linearly computed F matrix (rank 3)
Compute closest rank-2 approximation

11. Geometric distance: Gold standard
Maximum Likelihood Estimation
(= least-squares for Gaussian noise)
Initialize: normalized 8-point, (P,P‘) from F, reconstruct Xi
Parameterize:
(overparametrized)
Minimize cost using Levenberg-Marquardt
(preferably sparse LM, see book)

12. Some experiments:

13. Some experiments:

14. Some experiments:

15. Automatic computation of F
Recommendations:
Do not use unnormalized algorithms
Quick and easy to implement: 8-point normalized
Better: enforce rank-2 constraint during minimization
Best: Maximum Likelihood Estimation (minimal parameterization, sparse implementation)
Automatic computation of F
Interest points
Putative correspondences
RANSAC
(iv) Non-linear re-estimation of F
Guided matching
(repeat (iv) and (v) until stable)

16. The envelope of epipolar lines
What happens to an epipolar line if there is noise?
Monte Carlo
n=10
n=15
n=25
n=50

17. Other entities? Lines give no constraint for two view geometry
(but will for three and more views)
Curves and surfaces yield some constraints related to tangency

18. Structure from images: 3D Point reconstruction

19. Solve with SVD for nullvector
linear triangulation
Solve with SVD for nullvector
homogeneous
inhomogeneous

20. Linear triangulation Alternative way of linear intersection:
Formulate a set of linear equations explicitly solving for l’s
See our VR2003 tutorial p. 26

21. geometric error possibility to compute using LM (for 2 or more points)
or directly (for 2 points)

22. Geometric error Reconstruct matches in projective frame
by minimizing the reprojection error
Non-iterative optimal solution
(see Hartley&Sturm,CVIU´97)

23. Optimal 3D point in epipolar plane
Given an epipolar plane, find best 3D point for (x1,x2) x1 x2 l1 l2 l1 x1 x2 l2
x1´
x2´
Select closest points (x1´,x2´) on epipolar lines
Obtain 3D point through exact triangulation
Guarantees minimal reprojection error
(given this epipolar plane)

24. Optimal epipolar plane
Reconstruct matches in projective frame by minimizing the reprojection error
Non-iterative method
Determine the epipolar plane for reconstruction
Reconstruct optimal point from selected epipolar plane
3DOF
(Hartley and Sturm, CVIU´97)
1DOF m1 m2
l1(a)
l2(a)
(polynomial of degree 6
check all minima, incl ∞)

25. consider angle between rays
Reconstruction uncertainty
consider angle between rays

26. doesn‘t work for epipolar plane
Line reconstruction
doesn‘t work for epipolar plane