Epipolar geometry.

Slides:



Advertisements
Similar presentations
Epipolar Geometry.
Advertisements

Lecture 11: Two-view geometry
3D reconstruction.
MASKS © 2004 Invitation to 3D vision Lecture 7 Step-by-Step Model Buidling.
Two-View Geometry CS Sastry and Yang
Jan-Michael Frahm, Enrique Dunn Spring 2012
Two-view geometry.
Lecture 8: Stereo.
Camera calibration and epipolar geometry
Structure from motion.
Epipolar Geometry Class 7 Read notes Feature tracking run iterative L-K warp & upsample Tracking Good features Multi-scale Transl. Affine.
Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.
Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.
Uncalibrated Geometry & Stratification Sastry and Yang
Epipolar Geometry and the Fundamental Matrix F
Lecture 21: Multiple-view geometry and structure from motion
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Multiple-view Reconstruction from Points and Lines
3D reconstruction of cameras and structure x i = PX i x’ i = P’X i.
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Previously Two view geometry: epipolar geometry Stereo vision: 3D reconstruction epipolar lines Baseline O O’ epipolar plane.
Single-view geometry Odilon Redon, Cyclops, 1914.
3D Computer Vision and Video Computing 3D Vision Lecture 14 Stereo Vision (I) CSC 59866CD Fall 2004 Zhigang Zhu, NAC 8/203A
Projected image of a cube. Classical Calibration.
May 2004Stereo1 Introduction to Computer Vision CS / ECE 181B Tuesday, May 11, 2004  Multiple view geometry and stereo  Handout #6 available (check with.
Lec 21: Fundamental Matrix
Camera parameters Extrinisic parameters define location and orientation of camera reference frame with respect to world frame Intrinsic parameters define.
Structure Computation. How to compute the position of a point in 3- space given its image in two views and the camera matrices of those two views Use.
CS 558 C OMPUTER V ISION Lecture IX: Dimensionality Reduction.
3-D Scene u u’u’ Study the mathematical relations between corresponding image points. “Corresponding” means originated from the same 3D point. Objective.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
776 Computer Vision Jan-Michael Frahm, Enrique Dunn Spring 2013.
Automatic Camera Calibration
Computer vision: models, learning and inference
Lecture 11 Stereo Reconstruction I Lecture 11 Stereo Reconstruction I Mata kuliah: T Computer Vision Tahun: 2010.
Multi-view geometry.
1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained.
Projective cameras Motivation Elements of Projective Geometry Projective structure from motion Planches : –
Course 12 Calibration. 1.Introduction In theoretic discussions, we have assumed: Camera is located at the origin of coordinate system of scene.
Lecture 04 22/11/2011 Shai Avidan הבהרה : החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.
Multiview Geometry and Stereopsis. Inputs: two images of a scene (taken from 2 viewpoints). Output: Depth map. Inputs: multiple images of a scene. Output:
Two-view geometry Epipolar geometry F-matrix comp. 3D reconstruction
Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning.
Geometry of Multiple Views
Single-view geometry Odilon Redon, Cyclops, 1914.
Two-view geometry. Epipolar Plane – plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections of the.
Feature Matching. Feature Space Outlier Rejection.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.
Computer vision: models, learning and inference M Ahad Multiple Cameras
MASKS © 2004 Invitation to 3D vision Uncalibrated Camera Chapter 6 Reconstruction from Two Uncalibrated Views Modified by L A Rønningen Oct 2008.
Reconstruction from Two Calibrated Views Two-View Geometry
Uncalibrated reconstruction Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration.
Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : –
EECS 274 Computer Vision Projective Structure from Motion.
Lec 26: Fundamental Matrix CS4670 / 5670: Computer Vision Kavita Bala.
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
Calibrating a single camera
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
Two-view geometry Computer Vision Spring 2018, Lecture 10
Epipolar Geometry class 11
3D Photography: Epipolar geometry
Estimating 2-view relationships
3D reconstruction class 11
Uncalibrated Geometry & Stratification
Two-view geometry Epipolar geometry F-matrix comp. 3D reconstruction
Two-view geometry.
Two-view geometry.
Multi-view geometry.
Single-view geometry Odilon Redon, Cyclops, 1914.
Presentation transcript:

Epipolar geometry

Three questions: Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point x’ in the second image? Camera geometry (motion): Given a set of corresponding image points {xi ↔x’i}, i=1,…,n, what are the cameras P and P’ for the two views? Or what is the geometric transformation between the views? (iii) Scene geometry (structure): Given corresponding image points xi ↔x’i and cameras P, P’, what is the position of the point X in space?

The epipolar geometry C,C’,x,x’ and X are coplanar

The epipolar geometry All points on p project on l and l’

The epipolar geometry Family of planes p and lines l and l’ Intersection in e and e’

The epipolar geometry epipoles e,e’ = intersection of baseline with image plane = projection of projection center in other image = vanishing point of camera motion direction an epipolar plane = plane containing baseline (1-D family) an epipolar line = intersection of epipolar plane with image (always come in corresponding pairs)

Example: converging cameras

Example: motion parallel with image plane

Example: forward motion

Matrix form of cross product

Geometric transformation

Calibrated Camera Essential matrix

Uncalibrated Camera Fundamental matrix

Properties of fundamental and essential matrix Matrix is 3 x 3 Transpose : If F is essential matrix of cameras (P, P’). FT is essential matrix of camera (P’,P) Epipolar lines: Think of p and p’ as points in the projective plane then F p is projective line in the right image. That is l’=F p l = FT p’ Epipole: Since for any p the epipolar line l’=F p contains the epipole e’. Thus (e’T F) p=0 for a all p . Thus e’T F=0 and F e =0

Fundamental matrix Encodes information of the intrinsic and extrinisic parameters F is of rank 2, since S has rank 2 (R and M and M’ have full rank) Has 7 degrees of freedom There are 9 elements, but scaling is not significant and det F = 0

Essential matrix Encodes information of the extrinisic parameters only E is of rank 2, since S has rank 2 (and R has full rank) Its two nonzero singular values are equal Has only 5 degrees of freedom, 3 for rotation, 2 for translation

Scaling ambiguity Depth Z and Z’ and t can only be recovered up to a scale factor Only the direction of translation can be obtained

Least square approach We have a homogeneous system A f =0 The least square solution is smallest singular value of A, i.e. the last column of V in SVD of A = U D VT

Non-Linear Least Squares Approach Minimize with respect to the coefficients of F Using an appropriate rank 2 parameterization

Locating the epipoles SVD of F = UDVT.

Rectification Image Reprojection reproject image planes onto common plane parallel to line between optical centers

Rectification Rotate the left camera so epipole goes to infinity along the horizontal axis Apply the same rotation to the right camera Rotate the right camera by R Adjust the scale

3D Reconstruction Stereo: we know the viewing geometry (extrinsic parameters) and the intrinsic parameters: Find correspondences exploiting epipolar geometry, then reconstruct Structure from motion (with calibrated cameras): Find correspondences, then estimate extrinsic parameters (rotation and direction of translation), then reconstruct. Uncalibrated cameras: Find correspondences, Compute projection matrices (up to a projective transformation), then reconstruct up to a projective transformation.

Reconstruction by triangulation P’ If cameras are intrinsically and extrinsically calibrated, find P as the midpoint of the common perpendicular to the two rays in space.

Triangulation ap’ ray through C’ and p’, bRp + T ray though C and p expressed in right coordinate system R = ? T = ?

Point reconstruction

X is last column of V in the SVD of A= USVT Linear triangulation Linear combination of 2 other equations homogeneous Homogenous system: X is last column of V in the SVD of A= USVT

geometric error

Non-iterative optimal solution Geometric error Reconstruct matches in projective frame by minimizing the reprojection error Non-iterative optimal solution

Reconstruction for intrinsically calibrated cameras Compute the essential matrix E using normalized points. Select M=[I|0] M’=[R|T] then E=[Tx]R Find T and R using SVD of E

Decomposition of E Four solutions for the decomposition, E can be computed up to scale factor T can be computed up to sign (EET is quadratic) Four solutions for the decomposition, Correct one corresponds to positive depth values

SVD decomposition of E E = USVT

Reconstruction from uncalibrated cameras Reconstruction problem: given xi↔x‘i , compute M,M‘ and Xi for all i without additional information possible only up to projective ambiguity

Projective Reconstruction Theorem Assume we determine matching points xi and xi’. Then we can compute a unique Fundamental matrix F. The camera matrices M, M’ cannot be recovered uniquely Thus the reconstruction (Xi) is not unique There exists a projective transformation H such that

Reconstruction ambiguity: projective

From Projective to Metric Reconstruction Compute homography H such that XEi=HXi for 5 or more control points XEi with known Euclidean position. Then the metric reconstruction is

Rectification using 5 points

Affine reconstructions

From affine to metric Use constraints from scene orthogonal lines Use constraints arising from having the same camera in both images

Reconstruction from N Views Projective or affine reconstruction from a possible large set of images Given a set of camera Mi, For each camera Mi a set of image point xji Find 3D points Xj and cameras Mi, such that MiXj=xji

Bundle adjustment Solve following minimization problem Find Mi and Xj that minimize Levenberg Marquardt algorithm Problems many parameters 11 per camera, 3 per 3d point Useful as final adjustment step for bundles of rays