Single View Metrology Class 3

3D photography course schedule (tentative) LectureExercise Sept 26Introduction- Oct. 3Geometry & Camera modelCamera calibration Oct. 10Single View MetrologyMeasuring in images Oct. 17Feature Tracking/matching (Friedrich Fraundorfer) Correspondence computation Oct. 24Epipolar GeometryF-matrix computation Oct. 31Shape-from-Silhouettes (Li Guan) Visual-hull computation Nov. 7Stereo matchingProject proposals Nov. 14Structured light and active range sensing Papers Nov. 21Structure from motionPapers Nov. 28Multi-view geometry and self-calibration Papers Dec. 5Shape-from-XPapers Dec. 123D modeling and registrationPapers Dec. 19Appearance modeling and image-based rendering Final project presentations

Single View Metrology

Measuring in a plane Need to compute H as well as uncertainty

Direct Linear Transformation (DLT)

Equations are linear in h Only 2 out of 3 are linearly independent (indeed, 2 eq/pt) (only drop third row if w i ’≠0) Holds for any homogeneous representation, e.g. ( x i ’, y i ’,1)

Direct Linear Transformation (DLT) Solving for H size A is 8x9 or 12x9, but rank 8 Trivial solution is h =0 9 T is not interesting 1-D null-space yields solution of interest pick for example the one with

Direct Linear Transformation (DLT) Over-determined solution No exact solution because of inexact measurement i.e. “noise” Find approximate solution - Additional constraint needed to avoid 0, e.g. - not possible, so minimize

DLT algorithm Objective Given n≥4 2D to 2D point correspondences {x i ↔x i ’}, determine the 2D homography matrix H such that x i ’=Hx i Algorithm (i)For each correspondence x i ↔x i ’ compute A i. Usually only two first rows needed. (ii)Assemble n 2x9 matrices A i into a single 2 n x9 matrix A (iii)Obtain SVD of A. Solution for h is last column of V (iv)Determine H from h

Importance of normalization ~10 2 ~10 4 ~ orders of magnitude difference! Monte Carlo simulation for identity computation based on 5 points (not normalized ↔ normalized)

Normalized DLT algorithm Objective Given n≥4 2D to 2D point correspondences {x i ↔x i ’}, determine the 2D homography matrix H such that x i ’=Hx i Algorithm (i)Normalize points (ii)Apply DLT algorithm to (iii)Denormalize solution

Geometric distance measured coordinates estimated coordinates true coordinates Error in one image e.g. calibration pattern Symmetric transfer error d (.,.) Euclidean distance (in image) Reprojection error

Statistical cost function and Maximum Likelihood Estimation Optimal cost function related to noise model Assume zero-mean isotropic Gaussian noise (assume outliers removed) Error in one image Maximum Likelihood Estimate

Statistical cost function and Maximum Likelihood Estimation Optimal cost function related to noise model Assume zero-mean isotropic Gaussian noise (assume outliers removed) Error in both images Maximum Likelihood Estimate

Gold Standard algorithm Objective Given n≥4 2D to 2D point correspondences {x i ↔x i ’}, determine the Maximum Likelyhood Estimation of H (this also implies computing optimal x i ’=Hx i ) Algorithm (i)Initialization: compute an initial estimate using normalized DLT or RANSAC (ii)Geometric minimization of reprojection error: ● Minimize using Levenberg-Marquardt over 9 entries of h or Gold Standard error: ● compute initial estimate for optimal {x i } ● minimize cost over {H,x 1,x 2,…,x n } ● if many points, use sparse method

Uncertainty: error in one image (i)Estimate the transformation from the data (ii)Compute Jacobian, evaluated at (iii)The covariance matrix of the estimated is given by

Uncertainty: error in both images separate in homography and point parameters

Using covariance matrix in point transfer Error in one image Error in two images (if h and x independent, i.e. new points)

 =1 pixel  =0.5cm (Criminisi’97) Example:

 =1 pixel  =0.5cm Example: (Criminisi’97)

Example: (Criminisi’97)

Monte Carlo estimation of covariance To be used when previous assumptions do not hold (e.g. non-flat within variance) or to complicate to compute. Simple and general, but expensive Generate samples according to assumed noise distribution, carry out computations, observe distribution of result

Single view measurements: 3D scene

Background: Projective geometry of 1D The cross ratio Invariant under projective transformations 3DOF (2x2-1)

Vanishing points Under perspective projection points at infinity can have a finite image The projection of 3D parallel lines intersect at vanishing points in the image

Basic geometry

Allows to relate height of point to height of camera

Homology mapping between parallel planes Allows to transfer point from one plane to another

Single view measurements

Forensic applications 190.6±2.9 cm 190.6±4.1 cm A. Criminisi, I. Reid, and A. Zisserman. Computing 3D euclidean distance from a single view. Technical Report OUEL 2158/98, Dept. Eng. Science, University of Oxford, 1998.

Example courtesy of Antonio Criminisi

La Flagellazione di Cristo (1460) Galleria Nazionale delle Marche by Piero della Francesca ( )

More interesting stuff Criminisi demo spie98/criminis/index.html spie98/criminis/index.html work by Derek Hoiem on learning single view 3D structure and apps similar work by Ashutosh Saxena on learning single view depth

Next class Feature tracking and matching