Geometric Transformations EE 7700 Geometric Transformations

Geometric Transformation translation Rotation matrix scale Scale matrix rotation & scale Rigid flow Bahadir K. Gunturk

Affine Flow Bahadir K. Gunturk

Perspective Flow Bahadir K. Gunturk

Bahadir K. Gunturk

Bahadir K. Gunturk

Bahadir K. Gunturk

RANSAC: RANdom SAmple Consensus EE 7730 RANSAC: RANdom SAmple Consensus

Outliers Consider the least squares fit for optical flow: If some of the values are wrong, it will degrade the estimation. Bahadir K. Gunturk

Outliers It is best not to include outliers in the estimation. Line Fitting Problem: Given (x1,y1), …, (xN,yN); find the line y=ax+b Outliers Best fit is degraded due to the outliers Bahadir K. Gunturk

Robust Estimation Two step process: Classify data points as outliers or inliers Use inliers only to fit a model Bahadir K. Gunturk

RANSAC Repeat for k times: Randomly choose n points (the smallest number of points required) from the data. Estimate the parameters using these points. For each data point other than these n points: Check if the data point is within a threshold, t, distance of current model; if it is, the data point is consistent with current model. The total number of data points that are consistent is model’s support. If the support is larger than a predetermined number, d, then there is a good fit. Re-estimate the parameters using these inliers. Choose the best fit with the smallest fitting error. Bahadir K. Gunturk

RANSAC Two samples and their supports for line-fitting Bahadir K. Gunturk

Example Find the perspective parameters Bahadir K. Gunturk from Hartley & Zisserman Bahadir K. Gunturk

Example Apply corner detectors to both images Bahadir K. Gunturk from Hartley & Zisserman Bahadir K. Gunturk

Example Find the best match within a search window. Bahadir K. Gunturk from Hartley & Zisserman Bahadir K. Gunturk

Example Initial match results from Hartley & Zisserman 188 matched features in left image pointing to locations of corresponding right image features Bahadir K. Gunturk

Example Inliers and outliers after RANSAC 89 outliers 99 inliers from Hartley & Zisserman 89 outliers 99 inliers Bahadir K. Gunturk

Panoramic Image Reconstruction Find features Match features Fit parametric model Application: Mosaic construction Bahadir K. Gunturk

EE7730 Stereo Vision

Pinhole Camera Bahadir K. Gunturk

Review: Perspective Projection Bahadir K. Gunturk

Stereo p p’ p p’ scene point image plane optical center Bahadir K. Gunturk

Stereo Constraints M Image plane Epipolar Line Y1 p p’ Y2 X2 O1 Z1 X1 Epipole Focal plane Bahadir K. Gunturk

From Geometry to Algebra P p p’ All vectors shown lie on the same plane. Bahadir K. Gunturk

From Geometry to Algebra P p p’ Bahadir K. Gunturk

Matrix form of cross product a=axi+ayj+azk a×b=|a||b|sin(η)u b=bxi+byj+bzk Bahadir K. Gunturk

The Essential Matrix Essential matrix Bahadir K. Gunturk

A Simple Stereo System Right image: Left image: target reference LEFT CAMERA RIGHT CAMERA baseline Elevation Zw disparity Depth Z Right image: target Left image: reference Zw=0 Bahadir K. Gunturk

Parallel Cameras P Z xl xr f pl pr Ol Or Disparity: T T is the stereo baseline Bahadir K. Gunturk

Stereo View Left View Right View Bahadir K. Gunturk Disparity

Correlation Approach LEFT IMAGE (xl, yl) (0). Essential Equation represents actually the epipolar plane in either the left or the right image (1). Epipolar line in the right image given pl (Epl)Tpr=0 zr = fr extension of the equations in pr = (xr,yr,fr) (2). Epipolar line in the left image given pr (prTE) pl=0 zl = fl For Each point (xl, yl) in the left image, define a window centered at the point Bahadir K. Gunturk

Correlation Approach RIGHT IMAGE (xl, yl) (0). Essential Equation represents actually the epipolar plane in either the left or the right image (1). Epipolar line in the right image given pl (Epl)Tpr=0 zr = fr extension of the equations in pr = (xr,yr,fr) (2). Epipolar line in the left image given pr (prTE) pl=0 zl = fl … search its corresponding point within a search region in the right image Bahadir K. Gunturk

Correlation Approach RIGHT IMAGE (xr, yr) dx (xl, yl) (0). Essential Equation represents actually the epipolar plane in either the left or the right image (1). Epipolar line in the right image given pl (Epl)Tpr=0 zr = fr extension of the equations in pr = (xr,yr,fr) (2). Epipolar line in the left image given pr (prTE) pl=0 zl = fl … the disparity (dx, dy) is the displacement when the correlation is maximum Bahadir K. Gunturk

Stereo results Data from University of Tsukuba Scene Ground truth (Seitz) Bahadir K. Gunturk

Results with window correlation Estimated depth of field Ground truth (Seitz) Bahadir K. Gunturk

Results with better method A state of the art method Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, International Conference on Computer Vision, September 1999. Ground truth Bahadir K. Gunturk (Seitz)

Applications courtesy of Sportvision First-down line Bahadir K. Gunturk

Applications courtesy of Princeton Video Image Virtual advertising Bahadir K. Gunturk