MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence
MASKS © 2004 Invitation to 3D vision Given an image point in left image, what is the (corresponding) point in the right image, which is the projection of the same 3-D point Image Primitives and Correspondence
MASKS © 2004 Invitation to 3D vision Correspondence Lambertian assumption Rigid body motion Matching - Correspondence
MASKS © 2004 Invitation to 3D vision Translational model Affine model Transformation of the intensity values and occlusions Local Deformation Models
MASKS © 2004 Invitation to 3D vision Translational model RHS approx. by first two terms of Taylor series Small baseline Brightness constancy constraint Feature Tracking and Optical Flow
MASKS © 2004 Invitation to 3D vision Normal flow Aperture Problem
MASKS © 2004 Invitation to 3D vision Integrate around over image patch Solve Optical Flow
MASKS © 2004 Invitation to 3D vision rank(G) = 0 blank wall problem rank(G) = 1 aperture problem rank(G) = 2 enough texture – good feature candidates Conceptually: In reality: choice of threshold is involved Optical Flow, Feature Tracking
MASKS © 2004 Invitation to 3D vision Qualitative properties of the motion fields Previous method - assumption locally constant flow Alternative regularization techniques (locally smooth flow fields, integration along contours) Optical Flow
MASKS © 2004 Invitation to 3D vision Feature Tracking
MASKS © 2004 Invitation to 3D vision Compute eigenvalues of G If smallest eigenvalue of G is bigger than - mark pixel as candidate feature point Alternatively feature quality function (Harris Corner Detector) Point Feature Extraction
MASKS © 2004 Invitation to 3D vision Harris Corner Detector - Example
MASKS © 2004 Invitation to 3D vision Wide Baseline Matching
MASKS © 2004 Invitation to 3D vision Sum of squared differences Normalize cross-correlation Sum of absolute differences Region based Similarity Metric
MASKS © 2004 Invitation to 3D vision Compute image derivatives if gradient magnitude > and the value is a local maximum along gradient direction – pixel is an edge candidate Canny edge detector gradient magnitude original image Edge Detection
MASKS © 2004 Invitation to 3D vision x y Edge detection, non-maximum suppression (traditionally Hough Transform – issues of resolution, threshold selection and search for peaks in Hough space) Connected components on edge pixels with similar orientation - group pixels with common orientation Non-max suppressed gradient magnitude Line fitting
MASKS © 2004 Invitation to 3D vision Line fitting Lines determined from eigenvalues and eigenvectors of A Candidate line segments - associated line quality second moment matrix associated with each connected component Line Fitting
MASKS © 2004 Invitation to 3D vision Examples
MASKS © 2004 Invitation to 3D vision Feature Selection Compute Image Gradient Compute Feature Quality measure for each pixel Search for local maxima Feature Quality FunctionLocal maxima of feature quality function
MASKS © 2004 Invitation to 3D vision Feature Tracking Translational motion model Closed form solution 1. Build an image pyramid 2. Start from coarsest level 3. Estimate the displacement at the coarsest level 4. Iterate until finest level
Coarse to fine feature tracking 1. compute 2. warp the window in the second image by 3. update the displacement 4. go to finer level 5. At the finest level repeat for several iterations 0 2 1
MASKS © 2004 Invitation to 3D vision Integrate around over image patch Solve Optical Flow
MASKS © 2004 Invitation to 3D vision
MASKS © 2004 Invitation to 3D vision
MASKS © 2004 Invitation to 3D vision
MASKS © 2004 Invitation to 3D vision
MASKS © 2004 Invitation to 3D vision Affine feature tracking Intensity offset Contrast change
MASKS © 2004 Invitation to 3D vision Tracked Features
MASKS © 2004 Invitation to 3D vision Wide baseline matching Point features detected by Harris Corner detector
MASKS © 2004 Invitation to 3D vision Wide baseline Feature Matching 1. Select the features in two views 2. For each feature in the first view 3. Find the feature in the second view that maximizes 4. Normalized cross-correlation measure Select the candidate with the similarity above selected threshold