Feature Detection

Description Localization More Points Robust to occlusion Works with less texture More Repeatable Robust detection Precise localization More Robust Deal with expected variations Maximize correct matches More Selective Minimize wrong matches Trade offs smallbig smallbig

Harris Corner Detection FlatEdgeCorner no change in all direction no change along the edge direction large change Concept Shifting the window in any direction should yield a large change in appearance C. Harris and M. Stephens (1988). "A combined corner and edge detector". Proceedings of the 4th Alvey Vision Conference. pp. 147–151.

Harris Corner Detection IntensityShifted intensity Window function By Taylor expansion x, y window center Window-averaged change of intensity for the shift [u,v]: This produces written in matrix form where,

Harris Corner Detection 1, 2 are eigenvalues of M. then the following inferences can be made 1. 2 >> 1 or 1 >> 2 : Edge 2. 1 and 2 are large, 1 ~ 2 : Corner 3. 1 and 2 are small : Flat (k – empirical constant, k = ) Eigenvalue analysis Since the exact computation of the eigenvalues is computationally expensive, the following function is suggested

Harris Corner Detection R depends only on eigenvalues of M R is large for a corner R is negative with large magnitude for an edge |R| is small for a flat region (k – empirical constant, k = ) Corner Edge Flat

Harris Corner Detection Examples

SIFT Descriptor 1. Orientation assignment 2. Keypoint descriptor Detector 1. Scale-space extrema detection 2. Keypoint localization and filtering Scale-invariant feature transform Lowe, David G. (1999). "Object recognition from local scale-invariant features". Proceedings of the International Conference on Computer Vision. 2. pp. 1150– Choosing features that are invariant to image scaling and rotation

SIFT Convolve with Gaussian Downsample # of scales/octave => empirically Find extrema in 3D DoG space Scale-space extrema detection

SIFT Construct scale-space Take differences

SIFT Compare a pixel with its 26 neighbors in 3*3 regions at the current and adjacent scales Identify Min and Max Scale-space extrema detection

SIFT Sub-pixel Localization Fit Trivariate quadratic to find sub- pixel extrema Taylor Series Expansion Differentiate and set to 0 to get location

SIFT There are still a lot of points, some of them are not good enough. Filter Edge and Low Contrast

SIFT Reject points with bad contrast DOG smaller than 0.03 (image values in [0, 1]) Filter Edge and Low Contrast

SIFT Reject points with strong edge response in one direction only To check if ratio of principal curvature is below some threshold, r Filter Edge and Low Contrast

SIFT A histogram is formed by quantizing the orientations into 36 bins; Compute the orientation histogram within a region around the keypoint (16 16) Compute gradient magnitude and orientation using finite differences Orientation assignment

SIFT Peaks in the histogram correspond to the orientations of the patch; - for all peaks with value >= 0.8 max bin Orientation assignment

SIFT Keypoint descriptor Rotate the gradients and coordinates by the previously computer orientation Thresholded image gradients are sampled over 16x16 array of locations in scale space Create array of orientation histograms 8 bins x 4 x 4 histogram array = 128 dimensions

SIFT

Structure from Motion Feature Points Detection Feature Points Matching Relating Image Reconstruction Camera Calibration Dense Matching Bundle Adjustment Feature points Fundamental matrix Camera matrix Sparse reconstructed point Calibration matrix Correspondence point sets Reconstructed point 3D model Structure from Motion Flow