1. 2011.4.14 Reporter: Fei-Fei Chen

4. Wide-baseline matching Object recognition Texture recognition Scene classification Robot wandering Motion tracking Change in illumination 3D camera viewpoint etc.

7. … > 5000 images change in viewing angle

8. 22 correct matches

9. … > 5000 images change in viewing angle + scale change

12. Find corresponding features across two or more views.

13. Elements to be matched are image patches of fixed size Task: Find the best (most similar) patch in a second image.

14. Intuition: This would be a good match for matching, since it is very distinctive.

15. Intuition: This would be a BAD patch for matching, since it is not very distinctive.

16. Intuitively, junctions of contours. Generally more stable features over change of viewpoint. Intuitively, large variations in the neighborhood of the point in all directions. They are good features to match!

17. Detection of Scale-Space Extrema Accuracy Keypoint localization Orientation assignment Keypoint descriptor detectordescriptor

18. For scale invariance, search for stable features across all possible scales using a continuous function of scale, scale space. SIFT uses DoG filter for scale space because it is efficient and as stable as scale-normalized Laplacian of Gaussian.

19. Convolution with a variable-scale Gaussian Difference-of-Gaussian (DoG) filter Convolution with the DoG filter

20.  doubles for the next octave K=2 (1/s) Dividing into octave is for efficiency only.

22. X is selected if it is larger or smaller than all 26 neighbors

23. Reject (1) points with low contrast (flat) (2) poorly localized along an edge (edge) Fit a 3D quadratic function for sub-pixel maxima 1 6 5 0 +1

24. Taylor series of several variables Two variables

25. Taylor expansion in a matrix form, x is a vector, f maps x to a scalar Hessian matrix (often symmetric) gradient

28. x is a 3-vector Remove sample point if offset is larger than 0.5 Throw out low contrast (<0.03)

29. r=10 Let Keep the points with Hessian matrix at keypoint location

30. By assigning a consistent orientation, the keypoint descriptor can be orientation invariant. For a keypoint, L is the Gaussian-smoothed image with the closest scale, orientation histogram (36 bins) (Lx, Ly) m θ

34. σ=1.5*scale of the keypoint

37. accurate peak position is determined by fitting

38. 36-bin orientation histogram over 360°, weighted by m and 1.5*scale falloff Peak is the orientation Local peak within 80% creates multiple orientations About 15% has multiple orientations and they contribute a lot to stability

39. σ=0.5*width Thresholded image gradients are sampled over 16x16 array of locations in scale space Create array of orientation histograms (w.r.t. key orientation) 8 orientations x 4x4 histogram array = 128 dimensions Normalized for intensity variance, clip values larger than 0.2, renormalize

40. Detection of Scale-Space Extrema Accuracy Keypoint localization Orientation assignment Keypoint descriptor For scale invariance For rotation invariance Remove unstable feature points For illumination invariance

41. Image scale invariance. Image rotation invariance. Robust matching across a substantial range of (1) affine distortion, (2) change in 3D viewpoint, (3) addition of noise, (4) change in illumination.

42. For a feature x, he found the closest feature x 1 and the second closest feature x 2. If the distance ratio of d(x, x 1 ) and d(x, x 2 ) is smaller than 0.8, then it is accepted as a match.

58. Thanks for your attention!

59. Q&A