Distinctive Image Features from Scale-Invariant Keypoints

Note 이 ppt 에 있는 자료들은 Lowe 의 논문들이나 google에서 찾은 feature 관련 paper, ppt 파일들을 참조해서 개인용으로 만든 것입니다. 레퍼런스들을 명시해 놓는 것이 당연하지만 많은 자료들을 짜집기해서 레퍼런스들을 명시하지 못했습니다. 이 점을 감안해서 혹시 그림이나 사진 등을 참고할 생각이시라면 인터넷을 통해 해당하는 paper, ppt, web-page 등을 찾으셔서 레퍼런스로 활용해주시기 바랍니다. ( 레퍼런스 자료들의 제작자 분들께 매우 죄송하다는 말을 전합니다 ㅠㅠ )

Contents Introduction SIFT Overview Test and Result Conclusion Scale-space extrema detection Keypoint localization Orientation assignment Keypoint descriptor Test and Result Conclusion

Introduction Ideal Interest Points / Regions <Image Features>

Introduction Harris Corner Detector Rotation invariance Partial invariance to affine change <Feature Detection>

Introduction Harris Corner Detector Non-invariance to scale change edge corner

Introduction SIFT Application The Scale Invariant Feature Transform Choosing features that are invariant to image scaling and rotation Partially invariant to changes in illumination and 3D camera viewpoint Well localized in both spatial and frequency domain - Resistant to noise, clutter, and occlusion Features are highly distinctive, matched with high probability against large number of features See…… Application Object recognition Automatic mosaic Tracking Robot localization 3D scene modeling Panoramas

Introduction <Object Recognition>

Introduction <Automatic Mosaic>

SIFT overview Detector Scale-space extrema detection Keypoint localization and filtering Orientation assignment Keypoint descriptor Descriptor

SIFT overview Detector Scale-space extrema detection Keypoint localization and filtering Orientation assignment Keypoint descriptor Descriptor

1. Scale-space extrema detection A “good” function for scale detection has one stable sharp peak Good ! f region size bad bad L or DOG(Difference of Gaussians) kernel is a matching filter

1. Scale-space extrema detection DOG(Difference of Gaussians) Construct scale-space Take differences Downsample Convolve with Gaussian

1. Scale-space extrema detection For example Construct scale-space

1. Scale-space extrema detection For example Take differences

1. Scale-space extrema detection Compare a pixel with its 26 neighbors in 3*3 regions at the current and adjacent scales

1. Scale-space extrema detection For example Scale-space extrema

SIFT overview Detector Scale-space extrema detection Keypoint localization and filtering Orientation assignment Keypoint descriptor Descriptor

2. Keypoint localization and filtering Reject points with bad contrast DOG smaller than 0.03 (image values in [0, 1])

2. Keypoint localization and filtering Reject points with strong edge response in one direction only To check if ratio of principal curvature is below some threshold, r

SIFT overview Detector Scale-space extrema detection Keypoint localization and filtering Orientation assignment Keypoint descriptor Descriptor

3. Orientation assignment Descriptor computed relative to keypoint’s orientation achieves rotation invariance Let, for a keypointm L is the image with the closest scale Compute the orientation histogram - within a region around the keypoint (16 Ⅹ 16) Compute gradient magnitude and orientation using finite differences

3. Orientation assignment

3. Orientation assignment

3. Orientation assignment

3. Orientation assignment

3. Orientation assignment

SIFT overview Detector Scale-space extrema detection Keypoint localization and filtering Orientation assignment Keypoint descriptor Descriptor

Keypoint descriptor The computation of the keypoint descriptor A set of keypoints are obtained from each reference image Each such keypoint has a graphical descriptor – which is a 128 component vector (4Ⅹ4Ⅹ8) ← keypoint descriptor’s complexity

Keypoint descriptor Storing

Keypoint descriptor Matching Acceptance of a match Test image gives a new set of (keypoint, vector) pair Find the nearest (top 2) descriptors in database Acceptance of a match Storage using k-d trees - Use the Best-Bin-First(BBF) algorithm Ratio of distance to first nearest descriptor to that of second < threshold (0.8)

Test and Result <Recognition under occlusion> <3D object recognition>

Test and Result <Recognition under differing illumination>

Test and Result <Location recognition>

Test and Result SIFT didn’t work Large illumination change

Test and Result SIFT didn’t work Non-rigid deformation

Conclusion SIFT SIFT extensions A novel method for detecting interest points Invariant to - image scaling - translation - rotation Robust matching across substantial range of - distortion - change in 3D view point - addition of noise - change in illumination SIFT extensions PCA-SIFT SURF Approx SIFT GPU implementation