1. Scale Invariant Feature Transform (SIFT)
David G. Lowe
University of British Columbia 1

2. References
[1] D.G. Lowe, “Object recognition from local scale-invariant features,” in Proc. Seventh IEEE Int’l Conf. Computer Vision, vol. 2, pp , [2] D.G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int’l J. Computer Vision, vol. 2, no. 60, pp , 2004. 2

3. Outline
Introduction
Algorithm
Applications
Conclusions 3

4. Outline
Introduction
Algorithm
Applications
Conclusions 4

5. An Example: Build a Panorama5

6. How Do We Build Panorama?
We need to match (align) images 6

7. Matching with Features
Detect feature points in both images
Find corresponding pairs
Use these pairs to align images 7

8. An Example: Harris Detector
It is non-invariant to image scale
All points will be classified as edges
Corner 8

9. Scale Invariant Detection
Consider regions (e.g. circles) of different sizes around a point
Regions of corresponding sizes will look the same in both images 9

10. Scale Invariant Detection
The problem: how do we choose corresponding circles independently in each image? 10

11. Laplacian of Gaussian Operator
Maxima of Laplacian gives best notion of scale:
Characteristic scale of a local structure
scale
bad
scale
Good !

12. Introduction Generates image features, “keypoints”
invariant to image scaling and rotation
partially invariant to change in illumination and 3D camera viewpoint
many can be extracted from typical images
highly distinctive 12

13. Outline
Introduction
Algorithm
Applications
Conclusions 13

14. ( ) Algorithm Scale-space extrema detection Keypoint localization
detector
Scale-space extrema detection
Keypoint localization
Orientation assignment
Keypoint descriptor
descriptor
( )
local descriptor 14

15. Outline Introduction Algorithm Scale-space extrema detection
Keypoint localization
Orientation assignment
Keypoint descriptor
Applications
Conclusions 15

16. Detection of Scale-space Extrema
Detect locations that are invariant to scale change of the image
SIFT uses DoG filter for scale space because it is efficient and as stable as scale-normalized Laplacian of Gaussian 16

17. DoG Filtering Convolution with a variable-scale Gaussian
Convolution with the DoG filter 17

18. Scale Space 18

19. Scale Space: an Example19

20. Outline Introduction Algorithm Scale-space extrema detection
Keypoint localization
Orientation assignment
Keypoint descriptor
Applications
Conclusions 20

21. Keypoint Localization
X is selected if it is larger or smaller than all 26 neighbors 21

22. Accurate Keypoint Localization
Reject the points that have low contrast (and are therefore sensitive to noise) or are poorly localized along and edge
233x189
original image
832 keypoints at extrema
729 keypoints after contrast filtering
536 keypoints after curvature filtering 22

23. Outline Introduction Algorithm Scale-space extrema detection
Keypoint localization
Orientation assignment
Keypoint descriptor
Applications
Conclusions 23

24. Orientation Assignment
By assigning a consistent orientation, the keypoint descriptor can be orientation invariant
Choose a region around
the keypoint
Calculate the magnitude of the gradient
Gaussian kernel
Orientation histogram

25. Orientation Assignment: an Example (1)25

26. Orientation Assignment: an Example (2)26

27. Orientation Assignment: an Example (3)27

28. Orientation Assignment: an Example (4)
approximately 25 degrees
80% of peak value 28

29. Outline Introduction Algorithm Scale-space extrema detection
Keypoint localization
Orientation assignment
Keypoint descriptor
Applications
Conclusions 29

30. Descriptor
It computes a descriptor in order to get highly distinctive description
The descriptors are as invariant as possible to remaining variations 30

31. Keypoint Descriptor (1)
The best result: 8 orientations x 4x4 histogram array = 128 dimensions 31

32. Keypoint Descriptor (2)
The 128D vectors are normalized to unit length in order to reduces effect of contrast change
Because of using gradients, the discriptor is invariant to the changes in illumination 32

33. Outline
Introduction
Algorithm
Applications
Conclusion 33

34. Keypoint Matching
The best candidate match for each keypoint is found by identifying its nearest neighbor in the database
Using the ratio of distance (closest/next closest) chooses the correct matches
reject the distance ratio that is greater than 0.8 34

35. Results

36. Object Recognition (1)
It creates a Hough transform entry predicting the model location, orientation, and scale
Solution for affine parameters: 36

37. Object Recognition (2)
The LS solution can be determined by solving the corresponding normal equations 37

38. Recognition examples (1)38

39. Recognition examples (2)39

40. Outline
Introduction
Algorithm
Applications
Conclusions 40

41. Conclusions The SIFT keypoints are useful due to their distinctiveness
This approach transforms an image into a large collection of local feature vectors
It is invariant to image translation, scaling, and rotation, and partially invariant to illumination changes and affine or 3D projection 41