Distinctive Image Features from Scale- Invariant Keypoints Mohammad-Amin Ahantab Technische Universität München, Germany

2 Motivation Object recognition Motion tracking 3d scene recognition Stereo correspondence Panorama stitching…

Which requirements should image features meet? Invariance to scale Invariance to rotation Invariance to affine transformation/view point Invariance to illumination High distinctiveness 3

Scale Invariant Feature Transform (SIFT) -Transforms image data to scale-invariant coordinates -Creates large number of features -A 500x500 pixel pictures results (roughly) in 2000 image features 4

Scale Invariant Feature Transform (SIFT) 1.Scale-space extrema detection 2.Keypoint localization 3.Orientation assignment 4.Keypoint descriptor 5

Scale-space 6

How to blurr the image ? Gaussian function: 7

Next step : Calculate the Difference of Gaussian Images 8

Example 9

Why is the Diffrence of Gaussian computed? 10

Why is the Diffrence of Gaussian computed? 11

Local Maximas/Minimas in DoG images Locating maxima and minimas by comparing the maked pixel with all its neighbour pixels in the current image, the scale above and the scale below it. If the pixel is smaller or bigger than all of its 26 neighbours it will be selected. Problem: only an approximation 12

2.(Accurate) Keypoint localization 3D Ultrasound Mosaicing - Wachinger et al.13

Taylor expansion.14

Taylor expansion 15

Eliminate keypoints with low contrast ! 16

Remove Edges! Keypoints along edges are not stable ! How edges are found: Large principal curvature across the edge, small one in perpendicular direction The principal curvature can be computed with the Hessian matrix of the DoG function because the eigenvalues of H are propotional to the curvatures! 17

Calculation of principal curvatures.18

3.Orientation assignment Compute magnitude and orientation of gradients around the keypoint on smoothed images L (x,y) Create A histogram with 36 bins (each 10 degrees ) and quantize with respect to magnitude 19

20 Take the orientation with the highest peak and assign it to keypoint ! If there is a peak with at least 80% of the highest peak create a new keypoint with a diffrent orientation!

21

4.Keypoint descriptor 22

4.Keypoint descriptor 23 Image gradient magnitudes and orientations are sampled around keypoint in the region aound the keypoint A 16 x 16 sample array is used to compute a 4 x 4 descriptor position and orientation of each sample is rotated relatively to key point orientation (rotation invariance) In each field a histogram with 8 bins is created -> results in a 128 dim- vector Length of each arrow is computed with respect to the magnitude of the samples (Gaussian weight function )

Illumination Invariance Changes in contrast : normalize the vector Changes in brightness : does not affect the gradients Non-linear illumination changes: limit the values in the vector to not larger than 0.2 and renormalize to unit length 24

Key point matching matching keypoints by finding the nearest neighbour in a database of keypoints Nearest neighbour is the key point with the minimum Euclidean distance If the ratio between best and second best neighbour is bigger than 0.8 keypoints are not matched !.25

Testig Sift for Image matching 26

27

28

29

30

31

32

33

34

35

36

References Lowe, D. “Distinctive image features from scale-invariant keypoints” introduction/ 37