1. A NOVEL LOCAL FEATURE DESCRIPTOR FOR IMAGE MATCHING Heng Yang, Qing Wang ICME 2008

2. Outline Introduction Local feature descriptor Feature matching Experimental result and discussions Image matching experiments Image retrieval experiments Conclusion

3. Local feature descriptor Local invariant features have been widely used in image matching and other computer vision applications Invariant to Image rotation, scale, illumination changes and even affine distortion Distinctive, robust to partial occlusion, resistant to nearby clutter and noise Two concerns for extracting the local features Detect keypoints Assigning the localization, scale and dominant orientation for each keypoint Compute a descriptor for the detected regions To be highly distinctive As invariant as possible over transformations

4. Local feature descriptor At present, the SIFT descriptor is generally considered as the most appealing descriptor for practical uses Based on the image gradients in each interest point’s local region Drawback of SIFT in matching step High dimensionality (128-D) SIFT extensions PCA-SIFT descriptor Only change SIFT descriptor step Pre-compute an eigen-space for local gradient patches of size 41x41 2x39x39=3042 elements Only keep 20 components A more compact descriptor

5. Local feature descriptor GLOH (Gradient location-orientation histogram) Divides local circular region into 17 location bins gradient orientations are quantized in 16 bins Analyze the 17x16=272-d Eigen-space (PCA) keep 128 components Computationally more expensive and need extra offline computation of patch eigen space This paper presents a local feature descriptor (GDOH) Based on the gradient distance and orientation histogram Reduce the dimensional size of the descriptor Maintain distinctness and robustness as much as SIFT

6. Local feature descriptor First Image gradient magnitudes and orientations are sampled around the keypoint location Assign a weight to the magnitude of each point Gaussian weighting function with equal to half the width of the sample region is employed Reduce the emphasis on the points that are far from the center Second The gradient orientations are rotated relative to the keypoint dominant direction Achieve rotation invariance The distance of each gradient point to the descriptor center is calculated Final Build the histogram based on the gradient distance and orientation 8(distance bins) × 8(orientation bins) = 64 bins

7. Feature matching Given keypoint descriptors extracted from a pair of two images Find a set of candidate feature matches Using Best-Bin-First (BBF) algorithm Approximate nearest-neighbor searching method in high dimensional spaces Only consider the matches in which the distance ratio of nearest neighbor to the second-nearest neighbor is less than a threshold Correct matches should have the closest neighbor significantly closer than the closest incorrect match

8. Feature matching Find Nearest neighbor feature points A variant of the k-d tree search algorithm makes indexing in higher dimensional spaces practical. Best Bin First Approximate algorithm Finds the nearest neighbor for a large fraction of the queries A very close neighbor in the remaining cases Standard version of the K-D tree Beginning with a complete set of N points in R k Data space is split on the dimension i which the data exhibits the greatest variance A cut is made at the median value m of the data in that dimension equal number of points fall to one side or the other An internal node is created to store i and m Process iterates with both halves of the data This creates a balanced binary tree with depth d = log 2 N

9. NN search using K-D tree a b c d e f g c e b d g a f Nearest = ? dist-sqd =  NN(c, x) Nearer = e Further = b NN (e, x) nearer further

10. NN search using K-D tree a b c d e f g c e b d g a f Nearest = ? dist-sqd =  NN(e, x) Nearer = g Further = d NN (g, x) nearer further

11. NN search using K-D tree a b c d e f g c e b d g a f Nearest = ? dist-sqd =  NN(g, x) Nearest = g dist-sqd = r r

12. NN search using K-D tree a b c d e f g c e b d g a f Nearest = g dist-sqd = r NN(e, x) Check d2(e,x) > r No need to update r

13. NN search using K-D tree a b c d e f g c e b d g a f Nearest = g dist-sqd = r NN(e, x) Check further of e: find p d (p,x) > r No need to update r p

14. NN search using K-D tree a b c d e f g c e b d g a f Nearest = g dist-sqd = r NN(c, x) Check d2(c,x) > r No need to update r

15. NN search using K-D tree a b c d e f g c e b d g a f Nearest = g dist-sqd = r NN(c, x) Check further of c: find p d(p,x) < r !! NN (b,x) r p

16. NN search using K-D tree a b c d e f g c e b d g a f Nearest = g dist-sqd = r NN(b, x) Nearer = f Further = g NN (f,x) r

17. NN search using K-D tree a b c d e f g c e b d g a f Nearest = g dist-sqd = r NN(f, x) r’ = d2 (f,x) < r dist-sqd  r’ nearest  f r’

18. NN search using K-D tree a b c d e f g c e b d g a f Nearest = f dist-sqd = r’ NN(b, x) Check d(b,x) < r’ No need to update r’

19. NN search using K-D tree a b c d e f g c e b d g a f Nearest = f dist-sqd = r’ NN(b, x) Check further of b; find p d(p,x) > r’ No need to update r’ p

20. NN search using K-D tree a b c d e f g c e b d g a f Nearest = f dist-sqd = r’ NN(c, x) r’

21. Search Process: BBF Algorithm Set: v: query vector Q: priority queue ordered by distance to v (initially void) r: initially is the root of T vFIRST: initially not defined and with an infinite distance to v ncomp: number of comparisons, initially zero. While (!finish): Make a search for v in T from r => arrive to a leaf c Add all the directions not taken during the search to Q in an ordered way (each division node in the path gives one not-taken direction) If c is more near to v than vFIRST, then vFIRST=c Make r = the first node in Q (the more near to v), ncomp++ If distance(r,v) > distance(vFIRST,v), finish=1 If ncomp > ncompMAX, finish=1

22. BBF search example  1 >2  2 >3 [1,3]  2 >7  1 >6 Requested vector  1 >2 18 [2,7] [5,1500][9,1000] [20,7] [20,8] 18 20>2 Go right Queue: 1 8>7 Go right  2 >7 1  1 >2 18  2 >7 1  1 >2 18  1 >6 14 20>6 Go right

23.  1 >2  2 >3 [1,3]  2 >7  1 >6 Requested vector [2,7] [5,1500][9,1000] [20,7] [20,8] Queue: 992 Distance from best-in-queue is lesser than distance from C MIN Start new search from best in queue Delete best node in queue C MIN : [9,1000] 992  2 >7 1  1 >2 18  1 >6 14 BBF search example  1 >2 18  1 >6 14 [20,7] 1 We arrived to a leaf Store nearest leaf in C MIN Distance from best-in- queue is NOT lesser than distance from c MIN Finish 14

24. Experimental result Compare the performance of SIFT and GDOH by image matching experiments and an image retrieval application Dataset for image matching experiments contains test images of various transformation types Dataset for image retrieval experiment includes 30 images of 10 household items

25. Image matching experiments Target images are rotated by 55 degree and scaled by 1.6 Target images are rotated by 65 degree and scaled by 4

26. Image matching experiments Target images are distorted to simulate a 12 degree viewpoint change Intensity of target images is reduced 20%

27. Image matching experiments GDOH outperforms SIFT slightly GDOH can performs comparatively with SIFT over various transformation types of images Table lists the comparison result of average matching time of SIFT and GDOH, respectively GDOH is significantly faster than SIFT in the image matching stage GDOH requires about 63% of the time of SIFT to do 65 pairs of image matching

28. Image retrieval experiments We first extract the descriptors of each image in the image dataset Then we find matches between every pair of images Matches if the distance ratio of the nearest neighbor to the second- nearest neighbor is less than a threshold Similarity measure Number of matched feature vector as a similarity measure between images For each image, the top 2 images with most matched number are returned

29. Conslusion GDOH Is created based on the gradient distance and orientation histogram Can be invariant to image rotation, scale, illumination and partial viewpoint changes Distinctive and robust as SIFT descriptor. The dimensionality of GDOH is much lower than that of SIFT, which can result in high efficiency in image matching and image retrieval application.