1. P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University, Taiwan ICIP 2010

2. Introduction Approach – Locality-Sensitive Hashing (LSH) – Sketching the Feature Space Experiments – Fast Matching Conclusion

3. INTRODUCTION Local feature have been extensively used to represent image for various problem Lots of local feature detector and local feature descriptor have been proposed recent years

4. Recent History Maximally Stable Extremal Regions (MSER) [1] BMVC 2002 Difference-of-Gaussian and Scale-Invariant-Feature-Transform (SIFT) [2] IJCV 2004 Affine invariant detector [3], [4] IJCV 2004, TPAMI 2005 Histogram of oriented gradients (HOG) [5] CVPR 2005 ‘Visual words’ [6] ‘codebooks’ [7] ICCV 2003, BMVC 2003 For example

5. Present an empirical analysis of the feature space of interest points detected in natural image Perform an approximate method for the fast matching between two sets of interest points detected in two images Show that the complexity of matching M points to N points can be reduced from O(MN) to O(M+N) INTRODUCTION

6. Locality-Sensitive Hashing p-stable Distribution:

7. Locality-Sensitive Hashing based on 2-Stable Distribution

8. Hash Family a : random vector sampled from a Gaussian distribution b : real value chosen uniformly from the range [0, r] r : line width The dot-product a ‧ v projects each vector to the real line

9. Building Hash table

10. Choose the width r based on the minimum and maximum =？=？ θ a ‧ b = |a| |b| Index function t = 5, K=3 [5] [5] [5] = 125 = (5-1) * 5 2 + (5-1) * 5 1 + 4 * 5 0 + 1 = 4 * 25 + 20 + 4 + 1 = 125

11. Sketching the Feature Space Berkeley segmentation database [14] Use difference of Gaussian (DOG) [2] & Hessian-affine [3] detector detect about 200,000 interest points Extract image patches by SIFT descriptor [2] Create a hash table (L = 1) with five projection(K = 5) and 15 segments on each dot-product real line (t = 15) The total number of buckets is 15 5 = 759,375

12. Entropy = 4.2251 (a) DOG Entropy = 4.0622 (b) Hessian-affine Sketching the Feature Space Distribution and Entropy

13. Collect three image patches of different size 16x16, 32x32, 64x64 Each set consist of 200,000 patches. Natural image patches (from Berkeley segmentation database ) Noise image patches (Randomly-generated noise patches) Sketching the Feature Space

14. Distribution and Entropy

15. Fast Matching 3 3 33 33 3 3 Reference image Remaining Image (test)

16. Fast Matching We create L = 16 hash tables to probe the 128-dimensional SIFT- feature space Each table is equipped with five 2-stable Projections, and the projected values are quantized into 15 segments, i.e., K = 5 and t = 15 For LSH, we use two threshold values of dot-product, θ = 0.95 and θ = 0.97, to determine whether a pair of feature vectors in the same bucket yields a match LSH is 2 to 15 times faster than matching by exhaustive search a ‧ b = |a| |b| If a = b, then = 1

17. Fast Matching DoG detector + SIFT descriptorHessian-affine detector + SIFT descriptor

18. DoG detector + SIFT descriptor 2-stable LSH matching vs. exhaustive matching

19. Hessian-affine detector + SIFT descriptor

20. Conclusion Using the approximate nearest-neighbor probing scheme derived from 2-stable Locality-Sensitive Hashing Make use of the efficient representation of the SIFT feature space, and present a fast feature-matching method for finding correspondences between two sets of interest points. And,

21. THANK YOU SO MUCH