1. DIPTA 陳昱廷 r02944014@csie.ntu.edu.tw Feature Detector II

2. Agenda FAST - E. Rosten. Machine Learning for High-speed Corner Detection, 2006. ORB - Ethan Rublee, Vincent Rabaud, Kurt Konolige, Gary R. Bradski: ORB: An efficient alternative to SIFT or SURF. ICCV 2011: 2564-2571. BRISK - S. Leutenegger, M. Chli, and R. Siegwart, "BRISK: Binary Robust Invariant Scalable Keypoints," in IEEE International Conference on Computer Vision (ICCV), 2011

3. FAST Features from Accelerated Segment Test An improvement of Rosten’s previous work[1][2] The property of this detector is like it’s name -- FAST There is not any descriptor on this paper [1] Rosten, E., Drummond, T.: F using points and lines for high performance tracking. In: 10th IEEE International Conference on Computer Vision. Volume [2] Beijing, China, Springer (2005) 1508-1515 Rosten, E., Reitmayr, G., Drummond, T.: Real-time video annotations for augmented reality. In: International Symposium on Visual Computing. (2005)

4. FAST - Previous Work A circle of 16 pixels around the coner candidate p p is classified as a coner if there exists more than N continuous pixels in the circle which all brighter or darker than Ip + t or Ip-t (t is a threshold) N was chosen as 12, so we exam only 1, 5, 9 and 13 to exclude very large numbers of non-corners

5. FAST - Previous Work Weakness: 1. The high-speed test does not generalize well for n < 12. 2. The choice and ordering of the fast test pixels contains implicit assumptions about the distribution of feature appearance. 3. Knowledge from the first 4 tests is discarded. 4. Multiple features are detected adjacent to one another.

6. FAST - Improvment Use decision tree to address the first three points. Each node of decision tree represents what location on the circle to compare with current point. The process of every node has 2 stages 1. Partition all pixels of current set into 3 subsets by each location on the circle. 2. Calculate entropy, which implies impurity, to choose suitable location on the circle.

7. FAST - Improvment

8. Stage 2 employs the algorithm used in ID3[1] and begins by selecting the x which yields the most information about whether the candidate pixel is a corner, measured by the entropy H(P) of Kp. [1] Quinlan, J.R.: Induction of decision trees. Machine Learning 1 (1986) 81-106

9. FAST - Improvment The choice of x then yields the information gain: The process is applied recursively on all three subsets (i.e. x b is selected to partition P b into P b,d, P b,s, P b,b ) until the entropy of the subset is zero. This means that all p in this subset have same value of Kp (all corner or all non-corner) This creates a decision tree which can correctly classify all corners seen in the training set and therefore (to a close approximation) correctly embodies the rules of the chosen FAST corner detector.

10. FAST - Improvment Use non-maximal suppression to address the last point Non-maximal suppression - find the feature of maximum score in a circle of radius r - iterative r to find enough number of feature Score function V:

11. ORB Oriented FAST and Rotated BRIEF[1] Use oFAST for keypoint detector and rBRIEF for decriptor The properties of this detector are rotation invariant and resistant to noise [1] M. Calonder, V. Lepetit, C. Strecha, and P. Fua. Brief: Bi­nary robust independent elementary features. In In European Conference on Computer Vision, 2010.

12. oFAST - FAST Use FAST-9 (circular radius of 9) Employ a Harris corner measure to order FAST keypoint

13. oFAST - orientation Orientation by Intensity Centroid atan2 is the quadrant-aware version of arctan P. L. Rosin. Measuring corner properties. Computer Vision and Image Understanding, 73(2):291 - 307, 1999.

14. rBRIEF - BRIEF The BRIEF descriptor is a bit string description of an image patch constructed from a set of binary intensity tests. Consider a smoothed image patch p. A binary test T is defined by where p(x) is the intensity of p at a point x. The feature is defined as a vector of n binary tests (here n=256) Use Gaussian distribution around the center of the patch to randomly select test point pair (x, y)

15. rBRIEF – steered BRIEF

16. rBRIEF – Variance and Correlation High variance makes a feature more discriminative. Origin BRIEF has a property of a large variance but steered BRIEF lose it since the oriented corner keypoints present a more uniform appearance to binary tests Another desirable prop­erty is to have the tests uncorrelated, since then each test will contribute to the result, but steered BRIEF doesn’t has this

17. rBRIEF – learning good binary features Recover from the lose of variance and reduce correlation among the binary test Learning method for choosing good subset of binary tests, the algorithm is

18. BRISK Binary Robust Invariant Scalable Keypoints Scale-space FAST-based detector in combination with the assembly of a bit-string descriptor from intensity comparisons retrieved by dedicated sampling of each keypoint neighborhood Adaptive, high quality performance as in state-of-the-art algorithm, albeit at a dramaticlly lower computational cost The properties of this detector are rotation invariant and scale invariant

19. BRISK Create scale space. Compute FAST score across scale space. Pixel level non-maximal suppression. Compute sub-pixel maximum across patch. Compute continuous maximum across scales. Re-interpolate image coordinates from scale space feature point detection. www.cse.unr.edu/~bebis/CS491Y/Le ctures/BRISK.pptx

20. Create scale space

21. FAST feature detector FAST 9-16, requires at least 9 consecutive pixels in the 16- pixel circle to detect corner Special case In order to obtain FAST score of intra-octave, apply the FAST 5-8 mask on c 0

22. Non-maximal suppression

23. Subpixel maxima and scale selection Using points obtained by non- maximal suppression. 2D quadratic function is fit to the 3x3 patch surrounding the pixel and sub-pixel maximum is determined. The same is done for the layer above and below. These maxima are then interpolated using a 1D quadratic function across scale space and the local maximum is chosen as the scale for the feature is found.

24. Keypoint Description Sample pattern of smoothed pixels around feature. Separate pairs of pixels into two subsets, short-distance pairs and long-distance pairs. Compute local gradient between long-distance pairs. Sum gradients to determine feature orientation. Rotate short-distance pairs using orientation. Construct binary descriptor from rotated short-distance pairs.

25. Sampling pattern [1] E. T ola, V. Lepetit, and P. Fua. Daisy: an Efficient Dense Descriptor Applied to Wide Baseline Stereo. IEEE Transactions on P attern Analysis and Machine Intelligence (P AMI), 32(5):815– 830, 2010

26. Pattern pairs

27. Local gradient/keypoint orientation Strength of gradient between pairs is computed using Overall keypoint direction vector g is estimated by summing gradients of all pairs in long-distance set.

28. Building descriptor

29. Descriptor Matching Hamming distance computed for all pairs between images. Count matching bits between descriptors. Efficient computation because this is simply an XOR operation between two 512 bit strings. Threshold for matching is the number of bits able to be matched.