SURF: Speeded Up Robust Features 授課教授: 連震杰 教授 Group number: 20 Advisor: Tzuu-Hseng S. Li Group members: E24956552 何雅芳 E24951099 蕭信揚 N26984224 李佳樺 aiRobots Laboratory, Department of Electrical Engineering, National Cheng Kung University, Tainan, Taiwan, R.O.C.

Outline Introduction Related Work Fast-Hessian Detector SURF Descriptor Orientation Assignment Descriptor Components Matching Experiments Conclusion

Outline Introduction Related Work Fast-Hessian Detector SURF Descriptor Orientation Assignment Descriptor Components Matching Experiments Conclusion

Introduction The task of finding point correspondences between two images of the same scene or object is part of many computer vision applications. This article presents a novel scale- and rotation-invariant detector and descriptor, coined SURF (Speeded-Up Robust Features). SURF approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster.

Introduction (cont’d) The search for discrete image point correspondences can be divided into three main steps. Step1. Detector Interest points are selected Most valuable property: Repeatability (whether it reliably finds the same interest points under different viewing condition.) Step2. Descriptor Extract the vector for matching Focus on scale and image rotation invariant. Step3. Match Often based on a distance between the vector

Outline Introduction Related Work Fast-Hessian Detector SURF Descriptor Orientation Assignment Descriptor Components Matching Experiments Conclusion

Related Work Interest Point Detection Harris corner detector ‧most widely used ‧based on the eigenvalues ‧not scale-invariant Automatic scale selection detector ‧experimented both the determinant of the Hessian matrix as well as Laplacian. Scale-invariant feature detectors (Mikolajczyk , Schmid) ‧Harris-Laplace and Hessian-Laplace ‧The location is selected by the determinant of Hessian matrix. ‧The scale is selected by the Laplacian. SIFT ‧Approximated the LoG by a DoG filter. => (1) Using the determinant of the Hessian matrix rather than its trace (the Laplacian) seems advantageous, (2) approximations like the DoG can bring speed at a low cost in terms of lost accuracy.

Related Work (cont’d) Interest Point Description SIFT ‧computes a histogram of local oriented gradients around the interest point and stores the bins in a 128-dimensional vector. PCA-SIFT ‧Yields a 36-dimensional descriptor (=>Fast) ‧To be less distinctive than SIFT GLOH ‧More distinctive with the same number of dimensions. ‧Computationally more expensive. => The SIFT descriptor still seems to be the most appealing descriptor for practical uses, and hence also the most widely used nowadays.

Related Work (cont’d) ★Question: Step1. Fast-Hessian detector Our approach Step1. Fast-Hessian detector Based on the Hessian matrix but use a very basic approximation – DoG Integral image: + Integral image (reduce the computation time) (x,y) (1)Fast implementation of box type convolution filters (2)Independent of its size A B C D B-D C-D Σ Property… Step2. SURF Descriptor Describes a distribution of Haar-wavelet Responses within the interest point neighborhood Answer… + Integral image (reduce the computation time) ★Question: Why can this method reduce the computation time? Step3. Match Present a new indexing step based on the sign of the Laplacian (Speed up & increase the robustness)

Outline Introduction Related Work Fast-Hessian Detector SURF Descriptor Orientation Assignment Descriptor Components Matching Experiments Conclusion

Fast-Hessian Detector Hessian matrix H( x , σ) in x at scale σ is defined as Approximation LoG with box filters => DoG Gaussian second order derivative x-dir y-dir xy-dir Box filters (instead of Gaussian) x-dir y-dir xy-dir 9x9 box filter with σ=1.2

Fast-Hessian Detector (cont’d) The scale space is analysed by up-scaling the filter size rather than iteratively reducing the image size. The scale space is divided into octaves. An octave represents a series of increasing filter response maps. scale 9 x 9 (σ=1.2) 15 x 15 (σ=2.0) 21 x 21 (σ=2.8) 27 x 27 (σ=3.6) Octave1 (increase:6) 15 x 15 (σ=2.0) 27 x 27 (σ=2.8) 39 x 39 (σ=5.2) 51 x 51 (σ=6.8) Octave2 (increase:12) It is selected as the interest point only if it is larger than all of these neighbors. For each new octave, the filter size increase is doubled. (going from 6 to 12 to 24…)

Q & A (Fast-Hessian Detector) Question1. 以放大filter的size代替將圖片縮小，有什麼好處? Answer1. 因為integral image的使用，使得計算量不會隨filter的size增加，且沒有將圖片縮小，圖片就不會失真。

Outline Introduction Related Work Fast-Hessian Detector SURF Descriptor Orientation Assignment Descriptor Components Matching Experiments Conclusion

SURF Descriptor Orientation Assignment Descriptor Components Interest Fixing a reproducible orientation based on information from a circular region around the interest point. Descriptor Components Construct a square region aligned to the selected orientation, and extract the SURF descriptor from it. Interest point Features

Outline Introduction Related Work Fast-Hessian Detector SURF Descriptor Orientation Assignment Descriptor Components Matching Experiments Conclusion

Orientation Assignment Haar Wavelet Orientation A B C D E F =A-B-D+E =B-C-E+F Σ=-A+2B-C+D-2E+F 4s a-b a b dx dy dx dy 6s Image (dx1,dy1) (dx2,dy2) π/3 Orientation Interest point

Outline Introduction Related Work Fast-Hessian Detector SURF Descriptor Orientation Assignment Descriptor Components Matching Experiments Conclusion

Descriptor Components Constructing a square region centered around the interest point, and oriented along the orientation. The region is split up regularly into smaller 4 × 4 square sub-regions. (4x4)x4=> a 64 dimensional vector Horizontal direction Vertical direction Haar wavelet (filter size 2s) 20s

Q&A(SURF Descriptor) Question1. Why to use Σ|dx| and Σ|dy| ? Answer1. Question2. Why to use Haar wavelet response? Answer2.

Outline Introduction Related Work Fast-Hessian Detector SURF Descriptor Orientation Assignment Descriptor Components Matching Experiments Conclusion

Matching For fast indexing during the matching stage, the sign of the Laplacian (i.e. the trace of the Hessian matrix) for the underlying interest point is included. In the matching stage, we only compare features if they have the same type of contrast.

Outline Introduction Related Work Fast-Hessian Detector SURF Descriptor Orientation Assignment Descriptor Components Matching Experiments Conclusion

Experiments Scale variant + rotation

Experiments (cont’d) Rotation

Experiments (cont’d) Blurred

Experiments (cont’d) Photometric deformations

Outline Introduction Related Work Fast-Hessian Detector SURF Descriptor Orientation Assignment Descriptor Components Matching Experiments Conclusion

Conclusion SURF outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster. Future work will aim at optimizing the code for additional speed up.

★ Thanks for your attention!!