1. Descriptors (Description of Interest Regions with Local Binary Patterns)
Yu-Lin Cheng
(03/07/2011)

2. Outline Scale Invariant Feature Transform (SIFT) Descriptor
Local Binary Pattern (LBP) Descriptor
Center-Symmetric LBP (CS-LBP) Descriptor
Histogram of Oriented Gradients (HOG) Descriptor

3. SIFT(Scale Invariant Feature Transform )
SIFT Algorithm:
descriptor

4. SIFT(Scale Invariant Feature Transform )
Scale-space Extrema Detection:
Stable feature points (scale invariant)
Principle:
A local maximum over scales by using combination of normalized derivatives can be treated as a characteristic point of local structure
Use LoG to find maximum
scale
bad
scale
Good !

5. SIFT(Scale Invariant Feature Transform )
Scale-space Extrema Detection:
Use DoG instead of LoG ---- (computational efficiency)

6. SIFT(Scale Invariant Feature Transform )
Scale-space Extrema Detection:

7. SIFT(Scale Invariant Feature Transform )
Scale-space Extrema Detection:
Local extrema detection:
Compare to 26 neighbors
Keep the same keypoint in all scale

8. SIFT(Scale Invariant Feature Transform )
Scale-space Extrema Detection:
Reject points with low contrast

9. SIFT(Scale Invariant Feature Transform )
Accurate keypoints localization:
Quadratic function to interpolate the location of maximum
Eliminate edge response:
r: threshold, H: Hessian matrix

10. SIFT(Scale Invariant Feature Transform )
Orientation Assignment:
Assign a consistent orientation to achieve orientation invariant
Method:

11. SIFT(Scale Invariant Feature Transform )
Orientation Assignment:
Calculate gradient magnitude and direction of neighboring pixels

12. SIFT(Scale Invariant Feature Transform )
Orientation Assignment:
Calculate weighted orientation histogram

13. SIFT(Scale Invariant Feature Transform )
Orientation Assignment:
Calculate weighted orientation histogram

14. SIFT(Scale Invariant Feature Transform )
Orientation Assignment:
Calculate weighted orientation histogram

15. SIFT(Scale Invariant Feature Transform )
Keypoints Descriptor:
Empirical result:
Cell size: 4×4 pixels
Block size: 4×4 cells
Dimension: 4×4 (cells) × 8 (bins) = 128
Weighted magnitude

16. SIFT(Scale Invariant Feature Transform )
Keypoints Descriptor:
Avoid all boundary effect
Use trilinear interpolation
Normalization: (illumination invariant)
Normalize to unit length
Threshlod the maximum value to 0.2
Match the magnitudes for large gradients is no longer important
Renormalize to unit length

17. LBP(Local Binary Pattern)
A powerful mean of texture description
LBP operator:
Standard LBP:
Illustration:

18. LBP(Local Binary Pattern)
Example:
Parameters:
P : Number of neighboring pixels
R : Radius

19. LTP(Local Trinary Pattern)
LTP operator:
t : threshold
Illustration:

20. CS-LBP(Center-Symmetric Local Binary Pattern)
CS-LBP operator:
Illustration:

21. CS-LBP Descriptor
Flow diagram:

22. CS-LBP Descriptor Interest Region Detection: Detectors: 41×41
1. Hessian-Affine (blob-like structure)
2. Harris-Affine (corner-like structure)
3. Hessian-Laplace (scale-invariant version)
4. Harris-Laplace (scale-invariant version)
41×41

23. CS-LBP Descriptor Feature Extraction: CS-LBP operator:
Parameters:
R: radius
R = 1, 2
N: number of neighboring pixels
N = 6, 8
T: threshold
T = 0.2
Descriptor Construction:
Location grids
3×3 cells/4×4 cells
Avoid boundary effects:
Using ‘bilinear interpolation’
41×41

24. CS-LBP Descriptor Descriptor Normalization: (illumination invariant)
Normalize to unit length
Thresholding
Renormalize to unit length
2 4 × 4×4 =256

25. Comparison(SIFT v.s. CS-LBP)
Assumption:
Computations cannot be reused from detection algorithm
Comparison:
Conclusion:
Computational efficiency and better performance than SIFT

26. HOG(Histogram of Oriented Gradients)
Idea: local object appearance and shape can be rather well characterized by the distribution of local intensity gradients or edge direction

27. HOG(Histogram of Oriented Gradients)
Gradient Computation:

28. HOG(Histogram of Oriented Gradients)
Gradient Computation:

29. HOG(Histogram of Oriented Gradients)
Spatial/Orientation Binning:
Weighted votes
Function of magnitude
Avoid aliasing
Interpolation
Parameters:
Number of orientation bins
Cell size
Block size
Cell
Block

30. HOG(Histogram of Oriented Gradients)
Spatial/Orientation Binning:
Parameters:
Number of orientation bins: 9 bins/18bins
Cell size: 8×8 pixels
Block size: 2×2 cells

31. HOG(Histogram of Oriented Gradients)
Normalization:
Group cells to larger blocks and normalize each block separately (illumination invariant)
Normalization Schemes:

32. HOG(Histogram of Oriented Gradients)
Normalization:
Normalization Schemes:

33. Comparison(SIFT v.s. HOG)

34. HOG Variation Pixel-Level Feature Maps: ,(p = 9)
‘Object Detection with Discriminatively Trained Part Based Models’
Pixel-Level Feature Maps:
Use [-1, 0, 1] to calculate gradient
Contrast sensitive(B1), Contrast insensitive(B2)
,(p = 9)
Quantize into orientation bins
r: gradient magnitude

35. HOG Variation Spatial Aggregation: Normalization:
Rectangular cell: 8×8 pixels
Cell-based feature map:
Reduce the size of feature map
Avoid aliasing:
Bilinear interpolation
Normalization:

36. HOG Variation Truncation: maximum 0.2 Dimension: No renormalization
9 bins × 4 different normalization = 36 (contrast insensitive)

37. HOG Variation PCA analysis:
Top 11 eigenvectors captures most of information of HOG

38. HOG Variation PCA analysis:
Top eigenvectors lie (approximately) in a linear subspace
13-dimensional features:
Project 36-dimensional HOG feature into uk, vk
Projection into uk : sum over 4 normalization over fixed orientation
Projection into vk : sum over 9 orientation over fixed normalization

39. HOG Variation For Contrast Insensitive(B2):
9 bins × 4 different normalization = 36 (contrast insensitive)
For Contrast Sensitive(B1):
18 bins × 4 different normalization = 72 (contrast insensitive)
Reduce to (18 + 9) + 4 = 31 dimension

40. Reference
“Description of Interest Regions With Local Binary Patterns”, Pattern Regonization ’09 Marko Heikkilä
“Effective Pedestrian Detection Using Center-symmetric Local Binary/Trinary Patterns”, Youngbin Zheng
“Scale-space Theory” Tony Lindeberg
“Histogram of Oriented Gradients for Human Detection”, CVPR ‘05 Navneet Dalal
“Finding People in Images and Videos”, Navneet Dalal
“Feature matching” Yung-Yu Chuang
“Scale & Affine Invariant Interest Point Detectors”, IJCV ’04 Krystian Mikolajczyk

41. Reference
“Object Detection with Discriminatively Trained Part Based Models”
“Distinctive Image Features from Scale-Invariant Keypoints”, IJCV ’04 David G. Lowe