1. Hierarchical Matching with Side Information for Image Classification
CVPR 2012
Hierarchical Matching with Side Information for Image Classification
Authors Qiang Chen et al.
Presenter Hyung-seok Lee (이형석)

2. Motivation (1/2) Bag-of-Words (BoW)
Frequency histogram of visual words
Not emphasize any elements with regard to image layout

3. Motivation (2/2) Spatial Pyramid Matching (SPM)
Partitioning the image plane into fine sub-regions
Not optimum for object-centered recognition problem
Beyond Bags of Features : Spatial Pyramid Matching for Recognizing Natural Scene Categories, Svetlana Lazebnik et al.

4. Main Idea Generalized Hierarchical Matching (GHM)
Extends the popular matching work
Integrate prior knowledge for enhancing feature matching
side information

5. Overview Image Local features & side information
Hierarchical clustering
Hierarchical structure
representation
Matching over each cluster

6. Side Information (1/4) Object Confidence Map
Object position should be extremely beneficial
Shape model / Appearance model

7. Side Information (2/4) Shape-based object detection : HOG features
Appearance-based object detection : BoW features
Object Confidence Map

8. Side Information (3/4) Visual Saliency Map
Aim to detect the important content in images
Explain human visual search strategies
SUN : A Bayesian Framework for Saliency Using Natural Statistics, Lingyun Zhang et el.

9. Side Information (4/4)
Use the saliency model SUN (Saliency Using Natural statistics)
Based on natural image statistics

10. Side Information Combination
Combine object confidence map & visual saliency map
Design a 3 level hierarchical structure
- Object confidence map is used in level 2
- Background area will be further utilized in level 3

11. Encoding & Matching (1/2)

12. Encoding & Matching (2/2)
Encoding is operated on each cluster
Matching over each corresponding cluster

13. Experiments (1/2)
Caltech-UCSD Birds & Oxford Flowers 17 and 102

14. Experiments (2/2)
VOC 2007
VOC 2010

15. Conclusion Propose the Generalized Hierarchical Matching framework
- Extends the popular pyramid matching
Two novel kinds of side information are introduced to enhance object-oriented image classification tasks
Object confidence map
Visual saliency amp