1. Object Recognition on Mobile Platforms Using Mixture of Global Image Features
Cheng-Ming Huang, Wen-Hung Liao
Department of Computer Science
National Chengchi University, Taipei, TAIWAN
August 30, 2012

2. Object Recognition ‘Scene’ Recognition?

3. Outline Objective and system architecture Global vs. local descriptors
Proposed global image descriptors
Weighted gist feature
Average Effective Number of Neighbors
Experimental results
Taiwan landmark
Oxford dataset
Conclusion

4. Objective
Design a computationally efficient framework for automatic scene recognition on mobile platforms
Constraints: limited resources (CPU, communication bandwidth, storage)

5. System architecture Network Back-end Front-end Identification
Image
Feature Extraction
Feature Compression
Front-end
Query Data
Matching
Post-processing
Feature Database
Image Database
Network
Back-end

6. Global vs. local descriptors
Global features:
provide a succinct description of the scene structure
faster to compute
Examples: histogram, gist.
Local features:
Characterize distinct points/parts in an image
Robust features require substantial computation resources
Examples: SIFT, SURF or HoG

7. The proposed global features
Weighted gist descriptor: based on the ‘gist’ descriptor, but weighted by saliency measure of the image region.
Average effective number of neighbors (AENN): try to capture the overall structure of the scene based on the distribution of edge pixels.

8. The original gist feature
Computed by convolving an oriented filter with the image at several different orientations and scales.
The scores for the filter convolution at each orientation and scale are stored in an array, which is the gist feature for that image.

9. Illustration
4x4 blocks, 6 orientations and 5 frequencies (Oliva and Torralba, 2001)

10. Saliency Map
Graph-based visual saliency, J. Harel et al

11. Weighted gist descriptor

12. Effective Number of Neighbors (ENN)
(L):8, (R):8
Effective number of neighbors:
(L):8
(R):4/2+4/4=3
5x5 window

13. Computing Average ENN Step 1: Edge detection
Step 2: Keep top q percent of the edge pixels
Step 3: Compute ENN using a DxD neighborhood
Step 4: a Average the ENN in an image block to form the feature vector

14. Average ENN descriptor: some examples

15. Parameter Settings (I)
For weighted gist
partition the image into 4x4 blocks.
use 8 orientation channels at two different frequencies and 4 orientation channels at another frequency, totaling 20 coefficients for each block
feature dimension=4x4x20=320 for each channel
For color images: 960 dimension

16. Parameter Settings (II)
For AENN
Keep top 10% strong edges
Partition into 8x8 image blocks
Neighborhood size for computing ENN: 7x7

17. Classifier
Support vector machine (SVM) with probability estimate output.
Information fusion using linear combination

18. Experiment I: Taiwan Landmarks
Data collection
- A total of 9530 images from 50 landmarks have
been gathered. On average, each landmark
contains 190 images.
- Test dataset
- Randomly select 20 images from each category
(20x50=1000)
- Training dataset
- The remaining 8530 images

19. Experiment I: Results

20. Comparison of individual and hybrid approaches

21. Experiment II: Oxford buildings dataset
Publicly available dataset with ground truth
Contains 5062 high resolution images
11 different categories
Image quality:
- Good
- OK
- Bad
- Junk

22. Recognition rate using Oxford dataset
Retain only Good and OK images
Query data: Randomly select 3 images from each category
Training data
- The remaining as training data

23. Other approaches using Oxford dataset(1/3)
Local configuration of SIFT-like features by a shape context
- Image feature: PCA-SIFT local feature
- Utilize shape context to describe local configuration
The parameters are: scale of shape context in pixel, number of segmentation in scale/angle

24. Other approaches using Oxford dataset (2/3)
Improving bag-of-features for large scale image search
- Using Hessian-Affine detector and SIFT
- Employ bag-of-feature (BOF) framework to search for
approximate nearest neighbor
- Methods tested include original BOF, Hamming
embedding(HE), weak geometric consistency
constraints(WGC) and multiple assignment(MA)

25. Other approaches using Oxford dataset(3/3)

26. Conclusion
Proposed a framework for scene recognition on mobile platforms
Formulated two global image descriptors for recognition tasks
Experiment results and comparative analysis have demonstrated the efficacy of our proposed strategy.

27. Thank you
Q & A