Bag of Features Approach: recent work, using geometric information.
Published byModified over 4 years ago
Presentation on theme: "Bag of Features Approach: recent work, using geometric information."— Presentation transcript:
Bag of Features Approach: recent work, using geometric information
Problem Search for object occurrences in very large image collection
2 sub problems Object Category Recognition and Specific Object Recognition
Motivation Look for product information Look for similar products
Related work on large scale image search Most systems build upon the BoF framework [Sivic & Zisserman 03] – Large (hierarchical) vocabularies [Nister Stewenius 06] – Improved descriptor representation [Jégou et al 08, Philbin et al 08] – Geometry used in index [Jégou et al 08, Perdoc’h et al 09] – Query expansion [Chum et al 07] – … Efficiency improved by: – Min-hash and Geometrical min-hash [Chum et al. 07-09] – Compressing the BoF representation [Jégou et al. 09]
Use geometry Possible directions: – Change/optimize spatial verification stage – Insert a new geometric information to the index Ordered BOF Bundled features Visual phrases – Change the searching algorithm
Survey for today Spatial Bag-of-features [Cao, CVPR2010] Image Retrieval with Geometry-Preserving Visual Phrases [Zhang Jia Chen, CVPR2011] Smooth Object Retrieval using a Bag of Boundaries [Arandjelovi Zisserman, ICCV2011]
Spatial BOF Pros: – Gets better performance than BOF+RANSAC for large scale dataset* – Same format as standard BOF Cons: – Is dataset dependent because of need of training Do not present the results for large scale dataset with transfer learning from another dataset Future work – Check it with cross training for large dataset. Otherwise, it is not worth working further.
BOF for smooth objects Segmentation phase Over segmentation with super-pixels Classification of super-pixels: 3208 feature vector (median(Mag(Grad)), 4 bits, color histogram, BOF) SVM Post-processing
BOF for smooth objects Boundary description phase: Sample points on the boundary Calculate HoG at each point in 3 scales 340 dimensional L2 normalized vector * The descriptor is not rotation invariant
BOF for smooth objects Retrieval procedure: Boundary descripors are quantized (k=10k) Standard BOF scheme* Spatial verification for top 200 with loose affine homography (errors up to 100pixs) * No spatial information is recorded in the histogram
BOF for smooth objects Pros: – Solves the smooth object retrieval problem – Fast Cons: – Is dataset dependent because of need of training – Limited to objects with “solid” materials – segmentation has to catch the object’s boundary Future work – Eliminate the training step
Summary There is an active research in the field of CBIR to exploit geometry information. Each method with its limitations Still no widely accepted solution – Like spatial verification with RANSAC