Presentation on theme: "Universidad Católica San Pablo Cristina Patricia Cáceres Jáuregui"— Presentation transcript:
Universidad Católica San Pablo Cristina Patricia Cáceres Jáuregui
Motivation Fast image search is a useful component for a number of vision problems. Plenty of nuisance parameters (lighting, pose, background clutter, etc.)
Outline Scalable image search Fast correspondence-based search with local features Fast similarity search for learned metrics
Local image features
How to handle sets of features? Want to compare, index, cluster, etc. local representations, but: Each instance is unordered set of vectors Varying number of vectors per instance
Comparing sets of local features Previous strategies: Match features individually, vote on small sets to verify Explicit search for one-to- one correspondences Bag-of-words: Compare frequencies of prototype features
Pyramid match kernel optimal partial matching Optimal match: O(m 3 ) Pyramid match: O(mL) m = # features L = # levels in pyramid
Pyramid match: main idea descriptor space Feature space partitions serve to “match” the local descriptors within successively wider regions.
Pyramid match: main idea Histogram intersection counts number of possible matches at a given partitioning.
Image search with matching- sensitive hash functions Main idea: – Map point sets to a vector space in such a way that a dot product reflects partial match similarity (normalized PMK value). – Exploit random hyperplane properties to construct matching-sensitive hash functions. – Perform approximate similarity search on hashed examples.
Locality Sensitive Hashing (LSH) Q h r 1 …r k XiXi N h << N Q Guarantee “approximate”- nearest neighbors in sub- linear time, given appropriate hash functions. Randomized LSH functions
LSH functions for dot products The probability that a random hyperplane separates two unit vectors depends on the angle between them: A) High dot product: unlikely to split B) Lower dot product: likely to split Corresponding hash function:
Metric learning There are various ways to judge appearance/shape similarity… but often we know more about (some) data than just their appearance.
Metric learning Exploit partially labeled data and/or (dis)similarity constraints to construct more useful distance function Can dramatically boost performance on clustering, indexing, classification tasks. Various existing techniques
Fast similarity search for learned metrics Goal: – Maintain query time guarantees while performing approximate search with a learned metric Main idea: – Learn Mahalanobis distance parameterization – Use it to affect distribution from which random hash functions are selected LSH functions that preserve the learned metric Approximate NN search with existing methods
Fast Image Search for Learned Metrics It should be unlikely that a hash function will split examples like those having similarity constraints… …but likely that it splits those having dissimilarity constraints. h( ) = h( )h( ) ≠ h( ) Learn a Malhanobis metric for LSH
Local image features useful, important to handle efficiently Introduced scalable methods to allow fast similarity search methods with – Local feature matching – Learned Mahalanobis metrics Key idea: design hash functions that encode matching process, or the constraints provided Summary