Computer Vision Group Department of Computer Science University of Illinois at Urbana-Champaign

Multimodal Information Access & Synthesis Sang Hyun Park Joel Quintana Robert Rand David Forsyth Recognition and Efficient Retrieval of Similar Images in Large Datasets Using Visual Words

Multimodal Information Access & Synthesis  Abstract  Process  Demo  Results  Future Work  Questions Outline

Multimodal Information Access & Synthesis Problem: We want to... Identify pictures by content rather than color. Compare large sets of images to find near duplicates. Recognize similar pictures despite small changes. Challenges: Similar Images can be... in different filenames. in different formats. in different sizes and arrangements. stretched, skewed, colored and otherwise altered. Abstract

Multimodal Information Access & Synthesis Why would we want to find near duplicate images?  News Reports  Forged Photos  Social Networks  Picture/Mugshot Matching  Weapon & Symbol ID Application

Multimodal Information Access & Synthesis Process Images Database Part1: Get the Visual Words Extract SIFT Features Interest Points Database Group Similar Interest Points (Kmeans) List of General Points (Visual Words)

Multimodal Information Access & Synthesis Process Original Images Database Part2: Store Histograms of Visual Words of the Images on the Database Image Add Histogram to the Histograms Database Histograms Database Calculate Histogram of Visual Words Histogram of Visual Words

Multimodal Information Access & Synthesis Process Part3: Retrieval of Similar Images New Image Histograms Database Calculate Histogram of Visual Words Histogram of Visual Words Query For Nearest Neighbors List Of Nearest Neighbors

Multimodal Information Access & Synthesis Demo

 Current Configuration –10 Interest points per image –3000 Visual words (K-mean Clustering) –KDTree to get approximate nearest neighbors –Precision : ~0.50  Future Configuration –All Interest points from images –More Clustering Algorithms (Hierarchical K-means / KDTree) –Usage of Full Potential of FLANN Results

Multimodal Information Access & Synthesis  Bigger Database – Web (Image Search Engine / Flicker / Facebook)  Multiple Queries –Parallelized Processing (Efficient Processing of Queries)  Other Application –Detection of objects inside images: logos, symbols, tattoos, weapons, etc –Finding relationships between people according to their common pictures on social networks Future Work

Multimodal Information Access & Synthesis Questions