Challenges in Mining Large Image Datasets Jelena Tešić, B.S. Manjunath University of California, Santa Barbara

Vision Research Lab Introduction Data and event representation Meaningful data summarization Modeling of high-level human concepts Learning events Feature space and perceptual relations Mining image datasets Feature set size and dimension Size and nature of image dataset Aerial Images of SB county 54 images x5428 pixels 177,174 tiles - 128x128 pixels

Vision Research Lab Visual Thesaurus Perceptual Classification 1.T=1; SOM dim. red. of input training feature space 2.Assign labels to SOM output 3.LVQ finer tuning of class boundaries 4.It T< number_of_iterations { T=T+1; go back to step 2. } else END. Perceptual and feature space brought together: same class (16) and class 17 Thesaurus Entries Generalized Lloyd Algorithm 330 codewords

Vision Research Lab Spatial Event Cubes Image tile raster space Thesaurus entries Spatial binary relation ρ SEC face values Multimode SEC distance direction x y p q u v Cρ(u,v) COLORCOLOR TEXTURE SEC

Vision Research Lab Visual Data Mining SEC Cluster Analysis

Vision Research Lab Spatial Data Mining Generalized Apriori 1. Find all sets of tuples that have minimum support 2. Use the frequent itemsets to generate the desired rules Low-level mining Occurrence of the ocean in the image dataset 2D 3D

Vision Research Lab Higher level Mining Ocean analysis

Vision Research Lab Conclusion Visual mining framework Spatial event representation Image analysis at a conceptual level Perceptual knowledge discovery Demos: Amazon forest DV 40 hours – 5tbytes Mosaics from 2 h

Vision Research Lab Adaptive NN Search for Relevance Feedback Relevance Feedback learn user’s subjective similarity measures Scalable solution Explore the correlation of consecutive NN search VA-file indexing Feature space Query Distance Measure - K nearest neighbors at iteration t - distance between Q and the K-th farthest object upper bound - K-th largest upper bound of all approximations

Vision Research Lab Adaptive NN Search for Relevance Feedback If is a qualified one in its lower bound must satisfy When, it is guaranteed that more candidates can be excluded as compared with traditional search

Vision Research Lab Performance Evaluation - 685,900 images vs. Their difference is larger at a coarser resolution vs. At coarser resolution, the estimate is better

Vision Research Lab Performance Evaluation Adaptive NN search Utilizing the correlation to confine the search space The constraints can be computed efficiently Significant savings on disk accesses