Euripides G.M. PetrakisIR'2001 Oulu, Sept Indexing Images with Multiple Regions Euripides G.M. Petrakis Dept. of Electronic and Computer Engineering Technical University of Crete (TUC)

Euripides G.M. PetrakisIR'2001 Oulu, Sept Problem Definition: Given a database with N images. Retrieve images similar to a query Q.  Similar objects;  Similar spatial relationships. Respond faster than sequential scanning. Use an index to answer two type of image queries.  D(Q,I) <= t (range queries);  Retrieve the k most similar images (NN queries).

Euripides G.M. PetrakisIR'2001 Oulu, Sept Indexing Approach Each object is represented by an n-dimensional feature vector (v 1 v 2 …v n ).  E.g., (size, orientation, roundness, colour, texture).  Distance between objects D f : any vector distance like Euclidean, Manhattan etc. Map each vector to a n-dimensional feature space.  Each region  one point;  Image (query) with many regions  multiple points. Apply a SAM for indexing (R-tree, SR-tree etc).

Euripides G.M. PetrakisIR'2001 Oulu, Sept Mapping images I=(I 1,I 2, I 3 ) and J=(J 1,J 2 ) and query Q=(Q 1,Q 2 ) Q1Q1 Q 2 I1I1 I2I2 I3I3 J1J1 J2J2 t t size roundnessroundness

Euripides G.M. PetrakisIR'2001 Oulu, Sept Problems with SAMs  A SAM can treat only one point (region in our case) per image or query.  Existing algorithms can treat range or NN queries for each Q 1 or Q 2 but not for Q as a whole. Eg., find the k –NNs of Q 1 or Q 2 ; Similarly for range queries.  A SAM retrieves the k-NNs with respect to D f not to D (distance between whole images). D = function (D f )

Euripides G.M. PetrakisIR'2001 Oulu, Sept Our contributions We formulate the problem of image indexing as one of spatial searching using existing SAMs. We show how a SAM can be used treat images and queries with multiple objects and answer  Nearest Neighbor queries;  Range queries. Two algorithms are proposed, one for each type of query.

Euripides G.M. PetrakisIR'2001 Oulu, Sept Range Queries Input: query Q, distances D, D f, tolerance t. Output: images I satisfying D(Q,I) <= t. 1.Decompose Q = (Q 1,Q 2,…,Q n ); 2.Apply D f (Q i,I j ) <= t  store results in A i ; 3.Compute ; 4.For each I in A compute D(Q,I); 5.Output images satisfying D(Q,I) <= t;

Euripides G.M. PetrakisIR'2001 Oulu, Sept Nearest Neighbor (NN) Queries Input: query Q, distance D, D f, number k. Output: the k images most similar to Q. 1.Decompose Q = (Q 1,Q 2,…,Q n ); 2.Apply a k-NN query for each Q i. Retrieve k distinct images (incremental k-NN search); Compute t i = their max distance from Q; 3.Compute t = min{t i }; 4.Apply a range query D(Q,I) <= t; 5.Output the k images closest to Q.

Euripides G.M. PetrakisIR'2001 Oulu, Sept Comments on the Two Algorithms Assumption: image distance satisfies the Lower Bounding Principle D f (Q,I) <= D(Q,I).  Careful design of distance is necessary;  No false dismissals or false drops. The performance depends on t: the lower the t the faster the algorithms are.  NN queries are slower than range queries;  Optimization: do not apply all Q i ’s. NN search requires incremental k-NN search.

Euripides G.M. PetrakisIR'2001 Oulu, Sept Definition Image Distance (1) Image matching as an assignment problem (Hungarian algorithm). D(Q,I) : cost of the best mapping of objects of Q to objects in I. Cost of a mapping. C(Q,I) = Σ D f (i,j). D(Q,I) = min { C(Q,I) }. D f (Q,I) <= D(Q,I) ! Ignores relationships.

Euripides G.M. PetrakisIR'2001 Oulu, Sept Experiments Dataset: 13,500 synthetic images.  each image contains 4-8 objects;  90,000 vectors are stored in an R-tree;  search in the main memory. The results are averages over 20 queries. Demonstrate the superiority of the proposed approach over sequential scan searching.

Euripides G.M. PetrakisIR'2001 Oulu, Sept Speed-up: Range Queries

Euripides G.M. PetrakisIR'2001 Oulu, Sept Speed-up: NN queries

Euripides G.M. PetrakisIR'2001 Oulu, Sept Scale-up: Range Queries

Euripides G.M. PetrakisIR'2001 Oulu, Sept Scale-up: NN Queries

Euripides G.M. PetrakisIR'2001 Oulu, Sept Conclusions Interesting problem.  image, video retrieval, data mining etc. Disadvantages of the proposed solution:  Suitable for “small” images with 4-8 objects;  Require careful design of the distance;  Use of incremental NN search. More efficient algorithms are necessary.

Euripides G.M. PetrakisIR'2001 Oulu, Sept Definition of Image Distance (2) Image matching as a transformation of the ARG of I to the ARG of Q (A* algorithm).  D(Q,I): minimum cost transformation. Cost of a transformation C(Q,I) = max { D f (i,j) }. D f (Q,I) <= D(Q,I)!

Euripides G.M. PetrakisIR'2001 Oulu, Sept Retrieval Example