1. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 20011 Indexing Images with Multiple Regions Euripides G.M. Petrakis petrakis@ced.tuc.gr Dept. of Electronic and Computer Engineering Technical University of Crete (TUC)

2. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 20012 Problem Definition: Given a database with N images. Retrieve images similar to a query Q.  Similar objects;  Similar spatial relationships. Respond faster than sequential scanning. Use an index to answer two type of image queries.  D(Q,I) <= t (range queries);  Retrieve the k most similar images (NN queries).

3. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 20013 Indexing Approach Each object is represented by an n-dimensional feature vector (v 1 v 2 …v n ).  E.g., (size, orientation, roundness, colour, texture).  Distance between objects D f : any vector distance like Euclidean, Manhattan etc. Map each vector to a n-dimensional feature space.  Each region  one point;  Image (query) with many regions  multiple points. Apply a SAM for indexing (R-tree, SR-tree etc).

4. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 20014 Mapping images I=(I 1,I 2, I 3 ) and J=(J 1,J 2 ) and query Q=(Q 1,Q 2 ) Q1Q1 Q 2 I1I1 I2I2 I3I3 J1J1 J2J2 t t size roundnessroundness

5. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 20015 Problems with SAMs  A SAM can treat only one point (region in our case) per image or query.  Existing algorithms can treat range or NN queries for each Q 1 or Q 2 but not for Q as a whole. Eg., find the k –NNs of Q 1 or Q 2 ; Similarly for range queries.  A SAM retrieves the k-NNs with respect to D f not to D (distance between whole images). D = function (D f )

6. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 20016 Our contributions We formulate the problem of image indexing as one of spatial searching using existing SAMs. We show how a SAM can be used treat images and queries with multiple objects and answer  Nearest Neighbor queries;  Range queries. Two algorithms are proposed, one for each type of query.

7. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 20017 Range Queries Input: query Q, distances D, D f, tolerance t. Output: images I satisfying D(Q,I) <= t. 1.Decompose Q = (Q 1,Q 2,…,Q n ); 2.Apply D f (Q i,I j ) <= t  store results in A i ; 3.Compute ; 4.For each I in A compute D(Q,I); 5.Output images satisfying D(Q,I) <= t;

8. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 20018 Nearest Neighbor (NN) Queries Input: query Q, distance D, D f, number k. Output: the k images most similar to Q. 1.Decompose Q = (Q 1,Q 2,…,Q n ); 2.Apply a k-NN query for each Q i. Retrieve k distinct images (incremental k-NN search); Compute t i = their max distance from Q; 3.Compute t = min{t i }; 4.Apply a range query D(Q,I) <= t; 5.Output the k images closest to Q.

9. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 20019 Comments on the Two Algorithms Assumption: image distance satisfies the Lower Bounding Principle D f (Q,I) <= D(Q,I).  Careful design of distance is necessary;  No false dismissals or false drops. The performance depends on t: the lower the t the faster the algorithms are.  NN queries are slower than range queries;  Optimization: do not apply all Q i ’s. NN search requires incremental k-NN search.

10. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 200110 Definition Image Distance (1) Image matching as an assignment problem (Hungarian algorithm). D(Q,I) : cost of the best mapping of objects of Q to objects in I. Cost of a mapping. C(Q,I) = Σ D f (i,j). D(Q,I) = min { C(Q,I) }. D f (Q,I) <= D(Q,I) ! Ignores relationships.

11. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 200111 Experiments Dataset: 13,500 synthetic images.  each image contains 4-8 objects;  90,000 vectors are stored in an R-tree;  search in the main memory. The results are averages over 20 queries. Demonstrate the superiority of the proposed approach over sequential scan searching.

12. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 200112 Speed-up: Range Queries

13. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 200113 Speed-up: NN queries

14. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 200114 Scale-up: Range Queries

15. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 200115 Scale-up: NN Queries

16. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 200116 Conclusions Interesting problem.  image, video retrieval, data mining etc. Disadvantages of the proposed solution:  Suitable for “small” images with 4-8 objects;  Require careful design of the distance;  Use of incremental NN search. More efficient algorithms are necessary.

17. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 200117 Definition of Image Distance (2) Image matching as a transformation of the ARG of I to the ARG of Q (A* algorithm).  D(Q,I): minimum cost transformation. Cost of a transformation C(Q,I) = max { D f (i,j) }. D f (Q,I) <= D(Q,I)!

18. Euripides G.M. PetrakisIR'2001 Oulu, 19-22 Sept. 200118 Retrieval Example