1. 2008/7/10 1 以空間關係相鄰圖為基礎之空間關 係相似性量測方法 Retrieval by spatial similarity based on interval neighbor group 研究生：黃彥人 指導教授：蔣依吾博士 中山大學資訊工程學系

2. 2008/7/10 2 Outline  Introduction  Retrieval by Spatial Similarity (RSS) –Single-Instance  Between two pairwise spatial relations  Between a set of pairwise spatial relations and a pairwise spatial relation  Between two sets of pairwise spatial relations –Multiple-Instance  Experimental study  Conclusions and future work

3. 2008/7/10 3 Image retrieval  Symbolic manipulation –Google image search –Drawback: ambiguity  Content-Based Image Retrieval (CBIR) –Visual features  Color [S. Berretti,2002]  Shape[W. C. Lin, 2007]  Texture[P. W. Huang, 2003] –Relationships features

4. 2008/7/10 4 Motive  Retrieval by Spatial Similarity (RSS) –The fine granularity of spatial similarity measure –Similarity ranking Query image

5. 2008/7/10 5 Flowchart Image Image Database Image Indexing Image Retrieval Spatial relation feature retrieval Query image Spatial relation feature Similarity measure SRS Results Similarity ranking Similarity measure Image Database Spatial relation feature retrieval Spatial relation feature Spatial relation feature retrieval  One v.s One  Many v.s One  Many v.s Many

6. 2008/7/10 6 Image Indexing True image picture Symbolic picture[1991] Spatial Reasoning and Indexing Image Database MBR (Minimum Bounding Rectangle )

7. 2008/7/10 7 Spatial Reasoning and Indexing 9DLT Matrix 2d string Pointset SMRSRC SRR PN Matrix 2d G-string 2d B-string 2d H-string 2d PIR-string 2d C-string2d C+ string 2d Bε-string GPN Matrix SRRMSR SBA WeiRe’s method 2d+ string [91] [87] [91] [97] [2001] [88] [92] [95] [88] [92] [97][2001] [2004][2005] [99] [2001] [2005] [94] [92] 2d string

8. 2008/7/10 8 RSS (Retrieval by Spatial Similarity) Query image Image Database Exact match [chang, 2007] Subimage retrieval Similarity retrieval RSS (Retrieval by Spatial Similarity) YesNo Ranking

9. 2008/7/10 9 13 1D spatial relations  Interval Neighbor Group [Lee and Hsu, 1992] [C. Freska, 1994]  Neighbor  Path  The shortest path  Pairwise Node  Distance  The shortest distance  Similarity 1

10. 2008/7/10 10 One v.s One 1, 1: exact match, 0: total irrelevance

11. 2008/7/10 11 Query image Q Database image D 4+1=5

12. 2008/7/10 12 Many v.s One  n objects : spatial relations Query image Q Database image D

13. 2008/7/10 13

14. 2008/7/10 14 Many v.s Many  Frequently encountered in the database retrieval applications Query image Q Database image D

15. 2008/7/10 15

16. 2008/7/10 16 Image Image Database Image Indexing Image Retrieval Spatial relation feature retrieval Query image Spatial relation feature Similarity measure SRS Spatial relation feature Spatial relation feature retrieval Similarity ranking Results Spatial relation bit sequence coding

17. 2008/7/10 17 Node bit Label bit Distance 8-bit Distance 4-bit SuperNode SimpleNode SuperNode v.s SuperNode SuperNode v.s SimpleNode SimpleNode v.s. SimpleNode 1101 Exclusive-OR 0001 1100 1101 OR 0011 1111

18. 2008/7/10 18 Image transformation  Rotated by [2007] A before B A ~overlap BA after B A ~overlap B

19. 2008/7/10 19

20. 2008/7/10 20 Database simulation  1000 fundamental pictures –MBR-covered object random dimension and positioning –2 to 5 MBR-covered objects –7000 pictures in a database  2 to 7 MBR-covered objects

21. 2008/7/10 21 Experimental study  Comparison:RSS-ING v.s 2D Be-string[2003] –13 1D spatial relations Query image Q Database image D 12 longest common subsequence (LCS)

22. 2008/7/10 22 Query image 2D Be-string RSS-ING Single-Instance retrieval results based on similarity measure

23. 2008/7/10 23 RSS-ING v.s 2D Be-string Single-Instance retrieval results based on similarity measure Image transformation (Rotated by ) Query image RSS-ING Single- Instance 2D Be-string Single-Instance

24. 2008/7/10 24 Image Database Image Indexing Image Retrieval Spatial relation feature retrieval Spatial relation feature Similarity measure SRS Results Similarity ranking Spatial relation feature Spatial relation feature retrieval Image Query image Flowchart Multiple query images Spatial relation feature retrieval MIL

25. 2008/7/10 25 MIL (Multiple-Instance Learning)  Using a single image to query a database might employ spatial relations that do not match user ’ s expectation Concept = { Stone, Waterfall }– { Sky, Cloud, Grass, Stone } ={ Waterfall } ∩ ∩ ━ ： ━ = ?? ∩ ∩

26. 2008/7/10 26 Common spatial feature Retrieval by Spatial Similarity Two Positive images Diverse Density [Maron, 1998] Ideal spatial feature MIL

27. 2008/7/10 27 Single-Instance v.s. Multiple-Instance retrieval results based on similarity measure by two Positive images RSS-ING Single-Instance Ideal spatial feature RSS-ING Multiple-Instance Single query image

28. 2008/7/10 28 Multiple-Instance v.s. Multiple- Instance retrieval results based on similarity measure by two Positive images and one negative image Ideal spatial feature one negative image RSS-ING Multiple-Instance 2 example images RSS-ING Multiple-Instance 3 example images

29. 2008/7/10 29 RSS-ING v.s 2D Be-string Multiple-Instance retrieval results based on similarity measure 2D Be-string Multiple-Instance 3 example images Ideal spatial feature RSS-ING Multiple-Instance 3 example images

30. 2008/7/10 30 Execution time and memory space 2D Be-stringRSS-ING Indexing for a database with 7000 images 5.65 sec 25983 KB 3.06 sec 14046 KB Similarity retrieval of a query with one pairwise spatial relation 117.8 ms75.4 ms Similarity retrieval of a query with three pairwise spatial relations 194.4 ms112.6 ms Similarity retrieval of a query with ten pairwise spatial relations 526.7ms242.8ms

31. 2008/7/10 31 Conclusions and future work  The degree of similarity between two spatial relations is linked to the distance between the associated nodes in an ING  Quantitatively ranked according to the degree of similarity with the query  Fine granularity  MIL procedure identifies common positive features and excludes negative ones to further clarify the user ’ s searching criteria  Future work –Video employment –Other visual features