PMLAB Finding Similar Image Quickly Using Object Shapes Heng Tao Shen Dept. of Computer Science National University of Singapore Presented by Chin-Yi Tsai

PMLAB 2 Outline Motivation Motivation Related Work Related Work A Hierarchical Partitioning Framework Via Angle Mapping A Hierarchical Partitioning Framework Via Angle Mapping Hierarchical Partitioning With Shape Representations Hierarchical Partitioning With Shape Representations Hierarchical Partitioning With Angle Vectors Hierarchical Partitioning With Angle Vectors Experiments Experiments Conclusion Conclusion

PMLAB 3 Introduction There are two requirements for a content- based image -> effectiveness, efficiency There are two requirements for a content- based image -> effectiveness, efficiency Identifying relevant image quickly from a large images Identifying relevant image quickly from a large images A framework for fast image retrieval based on object shapes extracted from object within images A framework for fast image retrieval based on object shapes extracted from object within images

PMLAB 4 Introduction (cont ’ d) Images Logical Level N Level N-1 Level N-2 … coarser fewer partitions fewer dimensionality

PMLAB 5 Introduction (cont ’ d) Angle mapping (AM) replaces a sequence of connected edges by a smaller number of edges Angle mapping (AM) replaces a sequence of connected edges by a smaller number of edges –angle > ? –length < ? –Dimensionality

PMLAB 6 Introduction (cont ’ d) Two hierarchical structure to facilitate speedy retrieval Two hierarchical structure to facilitate speedy retrieval Hierarchical Partitioning on Shape Representation (HPSR) Hierarchical Partitioning on Shape Representation (HPSR) –shape representation as the indexing key Hierarchical Partitioning on Angle Vector (HPAV) Hierarchical Partitioning on Angle Vector (HPAV) –angle information as the indexing key

PMLAB 7 Related Work For content-based retrieval system, to map physical objects into logical representation For content-based retrieval system, to map physical objects into logical representation –Color histogram, 2D-strings, symbolic image Decomposing an image into its individual object Decomposing an image into its individual object Several indexing structure (dimensionality) Several indexing structure (dimensionality) –R-tree, R + -tree, R * -tree, TV-tree, NR-tree –HPSR, HPAV ( Angle Mapping )

PMLAB 8 A Hierarchical Partitioning Framework Via Angle Mapping The framework maps high-D into multiple level low-D The framework maps high-D into multiple level low-D AM approximates a shape based on the angles between edges AM approximates a shape based on the angles between edges Sharper angles and longer edges carry more important information about shape Sharper angles and longer edges carry more important information about shape Angle Interval (AI) Angle Interval (AI) –AI[i] = ( * (i - 1) / N, * i / (N)] or –AI[i] = ( * (i - 1) / N, * i / (N)] –If N=3, (150, 180), (120, 150), (90, 120) Prune Length Threshold (PLT) Prune Length Threshold (PLT)

PMLAB 9 A Hierarchical Partitioning Framework Via Angle Mapping Logical Level-2 Level-1 Level-3 AI[3] = ( 150, 180 ) AI[2] = ( 120, 150 ) AI[1] = ( 90, 120 )

PMLAB 10 A Hierarchical Partitioning Framework Dimension Reduction Ratio Dimension Reduction Ratio –DRR(i, i-1)=[Dim(R(i)) – Dim(R(i-1))] / Dim(R(i)) (9-3)/9 = 2/3

PMLAB 11 1.Find the shape dominating outline and initialize it as level N representation 1.Find the shape dominating outline and initialize it as level N representation 2.for level i from N to 1 do 2.for level i from N to 1 do –2.1 check if angle lies in AI[i] –2.2 check edge < PLT –2.3 go back 2.1 or 2.2 –2.4 get level-i representation –2.5 if there are too many shapes at level i  cluster similar shapes  identify a representative from the shape Algorithm: A Hierarchical Partitioning Framework ( N = 2 )

PMLAB 12 Hierarchical Partitioning With Shape Representations Two shapes that are similar are grouped as a cluster Two shapes that are similar are grouped as a cluster Representation Reduction Ratio Representation Reduction Ratio –RRR(i, i-1) = NumberOfNodesAtLevel(i) / NumberOfNodesAtLevel(i-1) RRR(N,N-1) =(6-4)/6=1/3 Level N Level N-1

PMLAB 13 Algorithm: HPSR ( 2-level HPSR Indexing ) 1. Initialize n to N 1. Initialize n to N 2. For all logical shapes, apply AM 2. For all logical shapes, apply AM to get their level-n representation to get their level-n representation 3. Merge identical representation into 3. Merge identical representation into partition partition 4. If number of partition is too big, 4. If number of partition is too big, cluster similar representation given cluster similar representation given distance threshold distance threshold 5. For each partition, produce partition ’ s 5. For each partition, produce partition ’ s representative as a node at level n representative as a node at level n 6. Build connection between level-n and 6. Build connection between level-n and their children their children 7. if n>1, map level n nodes to level n-1 7. if n>1, map level n nodes to level n-1 representations and decrease n by 1, then go to step 3. representations and decrease n by 1, then go to step 3.

PMLAB 14 Algorithm: Query Processing in HPSR ( 2-level HPSR Indexing ) 1. Generate query shape ’ s N multi-level 1. Generate query shape ’ s N multi-level representation representation 2. Initialize level n as 1 2. Initialize level n as 1 3. Compute the similarity between query 3. Compute the similarity between query shape ’ s level-n representation and level n shape ’ s level-n representation and level n nodes nodes 4. If no similar node is found, return null 4. If no similar node is found, return null 5. If leaf level is reached, get shape ’ s logical 5. If leaf level is reached, get shape ’ s logical representation and compute the similarity representation and compute the similarity with query shape. Return those similar with query shape. Return those similar shapes shapes 6. Increase n by 1 and retrieval the similar 6. Increase n by 1 and retrieval the similar node ’ s children as level-n nodes. Go back to step 3 node ’ s children as level-n nodes. Go back to step 3

PMLAB 15 HPSR Matching To get the similarity between two dominating outlines, this paper applies the turning function To get the similarity between two dominating outlines, this paper applies the turning function V x a(x) v 1

PMLAB 16 Hierarchical Partitioning With Angle Vectors AV for level-1 representation AV for level-2 representation AV for level-3 representation AV for logical representation AI[3] = ( 150, 180 ) AI[2] = ( 120, 150 ) AI[1] = ( 90, 120 ) AI[0] = ( 0, 90)

PMLAB 17 ( HPSR )( HPAV ) 1. To construct the HPAV structure, we need to build the HPSR structure first 2. Only difference between HPSR and HPAV is the node representation 3. In HPAV tree, each level has much lower fixed dimensions 4. HPAV has fewer storage requirement

PMLAB 18 Similarity measure for AV comparison Definition:

PMLAB 19 Experiments Set up Set up –P-III 700MHz with 128 Mbytes of RAM To evaluate the two structure HPSR and HPAV on their effectiveness and efficiency, as well as storage requirement To evaluate the two structure HPSR and HPAV on their effectiveness and efficiency, as well as storage requirement

PMLAB 20 Tuning Mapping Level and PLT During the Angle Mapping process, two parameter may affect the results During the Angle Mapping process, two parameter may affect the results –Mapping level N, Prune Length Threshold (PLT) Choose the optimal value for N and PLT Choose the optimal value for N and PLT

PMLAB 21 lower-dimension more Angle Interval too many level coarser approximation

PMLAB 22 Effectiveness and Efficiency of HPAR and HPAV

PMLAB 23 Storage Space for IMLR and IAV Tree

PMLAB 24 Conclusion A framework for partitioning image database based on shapes of objects A framework for partitioning image database based on shapes of objects Use hierarchy of approximation of shapes that reduces the dimensionality of shapes... (AM) Use hierarchy of approximation of shapes that reduces the dimensionality of shapes... (AM) Meet the user ’ s performance requirement Meet the user ’ s performance requirement Two indexing structure based on the framework Two indexing structure based on the framework –HPSR, HPAV