1. Unsupervised Learning of Categories from Sets of Partially Matching Image Features Dominic Rizzo and Giota Stratou

2. 2 Grauman and Darrell’s Work Detects categories Detects categories Pyramid match kernel Pyramid match kernel Graph partitioning Graph partitioning Unsupervised or Semi- supervised Unsupervised or Semi- supervised Prototypical image selection Prototypical image selection Caltech-4 Caltech-4 Subset we implemented Subset we implemented Unsupervised category detection Unsupervised category detection Pyramid match kernel Pyramid match kernel Normalized cuts Normalized cuts Subset of Caltech-256 (8) Subset of Caltech-256 (8)

3. 3 Algorithm Outline DetectInterestPointsExtractFeaturesTruncateFeature Vectors & LexicographicallySort CalculatePyramidMatchScore Partition Affinity Matrix

4. 4 Feature Extraction The Grid Harris-AffinePCA-SIFT Harris-AffinePCA-SIFT Harris-AffinePCA-SIFT Caltech-256 Image Dataset (Raw Images) Caltech-256 Image Dataset (Feature Vectors)

5. 5 Pyramid Match Kernel PMK histograms indirectly calculated PMK histograms indirectly calculated If direct, First histogram ~ 10,000^10 elements If direct, First histogram ~ 10,000^10 elements Size decreases by 1/2 each increment Size decreases by 1/2 each increment Matlab can’t handle that Matlab can’t handle that Calculate intersections Calculate intersections Histograms are sparse Histograms are sparse Compare feature locations directly Compare feature locations directly Lexicographic sort Lexicographic sort Pointer walking Pointer walking C-MEX function for inner loop C-MEX function for inner loop Still the slowest part of the code Still the slowest part of the code

6. 6 Grouping as Graph Partitioning G=(V,E)→ DISJOINT A,B CUT(A,B)= OPTIONAL BIPARTITIONING: min CUT(A,B) NORMALISED CUT:

7. 7 Grouping as Graph Partitioning (2) G=(V,E) D=diagonal d(i)= W=similarity weights

8. 8 Recursive Two-Way N-Cut USE SECOND SMALLER EIGENVECTOR TO PARTITION INTO TWO PARTS USE SECOND SMALLER EIGENVECTOR TO PARTITION INTO TWO PARTS NEED DECISION THRESHOLD NEED DECISION THRESHOLD CAN RUN RECURSIVELY FOR MORE CATEGORIES CAN RUN RECURSIVELY FOR MORE CATEGORIES

9. 9 Simultaneous K-Way Cut with Multiple Eigenvectors KEEP TOP K EIGENVECTORS KEEP TOP K EIGENVECTORS NEED CLUSTERING ALGORITHM (i.e k-MEANS) NEED CLUSTERING ALGORITHM (i.e k-MEANS) min Ncut(A1,A2,…Ak) min Ncut(A1,A2,…Ak)

10. 10 Results Example: TWO CATEGORIES: 95 images 100 images Ncut 16 + 179 hit ratio: 111/195= 0.5692 false alarm ratio: 79/195= 0.4051

11. 11 Pyramid Match Kernel Weighted intersection of multi-resolution histograms Weighted intersection of multi-resolution histograms Similarity scoring Similarity scoring O(dm log D) O(dm log D) d-dimensional features d-dimensional features m features m features Maximal range D Maximal range D Relatively fast Relatively fast

12. 12 Affinity Matrix Fully connected graph of images Fully connected graph of images N x N matrix N x N matrix Nodes are images Nodes are images Edges are affinities Edges are affinities Affinity scores Affinity scores Computed via pyramid match kernel Computed via pyramid match kernel

13. 13 Graph Partitioning Clusters categories of images Clusters categories of images Recursively partition Recursively partition Min-normalized cut Min-normalized cut NP-complete NP-complete Approximate solution Approximate solution Eigenvalue problem Eigenvalue problem K is the image affinity matrix K is the image affinity matrix D = f(K) D = f(K) x is the partition x is the partition