1. VisualRank: Applying PageRank to Large-Scale Image Search Yushi Jing, Member, IEEE Shumeet Baluja, Member, IEEE IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, NOVEMBER 2008 [24] Y. Jing, S. Baluja, and H. Rowley, “Canonical Image Selection from the Web,” Proc. Sixth Int’l Conf. Image and Video Retrieval, pp. 280-287, 2007.

2. Outline Introduction Similarity graph[24] PageRank & VisualRank Hashing Experiments Conclusion

3. Outline Introduction Similarity graph[24] PageRank & VisualRank Hashing Experiments Conclusion

4. Search for “d80” & “coca cola” by traditional search engine

5. Introduction Visual theme, ex: “coca cola” logo CBIR: content-based image retrieval – Pure – Composite “Visual-filter” via Probabilistic Graphical Models(PGMs)[7] Compare: – Object category learner – image search engine [7] R. Fergus, P. Perona, and A. Zisserman, “A Visual Category Filter for Google Images,” Proc. Eighth European Conf. Computer Vision, pp. 242-256, 2004.

6. Introduction

7. Combine[24] – pairwise visual similarity among images – nonvisual signals VisualRank – Based on PageRank – Large number of queries & images Goal – More accurate search ranking

8. introducton

9. Outline Introduction Similarity graph[24] PageRank & VisualRank Hashing Experiments Conclusion

10. Features generation Local descriptor – SIFT & compare[29] [29] K. Mikolajczyk and C. Schmid, “A Performance Evaluation of Local Descriptors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615-1630, Oct. 2005.

11. Similarity graph pairwise

12. Similarity graph Top 1000 results of “Mona-lisa”

13. Outline Introduction Similarity graph[24] PageRank & VisualRank Hashing Experiments Conclusion

14. PageRank Conception – Vote – eigenvector centrality A BDC PR(A) = PR(B) + PR(C) + PR(D)

15. PageRank A B D C

16. q=0.15 Random walk

17. PageRank Markov matrix

18. VisualRank usually d>0.8

19. Link spam Well connected image V.S. VisualRank, “Nemo”

20. Outline Introduction Similarity graph[24] PageRank & VisualRank Hashing Experiments Conclusion

21. Matching Precluster – “Paris”, “Eiffel Tower”, and “Arc de Triomphe” Top-N, and compute VisualRank Hashing – Locality Sensitive Hashing (LSH) – Feature descriptor as the key

22. Locality Sensitive Hashing (LSH) An approximate k-NN technique Hash function: – a is d-dimensional random vector – b is real number from range – W defines the quantization of the features – V is the original feature vector

23. Flow(1/3) 1.Resize 500*500 pix, 1000 web images 3000,000 to 700,000 feature vectores 2.L hash table H=H1, H2,…,HL, each with K hash functions, L=40, W=100, K=3

24. Flow(2/3) 3.Matched descriptor – Have same key more than C=3 hash table 4.Hough Transform

25. Flow(3/3) 5.Similarity – Matched images More than 3 features – no. of matches divide by their avg. number of local features 6.Given similarity matrix S, and use VisualRank

26. Outline Introduction Similarity graph[24] PageRank & VisualRank Hashing Experiments Conclusion

27. Experiments 2,000 most popular product queries on Google, ex: “ipod”, “Xbox” the top 1,000 search results each query in July 2007 Google Filter – Fewer than 5% images at least one connection – Remaining 1,000 queries

28. Experiment 1 Evaluate – “irrelevancy” of our ranking Mixed Top 10 VisualRank & top 10 google Remove duplicates and ask “which are least relevant?” Ask 150 evaluators, randomly 50 queries

34. Experiment 2 VisualRank bias, p T =V j T =[1/m, …, 1/m, 0, …, 0] HeuristicRank – a pure CBIR system j

36. Experiment 3 Collected 40 top images each click numbers from google Compare – Sum of VisualRank top 20 click numbers – Sum of default ranking top 20 click numbers VisualRank exceeds 17.5% than default Google ranking

37. Landmarks 80 common landmark, ex: “Eiffel Tower,”“Big Ben,” “Coliseum,” and “Lincoln Memorial.”

38. Outline Introduction Similarity graph[24] PageRank & VisualRank Hashing Experiments Conclusion

39. VisualRank applying PageRank conception and combined – Default Google ranking – similarity graph between images VisualRank can outperform the default Google on the vast majority of queries Reduce the number of irrelevant images efficiently