Image Retrieval and Ranking using L.S.I and Cross View Learning Sumit Kumar Vivek Gupta 11739 11816.

Slides:

Advertisements

Similar presentations

A Support Vector Method for Optimizing Average Precision

Advertisements

Optimizing search engines using clickthrough data

Dimensionality Reduction PCA -- SVD

CUBELSI : AN EFFECTIVE AND EFFICIENT METHOD FOR SEARCHING RESOURCES IN SOCIAL TAGGING SYSTEMS Bin Bi, Sau Dan Lee, Ben Kao, Reynold Cheng The University.

Automatic Feature Extraction for Multi-view 3D Face Recognition

Hinrich Schütze and Christina Lioma

DIMENSIONALITY REDUCTION BY RANDOM PROJECTION AND LATENT SEMANTIC INDEXING Jessica Lin and Dimitrios Gunopulos Ângelo Cardoso IST/UTL December

Search and Retrieval: More on Term Weighting and Document Ranking Prof. Marti Hearst SIMS 202, Lecture 22.

Principal Component Analysis

Automatic Image Annotation and Retrieval using Cross-Media Relevance Models J. Jeon, V. Lavrenko and R. Manmathat Computer Science Department University.

1 Latent Semantic Indexing Jieping Ye Department of Computer Science & Engineering Arizona State University

Vector Space Information Retrieval Using Concept Projection Presented by Zhiguo Li

1 Algorithms for Large Data Sets Ziv Bar-Yossef Lecture 4 March 30, 2005

Scott Wen-tau Yih Joint work with Kristina Toutanova, John Platt, Chris Meek Microsoft Research.

A novel log-based relevance feedback technique in content- based image retrieval Reporter: Francis 2005/6/2.

Singular Value Decomposition in Text Mining Ram Akella University of California Berkeley Silicon Valley Center/SC Lecture 4b February 9, 2011.

Automating Keyphrase Extraction with Multi-Objective Genetic Algorithms (MOGA) Jia-Long Wu Alice M. Agogino Berkeley Expert System Laboratory U.C. Berkeley.

TFIDF-space  An obvious way to combine TF-IDF: the coordinate of document in axis is given by  General form of consists of three parts: Local weight.

Singular Value Decomposition

1/ 30. Problems for classical IR models Introduction & Background(LSI,SVD,..etc) Example Standard query method Analysis standard query method Seeking.

The Terms that You Have to Know! Basis, Linear independent, Orthogonal Column space, Row space, Rank Linear combination Linear transformation Inner product.

Introduction to Information Retrieval Introduction to Information Retrieval Hinrich Schütze and Christina Lioma Lecture 18: Latent Semantic Indexing 1.

1 Algorithms for Large Data Sets Ziv Bar-Yossef Lecture 6 May 7, 2006

E.G.M. PetrakisDimensionality Reduction1  Given N vectors in n dims, find the k most important axes to project them  k is user defined (k < n)  Applications:

IIIT Hyderabad Atif Iqbal and Anoop Namboodiri Cascaded.

Joint Image Clustering and Labeling by Matrix Factorization

Longbiao Kang, Baotian Hu, Xiangping Wu, Qingcai Chen, and Yan He Intelligent Computing Research Center, School of Computer Science and Technology, Harbin.

Jinhui Tang †, Shuicheng Yan †, Richang Hong †, Guo-Jun Qi ‡, Tat-Seng Chua † † National University of Singapore ‡ University of Illinois at Urbana-Champaign.

COMP423: Intelligent Agent Text Representation. Menu – Bag of words – Phrase – Semantics – Bag of concepts – Semantic distance between two words.

Introduction to Machine Learning for Information Retrieval Xiaolong Wang.

1 Information Retrieval through Various Approximate Matrix Decompositions Kathryn Linehan Advisor: Dr. Dianne O’Leary.

Non Negative Matrix Factorization

INF 141 COURSE SUMMARY Crista Lopes. Lecture Objective Know what you know.

Thanks to Bill Arms, Marti Hearst Documents. Last time Size of information –Continues to grow IR an old field, goes back to the ‘40s IR iterative process.

Indices Tomasz Bartoszewski. Inverted Index Search Construction Compression.

CpSc 881: Information Retrieval. 2 Recall: Term-document matrix This matrix is the basis for computing the similarity between documents and queries. Today:

Generic text summarization using relevance measure and latent semantic analysis Gong Yihong and Xin Liu SIGIR, April 2015 Yubin Lim.

June 5, 2006University of Trento1 Latent Semantic Indexing for the Routing Problem Doctorate course “Web Information Retrieval” PhD Student Irina Veredina.

LATENT SEMANTIC INDEXING Hande Zırtıloğlu Levent Altunyurt.

SINGULAR VALUE DECOMPOSITION (SVD)

Ranking objects based on relationships Computing Top-K over Aggregation Sigmod 2006 Kaushik Chakrabarti et al.

A NOVEL METHOD FOR COLOR FACE RECOGNITION USING KNN CLASSIFIER

1 Latent Concepts and the Number Orthogonal Factors in Latent Semantic Analysis Georges Dupret

Computer-assisted essay assessment zSimilarity scores by Latent Semantic Analysis zComparison material based on relevant passages from textbook zDefining.

CoNMF: Exploiting User Comments for Clustering Web2.0 Items Presenter: He Xiangnan 28 June School of Computing National.

V. Clustering 인공지능 연구실 이승희 Text: Text mining Page:82-93.

Web Search and Text Mining Lecture 5. Outline Review of VSM More on LSI through SVD Term relatedness Probabilistic LSI.

Generating Query Substitutions Alicia Wood. What is the problem to be solved?

Duc-Tien Dang-Nguyen, Giulia Boato, Alessandro Moschitti, Francesco G.B. De Natale Department to Information and Computer Science –University of Trento.

Learning to Rank: From Pairwise Approach to Listwise Approach Authors: Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li Presenter: Davidson Date:

Affine Registration in R m 5. The matching function allows to define tentative correspondences and a RANSAC-like algorithm can be used to estimate the.

Learning Kernel Classifiers 1. Introduction Summarized by In-Hee Lee.

Cross-modal Hashing Through Ranking Subspace Learning

COMP423: Intelligent Agent Text Representation. Menu – Bag of words – Phrase – Semantics Semantic distance between two words.

1 Dongheng Sun 04/26/2011 Learning with Matrix Factorizations By Nathan Srebro.

Search Engine and Optimization 1. Agenda Indexing Algorithms Latent Semantic Indexing 2.

Designing Cross-Language Information Retrieval System using various Techniques of Query Expansion and Indexing for Improved Performance Hello everyone,

Large-Scale Content-Based Audio Retrieval from Text Queries

Document Clustering Based on Non-negative Matrix Factorization

Deep Compositional Cross-modal Learning to Rank via Local-Global Alignment Xinyang Jiang, Fei Wu, Xi Li, Zhou Zhao, Weiming Lu, Siliang Tang, Yueting.

An Empirical Study of Learning to Rank for Entity Search

Chinese Academy of Sciences, Beijing, China

Statistical Learning Methods for Natural Language Processing on the Internet 徐丹云.

Deep Cross-Modal Hashing

Learning to Rank Shubhra kanti karmaker (Santu)

Example: Academic Search

Bilinear Classifiers for Visual Recognition

Large scale multilingual and multimodal integration

Learning to Rank with Ties

Bug Localization with Combination of Deep Learning and Information Retrieval A. N. Lam et al. International Conference on Program Comprehension 2017.

Presentation transcript:

Image Retrieval and Ranking using L.S.I and Cross View Learning Sumit Kumar Vivek Gupta

Why Mapping in latent space Most of the methods based on single modality (only text). Problems with above are o Different query representation for same search intent o Noise in the surrounding textual data o Tag data required in this approach is limited and expensive.

Problem Statement Given a set of images, a set of queries and query image click data we aim to learn a common latent subspace so that query and image can be directly compared. This space is used for retrieving and ranking of images for a given textual query on basis of their relevance.

Mapping into Common Latent Space q1q1 q2q2 q3q3 v1v1 v2v2 v3v3 q’ 1 q‘ 2 q’ 3 v’ 1 v’ 2 v’ 3 Query Space Image Space Common Latent Space Transformation Matrix

Latent Semantic Indexing Row1: Machine Learning Computer Vision Row2 : Machine Learning Row3 : Computer Vision Row4: Machine Computer Term Feature Matrix After LSI Col1: Machine Col2: Learning Col3: Computer Col4: Vision

Approach/Idea Extract Feature Vectors for Image and query. Apply LSI on query matrix. Find mapping functions, represented by transformation matrices. Find and Minimize Cross View Distance and Structure Preservation using suitable gradient decent algorithm. Optimize algorithm for faster computation.

Results We trained our transformation matrices on a dataset of 309 image query pairs and tested it on a data set of size 145 image query pairs. Two methods for finding accuracy were used: o If the relevant image was the top first result, then give score = 1 otherwise 0. o If the relevant image was within the top three search results then we give a score of 1 else 0. Convergence was found for only few selected value of μ.

Performance vs Tradeoff

Performance vs Tmax

Literature Review Click through based Cross View Learning for image search(2014) by Y.Pan,T.Yao et. Al. Indexing by Latent semantic Analysis(1990) by S.Deerwester, S.T.Dumanas et. Al A feasible method for optimization with orthogonally constraints(2010) by Zaiwen Ven and Watao Yin An Introduction to Latent Semantic Analysis(1998) by T.K. Landeauer,P.W.Foltz and D Laham WSDM 2012 tutorial,Machine Learning for Query document matching in Web Search by Hang Li and Jun Xu

Thank You