Dependence Language Model for Information Retrieval Jianfeng Gao, Jian-Yun Nie, Guangyuan Wu, Guihong Cao, Dependence Language Model for Information Retrieval,

Slides:

Advertisements

Similar presentations

Information Retrieval and Organisation Chapter 12 Language Models for Information Retrieval Dell Zhang Birkbeck, University of London.

Advertisements

Improvements and extras Paul Thomas CSIRO. Overview of the lectures 1.Introduction to information retrieval (IR) 2.Ranked retrieval 3.Probabilistic retrieval.

Language Models Naama Kraus (Modified by Amit Gross) Slides are based on Introduction to Information Retrieval Book by Manning, Raghavan and Schütze.

Information retrieval – LSI, pLSI and LDA

A Formal Study of Information Retrieval Heuristics Hui Fang, Tao Tao, and ChengXiang Zhai University of Illinois at Urbana Champaign SIGIR 2004 (Best paper.

1 Language Models for TR (Lecture for CS410-CXZ Text Info Systems) Feb. 25, 2011 ChengXiang Zhai Department of Computer Science University of Illinois,

A Maximum Coherence Model for Dictionary-based Cross-language Information Retrieval Yi Liu, Rong Jin, Joyce Y. Chai Dept. of Computer Science and Engineering.

Information Retrieval Models: Probabilistic Models

Language Model based Information Retrieval: University of Saarland 1 A Hidden Markov Model Information Retrieval System Mahboob Alam Khalid.

Chapter 7 Retrieval Models.

Incorporating Language Modeling into the Inference Network Retrieval Framework Don Metzler.

A Markov Random Field Model for Term Dependencies Donald Metzler and W. Bruce Croft University of Massachusetts, Amherst Center for Intelligent Information.

Introduction to Information Retrieval Introduction to Information Retrieval Hinrich Schütze and Christina Lioma Lecture 11: Probabilistic Information Retrieval.

1 CS 430 / INFO 430 Information Retrieval Lecture 12 Probabilistic Information Retrieval.

T.Sharon - A.Frank 1 Internet Resources Discovery (IRD) IR Queries.

Probabilistic IR Models Based on probability theory Basic idea : Given a document d and a query q, Estimate the likelihood of d being relevant for the.

1 CS 430 / INFO 430 Information Retrieval Lecture 12 Probabilistic Information Retrieval.

Latent Dirichlet Allocation a generative model for text

Integrating term dependencies according to their utility Jian-Yun Nie University of Montreal 1.

1 LM Approaches to Filtering Richard Schwartz, BBN LM/IR ARDA 2002 September 11-12, 2002 UMASS.

Scalable Text Mining with Sparse Generative Models

Language Modeling Approaches for Information Retrieval Rong Jin.

1 Advanced Smoothing, Evaluation of Language Models.

Modeling (Chap. 2) Modern Information Retrieval Spring 2000.

Multi-Style Language Model for Web Scale Information Retrieval Kuansan Wang, Xiaolong Li and Jianfeng Gao SIGIR 2010 Min-Hsuan Lai Department of Computer.

Query Rewriting Using Monolingual Statistical Machine Translation Stefan Riezler Yi Liu Google 2010 Association for Computational Linguistics.

Probabilistic Model for Definitional Question Answering Kyoung-Soo Han, Young-In Song, and Hae-Chang Rim Korea University SIGIR 2006.

1 Bayesian Learning for Latent Semantic Analysis Jen-Tzung Chien, Meng-Sun Wu and Chia-Sheng Wu Presenter: Hsuan-Sheng Chiu.

Topic Models in Text Processing IR Group Meeting Presented by Qiaozhu Mei.

1 Formal Models for Expert Finding on DBLP Bibliography Data Presented by: Hongbo Deng Co-worked with: Irwin King and Michael R. Lyu Department of Computer.

Bayesian Sets Zoubin Ghahramani and Kathertine A. Heller NIPS 2005 Presented by Qi An Mar. 17 th, 2006.

A Markov Random Field Model for Term Dependencies Donald Metzler W. Bruce Croft Present by Chia-Hao Lee.

Bayesian Extension to the Language Model for Ad Hoc Information Retrieval Hugo Zaragoza, Djoerd Hiemstra, Michael Tipping Presented by Chen Yi-Ting.

Topic Modelling: Beyond Bag of Words By Hanna M. Wallach ICML 2006 Presented by Eric Wang, April 25 th 2008.

Effective Query Formulation with Multiple Information Sources

Web Image Retrieval Re-Ranking with Relevance Model Wei-Hao Lin, Rong Jin, Alexander Hauptmann Language Technologies Institute School of Computer Science.

Less is More Probabilistic Models for Retrieving Fewer Relevant Documents Harr Chen, David R. Karger MIT CSAIL ACM SIGIR 2006 August 9, 2006.

LANGUAGE MODELS FOR RELEVANCE FEEDBACK Lee Won Hee.

1 Introduction to Natural Language Processing ( ) LM Smoothing (The EM Algorithm) AI-lab

Chapter 23: Probabilistic Language Models April 13, 2004.

Information Retrieval at NLC Jianfeng Gao NLC Group, Microsoft Research China.

Gravitation-Based Model for Information Retrieval Shuming Shi, Ji-Rong Wen, Qing Yu, Ruihua Song, Wei-Ying Ma Microsoft Research Asia SIGIR 2005.

Relevance-Based Language Models Victor Lavrenko and W.Bruce Croft Department of Computer Science University of Massachusetts, Amherst, MA SIGIR 2001.

Mining Dependency Relations for Query Expansion in Passage Retrieval Renxu Sun, Chai-Huat Ong, Tat-Seng Chua National University of Singapore SIGIR2006.

Language Modeling Putting a curve to the bag of words Courtesy of Chris Jordan.

NTNU Speech Lab Dirichlet Mixtures for Query Estimation in Information Retrieval Mark D. Smucker, David Kulp, James Allan Center for Intelligent Information.

Date: 2012/11/29 Author: Chen Wang, Keping Bi, Yunhua Hu, Hang Li, Guihong Cao Source: WSDM’12 Advisor: Jia-ling, Koh Speaker: Shun-Chen, Cheng.

Query Suggestions in the Absence of Query Logs Sumit Bhatia, Debapriyo Majumdar,Prasenjit Mitra SIGIR’11, July 24–28, 2011, Beijing, China.

Minimum Rank Error Training for Language Modeling Meng-Sung Wu Department of Computer Science and Information Engineering National Cheng Kung University,

A Generation Model to Unify Topic Relevance and Lexicon-based Sentiment for Opinion Retrieval Min Zhang, Xinyao Ye Tsinghua University SIGIR

A Framework to Predict the Quality of Answers with Non-Textual Features Jiwoon Jeon, W. Bruce Croft(University of Massachusetts-Amherst) Joon Ho Lee (Soongsil.

Introduction to Information Retrieval Introduction to Information Retrieval Lecture Probabilistic Information Retrieval.

1 ICASSP Paper Survey Presenter: Chen Yi-Ting. 2 Improved Spoken Document Retrieval With Dynamic Key Term Lexicon and Probabilistic Latent Semantic Analysis.

A Maximum Entropy Language Model Integrating N-grams and Topic Dependencies for Conversational Speech Recognition Sanjeev Khudanpur and Jun Wu Johns Hopkins.

Maximum Entropy techniques for exploiting syntactic, semantic and collocational dependencies in Language Modeling Sanjeev Khudanpur, Jun Wu Center for.

N-Gram Model Formulas Word sequences Chain rule of probability Bigram approximation N-gram approximation.

Recent Paper of Md. Akmal Haidar Meeting before ICASSP 2013 報告者：郝柏翰 2013/05/23.

A Study of Smoothing Methods for Language Models Applied to Ad Hoc Information Retrieval Chengxiang Zhai, John Lafferty School of Computer Science Carnegie.

Introduction to Information Retrieval Probabilistic Information Retrieval Chapter 11 1.

Language Model for Machine Translation Jang, HaYoung.

LEARNING IN A PAIRWISE TERM-TERM PROXIMITY FRAMEWORK FOR INFORMATION RETRIEVAL Ronan Cummins, Colm O’Riordan (SIGIR’09) Speaker : Yi-Ling Tai Date : 2010/03/15.

Bayesian Extension to the Language Model for Ad Hoc Information Retrieval Hugo Zaragoza, Djoerd Hiemstra, Michael Tipping Microsoft Research Cambridge,

A Formal Study of Information Retrieval Heuristics

Information Retrieval Models: Probabilistic Models

A Markov Random Field Model for Term Dependencies

Language Model Approach to IR

Topic Models in Text Processing

CS590I: Information Retrieval

INF 141: Information Retrieval

Conceptual grounding Nisheeth 26th March 2019.

Presentation transcript:

Dependence Language Model for Information Retrieval Jianfeng Gao, Jian-Yun Nie, Guangyuan Wu, Guihong Cao, Dependence Language Model for Information Retrieval, SIGIR 2004

Reference Structure and performance of a dependency language model. Ciprian, David Engle and et al. Eurospeech Parsing English with a Link Grammar. Daniel D. K. Sleator and Davy Temperley. Technical Report CMU-CS

Why we use independence assumption? The independence assumption is one of the assumptions widely adopted in probabilistic retrieval theory. Why? –Make retrieval models easier. –Make retrieval operation tractable. The shortage of independence assumption –Independence assumption does not hold in textual data.

Latest ideas of dependence assumption Bigram –Some language modeling approach try to incorporate word frequency by using bigram. –Shortage: Some of word dependencies not only exist between adjacent words but also exist at more distant. Some of adjacent words are not exactly connected. – Bigam language model showed only marginally better effectiveness than the unigram model. Bi-term –Bi-term language model is similar to the bigram model except the constraint of order in terms is relaxed. –“information retrieval” and “retrieval of information” will be assigned the same probability of generating the query.

Structure and performance of a dependency language model

Introduction This paper present a maximal entropy language model that incorporates both syntax and semantics via a dependency grammar. Dependency grammar: express the relations between words by a directed graph which can incorporate the predictive power of words that lie outside of bigram or trigram range.

Introduction Why we use Ngram –Assume if we want to record we need to store independent parameters The drawback of Ngram –Ngram blindly discards relevant words that lie N or more positions in the past.

Structure of the model

Develop an expression for the joint probability, K is the linkages in the sentence. Then we get Assume that the sum is dominated by a single term, then

A dependency language model of IR A query we want to rank –Previous work: Assume independence between query terms : –New work: Assume that term dependencies in a query form a linkage

A dependency language model of IR Assume that the sum over all the possible Ls is dominated by a single term Assume that each term is dependent on exactly one related query term generated previous.

A dependency language model of IR

Assume –The generation of a single term is independent of L By this assumption, we would have arrived at the same result by starting from any term. L can be represented as an undirected graph.

A dependency language model of IR 取 log

Parameter Estimation Estimating – Assume that the linkages are independent. –Then count the relative frequency of link l between and given that they appear in the same sentence. Have a link in a sentence in training data A score The link frequency of query i and query j

Parameter Estimation assumption Assumption:

Parameter Estimation Estimating –The document language model is smoothed with a Dirichlet prior Dirichilet distribution Constant discount

Parameter Estimation Estimating

Experimental Setting Stemmed and stop words were removed. Queries are TREC topics 202 to 250 on TREC disk 2 and 3.

The flow of the experimental Find the linkage of query query Find the max L by max l P(l|Q) Get documentTraining data For weight computation Count the frequency Get P(L|D) Count the frequency Get Count the frequency Get combine Ranking document

Result-BM & UG BM: binary independent retrieval UG: unigram language model approach UG achieves the performance similar to, or worse than, that of BM.

Result- DM DM: dependency model The improve of DM over UG is statistically significant.

Result- BG BG: bigram language model BG is slightly worse than DM in five out of six TREC collections but substantially outperforms UG in all collection.

Result- BT1 & BT2 BT: bi-term language model

Conclusion This paper introduce the linkage of a query as a hidden variable. Generate each term in turn depending on other related terms according to the linkage. –This approach cover several language model approaches as special cases. The experimental of this paper outperforms substantially over unigram, bigram and classical probabilistic retrieval model.