Advisor: Hsin-Hsi Chen Reporter: Chi-Hsin Yu Date:

Introduction Topic A: Relevance Assignment Using Query Log to Assign Actual Relevance of Documents for User Communities Topic B: Knowledge Transfer (cloud wisdom in QL) Transfer Learning in Query Log Topic C: Query Understanding Semantic Relation of Query Terms in Query Log Discussions

Query Log From: Intent Based Clustering of Search Engine Query Log, 2009

Issues Perceived relevance v.s. Actual relevance Clicks Click bias in positions Relevance Query – document User intent/goal – queries – documents User community – queries – documents Cost Editorial judgments v.s. model predicted judgments

From: SIGIR 2010 Tutorials

Task (original) Assign relevance judgment for a q-d pair Actual Relevance From: Intent Based Clustering of Search Engine Query Log, 2009

Applications of the predicted relevance judgments (p r ) As meta-features As actual relevance Low cost Samples (Matrix) prpr Ranking Algorithms Performance in dataset (editorial judgment)

But A Dynamic Bayesian Network Click Model for Web Search Ranking (WWW2009, Track: Data Mining/Session: Click Models) Experiments Predicting click-through rate Predicted relevance as a ranking feature Learning a ranking function with predicted relevance From: (WWW2009, Track: Data Mining/Session: Click Models)

Task (Revised) Assign relevance judgment for a ((user community, q), d) pair Not q-d pair Pseudo-query: (user community, q) Models GA DBN: same as proposed click models in WWW09 papers Difficulties Pseudo-query generation (include user information) User clustering/classification Evaluation Joint training (as in the WWW09 papers) Application For detail analysis of personalized search because we can use predicted relevance to substitute the editorial judgment

Query log = cloud wisdom Task: Mining/leverage cloud wisdom in QL Use transfer learning Use QL to learn meta-features

Task Leverage useful structure/knowledge in QL to boost performance of existing datasets (human judgment) Algorithm SCL: structure correspondence learning Difficulties Selection of extended feature s in QL Evaluation As common IR evaluation metrics Expected results (planned experiments) Can improve performance when use whole training dataset Can improve performance when using small training dataset

SCL ACL 2005, EMNLP 2006 Domain Adaptation with Structural Correspondence Learning (EMNLP 2006) From: EMNLP 2006

From Google search suggestions Interpretation “machine learning wiki/amazon”  Concept + in site “machine learning stanford”  concept + in organization “machine learning tutorial/tool/ppt/journal”  concept + in topic/resources “machine learning kernel”  concept + topic

Compare to compound noun semantics Diarmuid ´O S´eaghdha, 2008 From: Diarmuid ´O S´eaghdha, 2008

Beyond static semantic relations Dynamic semantic relations recognition What is the patterns in the process of query reformulation? Is this useful to identify user goal in a session? Can we build new click model based on semantic relation? Pseudo-session A 1.Apple 2.Apple ipod 3.Apple ipod discount Pseudo-session A

Current works (incomplete) From: SIGIR 2010 Tutorials

Research plan Definition of semantic relations in QL Use Google query suggestions to study the types of semantic relations Segmentation of query terms Mapping segmented query terms to ontology Classification of semantic relation in QL Mining important statistics from QL Applications Ranking strategies based on SRs Click models based on SRs

Task Study of semantic relations of query terms Algorithm Query Segmentation, classification, statistics mining Difficulties Depends... Evaluation Depends... Expected results New problem in NLP and in IR