Download presentation

Presentation is loading. Please wait.

Published byLayne Medd Modified over 3 years ago

1
A Two-Dimensional Click Model for Query Auto-Completion Yanen Li 1, Anlei Dong 2, Hongning Wang 1, Hongbo Deng 2, Yi Chang 2, ChengXiang Zhai 1 1 University of Illinois at Urbana-Champaign 2 Yahoo Labs at Sunnyvale, CA at SIGIR 2014

2
2 QACDocument Retrieval Query:prefixquery Objects:querydocument Method:learning -to-rank Labels: user clicks only editor labels QAC vs. Document Retrieval KeystrokeSugg List Clicked Query Query Auto-Completion (QAC)

3
3 Only last column on current query log [Arias PersDB’08] [Bar-Yossef WWW’11] [Shokouhi SIGIR’13] use all simulated columns No work has used real QAC log Questions: Can we do better with real QAC log? What’s the best way of exploiting QAC log? Existing Work on Relevance Modeling for QAC

4
4 1. Keystroke2. Cursor Pos3. Sugg List4. Clicked Query 5. Previous Query 6. Timestamp 7. User ID Potential uses: -- improve QAC relevance ranking -- understand user behaviors in QAC … New QAC Log: From Real User Interaction at Yahoo!. High Resolution: Record Every Keystroke in Milliseconds

5
5 MethodMRR RankSVM – Last0.514 RankSVM – All0.436 Experiment on Yahoo! QAC log First attempt on exploiting QAC log

6
6 A closer look at QAC log: 2-Dimensional Click Distribution

7
7 Vertical Position PCiPhone 5 Vertical Position Bias Assumption A query on higher rank tends to attract more clicks regardless of its relevance to the prefix User behavior observation 1: vertical position bias

8
8 Should emphasize clicks at lower positions Implications for Relevance Ranking

9
9 happens in 60% of all sessions Horizontal Skipping Bias Assumption A query will receive no clicks if the user skips the suggested list of queries, regardless of the relevance of the query to the prefix User behavior observation 2: horizontal skipping (user skips relevant results)

10
10 Train on examined columns Implications for Relevance Ranking

11
P(C) = P(Relevance)∙P(Horizontal)∙P(Vertical) 11 better models of horizontal skipping bias and vertical position bias => better relevance model Our Goal: Develop a unified generative model to account for positional bias and horizontal skipping

12
Several click models -- UBM [Dupret SIGIR’08], -- DBN [Chapelle WWW’09], -- BSS [Wang WWW’13] No existing click model is suitable: 12 1. horizontal skipping behavior is not modeled 2. not content-aware. They can’t handle unseen prefix-query pairs (67.4% in PC and 60.5% in iPhone 5). Starting point: Existing Click Models for document retrieval

13
13 H Model: Horizontal Skipping BehaviorD Model: Vertical Position Bias D i = j: examine to depth j C Model: Relevance C i,j = 1: a click at position (i,j) New Model: Two-Dimensional Click Model (TDCM) H i =1: stop and examine H i =0: skip Features: Typing speed isWordBoundary Current position

14
14 H i =0 No click H i =1 D i =2 No click H i =1 D i =4 H i =1 D i =4 H i =1 D i =4 Click Only when examined and relevant, a click happens Disambiguate “no clicks”: Multiple scenarios Stop examine relevant clicked irrelevant Skip

15
15 E Step: evaluate the Q function by: M Step: maximize, while Solving the Model by E-M Algorithm

16
16 Data Random Bucket: shuffle query lists for each prefix; unbiased evaluation of R model with vertical position bias removed Metric MRR@All: average MRR across all columns Experiments: Data and Evaluation Metric

17
17 Comparison MethodDescription MPCMost Popular Completion UBM-last [Dupret SIGIR’08] User Browsing Model UBM-all [Dupret SIGIR’08] User Browsing Model DBN-last [Chapelle WWW’09] Dynamic Bayesian Network model DBN-all [Chapelle WWW’09] Dynamic Bayesian Network model BSS-last [Wang WWW’13] Bayesian Sequential State model BSS-all [Wang WWW’13] Bayesian Sequential State model TDCMOur model non content-aware modelsContent-aware models Experiments: Models Evaluated

18
18 MRR on Normal Bucket MethodPC MRR@All iPhone 5 MRR@All MPC0.4470.542 UBM-last0.4160.409 UBM-all0.4450.431 DBN-last0.4180.405 DBN-all0.4540.435 BSS-last0.515 ‡ 0.510 BSS-all0.4950.480 TDCM0.525 ‡ 0.580 ‡ Note: ‡ indicates p-value<0.05 compared to MPC MRR on Random Bucket (PC data only) MethodMRR@All MPC0.429 UBM-last0.381 UBM-all0.397 DBN-last0.373 DBN-all0.388 BSS-last0.471 ‡ BSS-all0.460 TDCM0.493 ‡ Results

19
19 Viewed columns: P(H i = 1) > 0.7 RankSVM Performance Validating the H Model: Using inferred p(H=1) to Enhance other Methods MRR@All

20
20 Feature Weights Learned by TDCM Understanding User Behavior via Feature Weights H Model: TypingSpeed is negatively proportional to p(H=1) IsWordBoundary is also important D Model: Top 3 positions occupy most of the examine probability R Model: QryHistFreq is important: user uses QAC as a memory GeoSense and TimeSense have valid contributions

21
Collect the first set of high-resolution query log specifically for QAC Analyze horizontal skipping bias and vertical position bias: implications for relevance modeling Propose a Two-Dimensional Click Model to model these user behaviors in a unified way, – Outperforming existing click models – Revealing interesting user behavior Future Work – More accurate component models (H, D, R) – Exploiting the model to character user groups (clustering users based on inferred model parameters) 21 Conclusions and Future Work

22
Questions? 22 Contact: Yanen Li University of Illinois at Urbana-Champaign yanenli2@illinois.edu A Two-Dimensional Click Model for Query Auto-completion

Similar presentations

OK

Squares and Square Root WALK. Solve each problem. 1. 2. REVIEW:

Squares and Square Root WALK. Solve each problem. 1. 2. REVIEW:

© 2018 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google

Ppt on uk economy Ppt on digital image processing using matlab Ppt on gujarati culture and traditions Ppt online open channel Ppt on hindu religion history Ppt on regional trade agreements in asia Ppt on abs system in bikes Download ppt on measures of dispersion Ppt on articles of association texas Difference between lcd and led display ppt online