1. Statistical Models for Web Search Click Log Analysis
Fan Guo Chao Liu Carnegie Mellon University Microsoft Research-Redmond

2. Prologue Search Results for “CIKM” # of clicks received 3/27/2017
CIKM'09 Tutorial, Hong Kong, China

3. Prologue Adapt ranking to user clicks? # of clicks received 3/27/2017
CIKM'09 Tutorial, Hong Kong, China

4. Prologue Tools needed for non-trivial cases # of clicks received
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

5. Motivation – Click Data Are Valuable
One of the most extensive (yet indirect) surveys of user experience.
For researchers:
Help understand human interaction with IR results
Design and calibrate novel models and hypotheses
For practitioners:
Measure, monitor and improve search engine performance.
Attract more page views and clicks, boost profit
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

6. Tutorial Goals
Introduce problems and applications in web search click modeling.
Present latest development of click models in web search.
Provide examples and discuss trade-offs for model design, implementation and evaluation.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

7. Presenters – Fan Guo
Ph.D. Student (exp. 2011), Computer Science Department, Carnegie Mellon University
Advisor: Christos Faloutsos
Dissertation topic: graph mining for large bioinformatics image databases
2008, M.S., CMU
2005, B.E., Tsinghua University, Beijing, China
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

8. Presenters – Chao Liu
Researcher, Internet Services Research Center (ISRC), MSR- Redmond.
Research focus: large-scale search/browsing log analysis for effective Web information access.
2007, Ph.D., UIUC 2005, M.S., UIUC
Advisor: Jiawei Han
Dissertation on statistical debugging and automated failure analysis
2003, B.S., Peking University, China
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

9. Outline Introduction Designing click models Bayesian click models
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

10. Outline Introduction Web search click logs
Interpret clicks as relevance feedback
Building statistical models for clicks
Applications of click models
Designing click models
Bayesian click models
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

11. Diverse User Feedbacks
Click-through
Browser action
Dwelling time
Explicit judgment
Other page elements
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

12. Web Search Click Log
Auto-generated data keeping important information about search activity.
Query
cikm
Session ID
f851c5af178384d12f3d
Position
URL
Click 1
cikm2008.org 2 3 4 5 6
cikmconference.org 7
Ir.iit.edu/cikm2004 8 9 10
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

13. Web search click log A real world example 3/27/2017
CIKM'09 Tutorial, Hong Kong, China

14. Web Search Click Log How large is the click log?
search logs: 10+ TB/day
In existing publications:
[Craswell+08]: 108k sessions
[Dupret+08] : 4.5M sessions (21 subsets * 216k sessions)
[Guo +09a] : 8.8M sessions from 110k unique queries
[Guo+09b]: 8.8M sessions from 110k unique queries
[Chapelle+09]: 58M sessions from 682k unique queries
[Liu+09a]: 0.26PB data from 103M unique queries
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

15. Web Search Click Log How large is one ? 3/27/2017
CIKM'09 Tutorial, Hong Kong, China

16. Outline Introduction Web search click logs
Interpret clicks as relevance feedback
Building statistical models for clicks
Applications of click models
Designing click models
Bayesian click models
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

17. Interpret Clicks: an Example
Clicks are good…
Are these two clicks equally “good”?
Non-clicks may have excuses:
Not relevant
Not examined
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

18. Eye-tracking User Study
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

19. Click Position-bias
Normal Position
Percentage
Higher positions receive more user attention (eye fixation) and clicks than lower positions.
This is true even in the extreme setting where the order of positions is reversed.
“Clicks are informative but biased”.
Reversed Impression
Percentage
[Joachims+07]
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

20. Clicks as Relative Judgments
“Clicked > Skipped Above” [Joachims02]
Preference pairs: #5>#2, #5>#3, #5>#4.
Use Rank SVM to optimize the retrieval function.
Limitation:
Confidence of judgments
Little implication to user modeling 1 2 3 4 5 6 7 8
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

21. Outline Introduction Web search click logs
Interpret clicks as relevance feedback
Building statistical models for clicks
Applications of click models
Designing click models
Bayesian click models
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

22. Problem Definition Given a set of web search click logs:
Predict clicks: output the probability of click vectors given a new order of URLs.
210 possibilities!
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

23. The Heart of Solution Given a set of web search click logs:
Estimate relevance: measures how good a URL is with regard to the information need of the query/user.
Relevance score = 0.5
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

24. Measuring Relevance
The probability of a click if the document appears at the top position.
Relevance score = 0.5 indicates that on average, the document will be clicked once per 2 sessions.
Bayesian click models characterize relevance using a probability distribution
Relevance score
Density function
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

25. Desired Properties
Effective: aware of the position-bias and address it properly
Scalable: linear complexity for both time and space, easy to parallel
Incremental: flexible for model update based on new data
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

26. Outline Introduction Web search click logs
Interpret clicks as relevance feedback
Building statistical models for clicks
Applications of click models
Designing click models
Bayesian click models
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

27. Applications of click models
Optimizing the retrieval function
Ranking alternation based on clicks [Liu+09b]
0.90
0.10
0.08
0.05
0.20
0.72
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

28. Applications of click models
Optimizing the retrieval function
Ranking alternation based on clicks
As a feature to a learning-to-rank system (e.g., RankNet [Burges+05] )
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

29. Applications of click models
Online advertising
User model for sponsored search auctions
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

30. Applications of click models
Online advertising
User model for sponsored search auctions
Click through rate (CTR) prediction [Zhu+10]
The image shows the CTR of ads on a commercial search engine for a typical day (Image by courtesy of Zeyuan Zhu)
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

31. Applications of click models
Search engine evaluation
Pskip [Wang+09]: click-through-rate above last clicks; dwelling time features could also be incorporated.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

32. Applications of click models
Search engine evaluation
Pskip [Wang+09]: click-through-rate above last clicks;
Search relevance score [Guo+09c]: average relevance score weighted by chance of examination
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

33. Applications of click models
User behavior analysis
A preliminary work showing different user behavior patterns for navigational and informational queries [Guo+09c]
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

34. Outline Designing click models Basic user hypotheses
Introduction
Designing click models
Basic user hypotheses
Modeling the first click
Extending to multiple clicks
Summary of model design
Bayesian click models
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

35. Examination Hypothesis [Richardson+07]
A document must be examined before a click.
The (conditional) probability of click upon examination depends on document relevance.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

36. Examination Hypothesis [Richardson+07]
The click probability could be decomposed:
Global component: the examination probability which reflects the position-bias
Local component: depends on the (query, URL) pair only
The building block for every existing model!
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

37. Cascade Hypothesis [Craswell+08]
The first document is always examined.
First-order Markov property:
Examination at position (i+1) depends on examination and click at position i only
Examination follows a strict linear order:
Position i
Position (i+1)
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

38. Cascade Hypothesis [Craswell+08]
The first document is always examined.
First-order Markov property:
Examination at position (i+1) depends on examination and click at position i only
Examination follows a strict linear order:
Position i
Position (i+1)
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

39. Cascade Hypothesis [Craswell+08]
Limitation: examination/click rate monotonically decreases with rank, which is not always true.
Some models do not follow this hypothesis (e.g., UBM)
Web search data in [Guo+09a]
Ads click data in [Zhu+10]
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

40. Outline Designing click models Basic user hypotheses
Introduction
Designing click models
Basic user hypotheses
Modeling the first click
Extending to multiple clicks
Summary of model design
Bayesian click models
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

41. Cascade model Put together two hypotheses:
Formal model specification:
P(Ci=1|Ei=0) = 0, P(Ci=1|Ei=1) = rui
P(E1=1) =1, P(Ei+1=1|Ei=0) = 0
P(Ei+1=1|Ei=1, Ci=0)=1
Cascade Model = [Craswell+08]
examination hypothesis
cascade hypothesis
modeling a single click
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

42. Cascade model The user behavior chart: Done Examine the URL Click? No
See Next URL?
Yes
Yes
Done
Index for URL at position i
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

43. The chance that user may immediately abandon examination w/o a click.
Alternatives
First click in Click Chain Model [Guo+09b] as well as Dynamic Bayesian Network model [Chapelle+09]
Examine the URL
Click? No
See Next URL?
Yes
Yes No
The chance that user may immediately abandon examination w/o a click.
Done
Done
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

44. Position-dependent parameters
Alternatives
First click in User Browsing Model [Dupret+08]
Examine the URL
Click? No
See Next URL?
Yes
No i ←i+1
Yes
Done
Position-dependent parameters
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

45. Outline Designing click models Basic user hypotheses
Introduction
Designing click models
Basic user hypotheses
Modeling the first click
Extending to multiple clicks
Summary of model design
Bayesian click models
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

46. Dependent Click Model [Guo+09a]
Generalize the cascade model to 1+ clicks:
P(Ci=1|Ei=0) = 0, P(Ci=1|Ei=1) = rui
P(E1=1) =1, P(Ei+1=1|Ei=0) = 0
P(Ei+1=1|Ei=1, Ci=0)=1
P(Ei+1=1|Ei=1, Ci=1)= λi
λ:global parameters characterizing user browsing behavior
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

47. Dependent Click Model [Guo+09a]
Generalize the cascade model to 1+ clicks:
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

48. Dependent Click Model [Guo+09a]
DCM Algorithms:
Input: for each query session, the query term, with (URL, clicked) tuple for all top-10 positions.
Output: relevance for each (query, URL) pair; global parameters for user behavior
Method: approximate* maximum-likelihood estimation.
*Footnote: the algorithm maximizes a lower bound of log-likelihood function.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

49. Detour: last clicked position
Query
cikm
Session ID
f851c5af178384d12f3d
Position
URL
Click 1
cikm2008.org 2 3 4 5 6
cikmconference.org 7
Ir.iit.edu/cikm2004 8 9 10
Last clicked position
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

50. Detour: last clicked position
Query
cikm
Session ID
ab8dee4c4dd21e6aaf03
Position
URL
Click 1
cikm2008.org 2 3 4 5
cikmconference.org 6 7
Ir.iit.edu/cikm2004 8 9 10
Last clicked position
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

51. DCM Algorithms [Guo+09a]
The estimation formula for relevance:
empirical CTR measured before last clicked position
The estimation formula for global (user behavior) parameters:
empirical probability of “clicked-but-not-last”
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

52. DCM Implementation [Guo+09a]
Details
DCM Implementation [Guo+09a]
Keep 3 counts for each (query, URL) pair
Then
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

53. Click Chain Model [Guo+09b]
The examine-next probability depends on the relevance of the URL clicked:
Not what I want, go to examine the next
Aha, this is the right one, and I’m done!
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

54. Click Chain Model [Guo+09b]
The examine-next probability depends on the relevance of the URL clicked:
P(Ei+1=1|Ei=1, Ci=1)= α2(1-rui) + α3rui
P(Ei+1=1|Ei=1, Ci=0)= α1 where 0 < α1 ≤ 1, 0 ≤ α3< α2≤ 1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

55. Click Chain Model [Guo+09b]
The full picture:
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

56. DBN Model [Chapelle+09] Conclusion: attractive, but not satisfactory.
There is a subtle difference between the relevance of the URL snippet and the landing page.
hmmm…, this looks pretty nice
errr…, it’s way out of date
Conclusion: attractive, but not satisfactory.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

57. DBN Model [Chapelle+09]
The examine-next probability depends on the “satisfaction score”:
P(Ei+1=1|Ei=1, Ci=1)= γ(1-sui) + 0sui
P(Ei+1=1|Ei=1, Ci=0)= γ where 0 < γ ≤1
The click probability is associated with “attractiveness score”:
P(Ci=1|Ei=1)= aui
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

58. DBN Model [Chapelle+09] The full picture: 3/27/2017
CIKM'09 Tutorial, Hong Kong, China

59. User Browsing Model [Dupret+08]
The examine-next probability depends on both the preceding clicked position r, and the distance to this position d.
Position
URL
Click 1
cikm2008.org 2 3 4 5
cikmconference.org 6 …
r = 0
d = 1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

60. User Browsing Model [Dupret+08]
The examine-next probability depends on both the preceding clicked position r, and the distance to this position d.
Position
URL
Click 1
cikm2008.org 2 3 4 5
cikmconference.org 6 …
r = 0
d = 2
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

61. User Browsing Model [Dupret+08]
The examine-next probability depends on both the preceding clicked position r, and the distance to this position d.
Position
URL
Click 1
cikm2008.org 2 3 4 5
cikmconference.org 6 …
r = 2
d = 1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

62. User Browsing Model [Dupret+08]
The examine-next probability depends on both the preceding clicked position r, and the distance to this position d.
Position
URL
Click 1
cikm2008.org 2 3 4 5
cikmconference.org 6 …
r = 2
d = 2
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

63. User Browsing Model [Dupret+08]
The examine-next probability depends on both the preceding clicked position r, and the distance to this position d.
Position
URL
Click 1
cikm2008.org 2 3 4 5
cikmconference.org 6 …
r = 2
d = 3
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

64. User Browsing Model [Dupret+08]
The examine-next probability depends on both the preceding clicked position r, and the distance to this position d.
Users would lose patience when they browse through without issuing a click.
The probability monotonically drops as d increases and r remains the same.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

65. User Browsing Model [Dupret+08]
The examine-next probability depends on both the preceding clicked position r, and the distance to this position d.
P(Ei=1|C1:i-1)= βri,di
55 parameters are needed for top-10 positions (0≤r<r+d≤10).
Cascade hypothesis is not assumed.
where ri = max{j| j <i , Cj=1}, di = i - ri
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

66. User Browsing Model [Dupret+08]
The full picture:
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

67. Outline Designing click models Basic user hypotheses
Introduction
Designing click models
Basic user hypotheses
Modeling the first click
Extending to multiple clicks
Summary of model design
Bayesian click models
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

68. Summary of Model Design
Probability of examine the first URL
Model
P(E1)
Cascade 1
DCM
CCM 1*
DBN
UBM
β0,1
* Footnote: it is flexible to add another parameter to specify this probability.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

69. Summary of Model Design
Probability of click upon examination
Model
P(Ci=1|Ei=1)
Cascade
rdi
DCM
CCM
rdi*
DBN
adi
UBM
*Footnote: the mean of the relevance distribution, detailed in the next part
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

70. Summary of Model Design
Probability of examine-next w/o a click
Model
P(Ei+1=1|Ei=1,Ci=0)
Cascade 1
DCM
CCM α1
DBN γ
UBM
βri+1,di+1 *
*Footnote: the probability does not depend on Ei
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

71. Summary of Model Design
Probability of examine-next after a click
Model
P(Ei+1=1|Ei=1,Ci=1)
Cascade --
DCM αi
CCM
α2(1-rdi) + α3rdi
DBN
γ(1-sdi)
UBM
βi,1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

72. Summary of Model Design
Probability of examine-next after a click
Model
P(Ei+1=1|Ei=1,Ci=1)
Cascade --
DCM αi
CCM
α2(1-rdi) + α3rdi
DBN
γ(1-sdi)
UBM
βi,1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

73. Summary of Model Design
Size of parameter sets
Model
# of global params
Cascade
DCM 9
CCM 3
DBN 1
UBM 55
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

74. Summary of Model Design
Inference and estimation algorithms
Model
Single-Pass
Details
DCM
Maximizing a lower bound of LL, fastest
CCM
No iteration needed, thanks to the Bayesian framework
DBN
EM-based, iterative algorithms
UBM
EM-based, usually takes ~30 iterations to converge
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

75. Summary of Model Design
Inference and estimation algorithms
Model
Single-Pass
Details
DCM
Maximizing a lower bound of LL, fastest
CCM
No iteration needed, thanks to the Bayesian framework
DBN
EM-based, iterative algorithms
UBM
EM-based, usually takes ~30 iterations to converge
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

76. Outline Bayesian click models Bayesian framework and the rationale
Introduction
Designing click models
Bayesian click models
Bayesian framework and the rationale
Bayesian Browsing Model: a case study
Click Chain Model in a nutshell
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

77. Primer: Coin-Toss Example1
Prior
Posterior
“probability” of p(H)
p(H)=0.8
p(H)
p(H)
Bayesian
Frequentist
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

78. Primer: Coin-Toss Example
Posterior
Prior
Density Function (not normalized)
x x x x3(1-x) x4(1-x)
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

79. Primer: Coin-Toss Example
Posterior
Prior
Density Function (not normalized)
x1(1-x) x2(1-x) x3(1-x) x3(1-x) x4(1-x)1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

80. Primer: Coin-Toss Example
The graphical model for coin-toss X C1 C2 C3 C4 C5
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

81. Primer: Coin-Toss Example
The graphical model for coin-toss X C1 C2 C3 C4 C5
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

82. Primer: Click Data Example
x1 (1-x)0 (1-0.6x)0 (1+0.3x)1 (1-0.5x)0 (1-0.2x)0 …
x1 (1-x)1 (1-0.6x)0 (1+0.3x)1 (1-0.5x)0 (1-0.2x)0 …
x2 (1-x)1 (1-0.6x)0 (1+0.3x)2 (1-0.5x)0 (1-0.2x)0 …
x3 (1-x)1 (1-0.6x)1 (1+0.3x)2 (1-0.5x)0 (1-0.2x)0 …
x3 (1-x)1 (1-0.6x)1 (1+0.3x)2 (1-0.5x)1 (1-0.2x)0 …
Prior
Density Function (not normalized)
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

83. Overview Representation of relevance
A probability distribution on [0,1] for each (query, URL) pair
The density function is in a polynomial form over a small set of linear factors.
The coefficients of such linear factors are shared between different (query, URL) pairs.
x3 (1-1x)1 (1-0.6x)1 (1+0.3x)2 (1-0.5x)1 (1-0.2x)0 …
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

84. Overview
Inference:
Go over each query session once, update the exponents for corresponding (query, URL) pair impressed*
Analytical or numerical integration may be needed to compute the normalization constant.
*Footnote: by virtue of the Bayes theorem and conditional independence relationship/assumption
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

85. Overview Key problems: Which is the right factor to update?
How to estimate all the coefficients?
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

86. Why Bayesian? Modeling Benefits: Computational Benefits:
Confidence for the URL relevance estimate
Relative judgments: probability of URL i is more relevant to the query than URL j
Easy to interpret: coefficients in linear factors reflect position-bias and user browsing patterns
Computational Benefits:
Single-pass, linear algorithms; no iterations
Paralleled version is easy to implement
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

87. Outline Bayesian click models Bayesian framework and the rationale
Introduction
Designing click models
Bayesian click models
Bayesian framework and the rationale
Bayesian Browsing Model: a case study
Click Chain Model in a nutshell
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

88. Variable Definition For a specific query session, let
where 1 ≤ i ≤ M=10. S1 S2 S3 SM … E1 E2 E3 EM … C1 C2 C3 CM …
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

89. Graphical Illustration
Relevance S1 S2 S3 SM …
Examination E1 E2 E3 EM …
Click C1 C2 C3 CM …
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

90. Inference Algorithms Compute the posterior distribution
Details
Inference Algorithms
Compute the posterior distribution
Conditional independence relationship induced from the graphical model
How many times URLj was not clicked when it is at position (r + d) with the preceding click at position r
How many times the URL j was clicked
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

91. A Toy Example
Only top M=3 positions are shown, 3 query sessions and 4 distinct URLs. 4 1 3 2
Position
Query Session 3
Query Session 2
Query Session 1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

92. A Toy Example Initialize M(M+1)/2+1 counts for each URL 4 Clicks
r=0 d=1
r=0 d=2
r=0 d=3
r=1 d=1
r=1 d=2
r=2 d=1 4
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

93. A Toy Example Update counts for URL 4 If not impressed, do nothing;
If clicked, increment “clicks” by 1;
Otherwise, locate the right r and d to increment.
URL
Clicks
r=0 d=1
r=0 d=2
r=0 d=3
r=1 d=1
r=1 d=2
r=2 d=1 4
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

94. A Toy Example Update counts for URL 4 If not impressed, do nothing;
If clicked, increment “clicks” by 1;
Otherwise, locate the right r and d to increment.
URL
Clicks
r=0 d=1
r=0 d=2
r=0 d=3
r=1 d=1
r=1 d=2
r=2 d=1 4 1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

95. A Toy Example Update counts for URL 4 If not impressed, do nothing;
If clicked, increment “clicks” by 1;
Otherwise, locate the right r and d to increment.
URL
Clicks
r=0 d=1
r=0 d=2
r=0 d=3
r=1 d=1
r=1 d=2
r=2 d=1 4 1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

96. A Toy Example The posterior for URL 4 Interpretation:
The larger the probability of examination, the stronger the penalty for a non-click.
URL
Clicks
r=0 d=1
r=0 d=2
r=0 d=3
r=1 d=1
r=1 d=2
r=2 d=1 4 1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

97. Parameter Estimation
Keep 2 counts for each parameter (one for click, and the other one for non-click)
Parameter
Click
Non-click
Non-Click
β0,1
β1,1
β0,2
β1,2
β0,3
β2,1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

98. Parameter Estimation
For each position in a query session, locate the right r and d to increment.
Parameter
Click
Non-click
Non-Click
β0,1 1
β1,1
β0,2
β1,2
β0,3
β2,1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

99. Parameter Estimation
For each position in a query session, locate the right r and d to increment.
Parameter
Click
Non-click
Non-Click
β0,1 1
β1,1
β0,2
β1,2
β0,3
β2,1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

100. Parameter Estimation
For each position in a query session, locate the right r and d to increment.
Parameter
Click
Non-click
Non-Click
β0,1 1 2
β1,1
β0,2
β1,2
β0,3
β2,1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

101. Parameter Estimation Maximum-Likelihood Estimate: β0,1 1 2 β1,1 β0,2
Click
Non-click
Non-Click
β0,1 1 2
β1,1
β0,2
β1,2
β0,3
β2,1
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

102. Algorithm Complexities
Details
Algorithm Complexities
Let
Initializing and updating the counts:
Time: Space:
Linear to the size of the click log
Almost constant storage required
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

103. Algorithm Complexities
Details
Algorithm Complexities
Let
Initializing and updating the counts:
Time: Space:
Computing relevance scores using numerical integration with B bins:
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

104. Summary of Algorithms Step 1: initialize counting statistics;
Step 2: scan through the click log once and update the counts for both inference and estimation
Step 3: compute parameter values;
Step 4: use numerical integration to obtain relevance scores.
Step 2 also applies for (linear) incremental computation!
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

105. Outline Bayesian click models Bayesian framework and the rationale
Introduction
Designing click models
Bayesian click models
Bayesian framework and the rationale
Bayesian Browsing Model: a case study
Click Chain Model in a nutshell
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

106. CCM Recap The user behavior model: 3/27/2017
CIKM'09 Tutorial, Hong Kong, China

107. CCM Recap Graphical model: Relevance … Examination Click S1 S2 S3 E1SM … E1 E2 E3 EM C1 C2 C3 CM
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

108. Details
CCM Inference
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

109. Comparing with UBM Number of user behavior parameters
Number of distinct factors for (query, URL)
Number of counts needed for parameters
CCM
UBM 3 55
CCM
UBM 22 56
CCM
UBM 5
110
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

110. Outline Selected topics on click models
Introduction
Designing click models
Bayesian click models
Selected topics on click models
Scaling click models for Petabyte-scale data
Click model evaluation
Tailoring user goals to click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

111. Petabyte-Scale Data Data collected in 8 weeks
Job k includes data between week 1 and k
Both time and space costs are prohibitive for a single node.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

112. Introducing Map-Reduce [Dean+04]
A Simple Task: counting # impression for each (query, URL) pair
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

113. Introducing Map-Reduce [Dean+04]
Machine #1
Machine #2
Machine #3
Machine #4
Extent
Extent
Extent
Extent
GetPairs
GetPairs
GetPairs
GetPairs
Map
Map
Map
Map
Sort
Sort
Sort
Sort
Count
Count
Count
Count
Output

114. Introducing Map-Reduce [Dean+04]
Machine #1
Machine #2
Machine #3
Machine #4
Extent
Extent
Extent
Extent
GetPairs
GetPairs
GetPairs
GetPairs
Map
Map
Map
Map
Sort
“Map” puts all of the same Pairs onto one machine. This allows you to group by various fields in subsequent processes.
Sort
Sort
Sort
Count
Count
Count
Count
Output

115. Introducing Map-Reduce [Dean+04]
A Simple Task: counting # impression for each (query, URL) pair
Map = Bucket: the intermediate key is (query, URL) pair
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

116. Introducing Map-Reduce [Dean+04]
Machine #1
Machine #2
Machine #3
Machine #4
Extent
Extent
Extent
Extent
GetPairs
GetPairs
GetPairs
GetPairs
“Count” carries out standard increment-by-1 over each distinct Pair.
Map
Map
Map
Map
“Count” REDUCES the amount of data since each Pair has only one output value
Sort
Sort
Sort
Sort
Count
Count
Count
Count
Output

117. Introducing Map-Reduce [Dean+04]
A Simple Task: counting # impression for each (query, URL) pair
Map = Bucket: the intermediate key is (query, URL) pair
Reduce = Count: it accepts a list of (key, value) tuple, and outputs the final result for each distinct key
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

118. Introducing Map-Reduce [Dean+04]
Machine #1
Machine #2
Machine #3
Machine #4
Extent
Extent
Extent
Extent
GetPairs
GetPairs
GetPairs
GetPairs
MAP
Map
Map
Map
Map
Sort
Sort
Sort
Sort
REDUCE
Count
Count
Count
Count
Output

119. MapReduce for BBM 0 for clicks 3 2 5 1 4 6 3/27/2017
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

120. MapReduce for BBM Map: scan the click log
Intermediate key: (query, URL)
Value: the index of linear factors (0~55 for top-10 positions)
Reduce: scan the list of (key, value)
The key indicates which exponent vector to update
The value indicates the index of the element in the exponent vector to increment
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

121. Empirical Results Linearly increasing computation load
Near-constant elapsed time
3 hours
265 TB log data
1.15 billion (query, url) pairs
Single machine computation load
Elapse time on SCOPE
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

122. Outline Selected topics on click models
Introduction
Designing click models
Bayesian click models
Selected topics on click models
Scaling click models for Petabyte-scale data
Click model evaluation
Tailoring user goals to click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

123. Evaluation Overview - Training
Impression Data
Click Data
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

124. Evaluation Overview - Training
Impression Data
Click Data
Global Parameters
M=10
Relevance Scores
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

125. Evaluation Overview - Test
New Impression Vector from an Existing Query
Relevance
Predicted Examination
Predicted Clicks
Global params
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

126. Data Preprocessing [Guo+09a, 09b]
Data are collected from a commercial search engine after query term normalization and spam removal.
For each query term, split query sessions evenly into training and test sets according to the timestamp.
Top frequent/infrequent query terms are removed.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

127. Evaluation Metrics Most popular metrics:
Average test data log-likelihood (LL) (probability of accurately predicting the click vector, 2^10 possibilities) [Guo+09a, Guo+09b, Liu+09a, Zhu+10]
Perplexity of prediction for each position (2^{average entropy} of click/no-click binary prediction for each position independently) [Dupret+08, Guo+09a, Guo+09b, Zhu+10]
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

128. Evaluation Metrics Other Metrics:
Click-through-rate (CTR) prediction (Especially for predicting [Chapelle+09, Zhu+10]
Predicting first/last clicked positions [Guo+09a, Guo+09b]
Position-bias sanity check (plot the click rate curve for top-10 positions v.s. the ground truth) [Guo+09a, Guo+09b]
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

129. Results – Log Likelihood [Guo+09b]
Average Log-likelihood
Random guess: log(2-10) = -3.01
Optimal value: 0
Model
CCM
UBM
DCM LL
-1.171
-1.264
-1.302
Improve-ment Ratio
9.7%
14%
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

130. Results – Log Likelihood [Guo+09b]
Better
Worse
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

131. Results – Log Likelihood [Liu+09a]
Better
Worse
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

132. Results – Perplexity [Guo+09b]
Average Perplexity over top 10 positions
Random guess: 2
Optimal value: 1
Model
CCM
UBM
DCM
Perplexity
1.1577
1.1590
Improve-ment Ratio
7.5%
8.3%
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

133. Results – Perplexity [Guo+09b]
Worse
Better
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

134. Examine/Click Position-Bias [Guo+09b]
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

135. Results – Efficiency
For 1M query sessions, the estimated time in seconds:
* Time for CCM and BBM includes computing posterior mean and variance using numerical integration w/ 100 bins.
** UBM converges in 34 iterations.
DCM
CCM*
BBM*
UBM** 80
150
165
5,000
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

136. Outline Selected topics on click models
Introduction
Designing click models
Bayesian click models
Selected topics on click models
Scaling click models for Petabyte-scale data
Click model evaluation
Tailoring user goals to click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

137. User Goals in Web Search
Queries could be categorized into 2 sets:
Navigational: to find the link to an existing website, e.g., bing;
Informational: more exploration, multiple clicks may arise, e.g., iron man.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

138. Fitting Multiple Click Models
Different user goals result in different browsing and click patterns.
The straightforward mixture-modeling approach is not practical. [Dupret+08]
Solution:
Classify query terms a priori based on user goals.
Fitting and learning 2 sets of model parameters for navigational and informational queries.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

139. Determining User Goals [Lee+05]
Two-way classification for query terms based on click data using…
Median position of click distribution
Mean position of click distribution
Average # clicks per query session …
Pick the one which has best click prediction
If a position receives 50% of the click, then navigational, else informational
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

140. Empirical Results [Guo+09c]
Improvement of click prediction for DCM:
Log-Likelihood: 4.0%
Perplexity: 1.3%
Examination/Click position-bias:
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

141. Outline Introduction Designing click models Bayesian click models
Selected topics on click models
Conclusion
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

142. Summary
Click models
A statistical tool to leverage valuable user implicit feedback in terabyte/petabyte search logs.
Provide click prediction as well as relevance estimates.
Application domains include learning to rank, measuring search performance, online advertising, user behavior analysis…
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

143. Summary
Click models
Different model designs reflect various assumption of user behaviors to explain the position-bias.
The modeling choice may depend on the application scenario.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

144. Summary
Click models
Efficient, single-pass, parallelizable algorithms are desired in real-world applications.
Bayesian framework could be applied to click models for both modeling benefits and computational benefits.
Click Chain Model and Bayesian Browsing Model represent state-of-the-art examples.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

145. Future Directions Bigger Context Richer inputs
Query reformulations
Personalization
Richer inputs
Universal search
Diverse user feedback
Click model v.s. Human judgments
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

146. References
[Burges+05]: C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to rank using gradient descent. ICML’05.
[Chapelle+09]: O. Chapelle and Y. Zhang. A dynamic Bayesian network click model for web search ranking. WWW’09.
[Craswell+08]: N. Craswell, O. Zoeter, M. Taylor, and B. Ramsey. An experimental comparison of click position-bias models. WSDM ’08.
[Dean+04]: J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. OSDI’04.
[Dupret+08]: G. Dupret and B. Piwowarski. A user browsing model to predict search engine click data from past observations. SIGIR’08.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

147. References
[Guo+09a]: F. Guo, C. Liu, and Y.-M. Wang. Efficient multiple-click models in web search. WSDM’09.
[Guo+09b]: F. Guo, C. Liu, A. Kannan, T. Minka, M. Taylor, Y.-M. Wang, and C. Faloutsos. Click chain model in web search. WWW’09.
[Guo+09c]: F. Guo, L. Li, and C. Faloutsos. Tailoring click models to user goals. WSCD’09.
[Joachims02]: T. Joachims. Optimizing search engines using clickthrough data. KDD’02.
[Joachims+07]: T. Joachims, L. Granka, B. Pan, H. Hembrooke, F. Radlinski, and G. Gay. Accurately interpreting clickthrough data as implicit feedback, ACM TOIS, 25(2), 2007.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

148. References
[Lee+05]: U. Lee, Z. Liu, and J. Cho. Automatic identification ofuser goals in web search. WWW’05.
[Liu+09a]: C. Liu, F. Guo, and C. Faloutsos. BBM: Deriving click models from petabyte-scale data. KDD’09.
[Liu+09b]: C. Liu, M. Li, and Y.-M. Wang. Post-rank reordering: resolving preference misalignments between search engines and end users. CIKM’09.
[Richardson+07]: M. Richardson, E. Dominowska, and R. Ragno. Predicting clicks: estimating the click-through rate for new ads. WWW’07.
[Zhu+10]: Z. Zhu, W. Chen, T. Minka, C. Zhu and Z. Chen. A novel click model and its applications to online advertising. To appear in WSDM’10.
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

149. Acknowledgements Nick Craswell Christos Faloutsos Li-Wei He Tom Minka
MSR, Cambridge
Carnegie Mellon University
MSR, ISRC-Redmond
Tom Minka
Anitha Kannan
MSR, Cambridge
MSR, Search Lab
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

150. Acknowledgements Yi-Min Wang Mike Taylor Ethan Tu MSR, ISRC-Redmond
MSR, Cambridge
MSR, ISRC-Redmond
3/27/2017
CIKM'09 Tutorial, Hong Kong, China

151. End
3/27/2017
CIKM'09 Tutorial, Hong Kong, China