1. Notices Homework 2 due in class May 21 Uncollected Homework 1
Coursera class on Machine Learning by Andrew Ng (Stanford):
(particularly video lectures from Week 3+, Logistic Regression)
Coursera class on Data Science by Bill Howe (UW):
Predict y in {1,2,3,4,5} ... use SVR (SVM in regression mode)
No lecture next Tuesday ... continue to work on projects
(No office hours Monday)
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

2. Re-cap on Page Rank
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

3. Matrix-Vector form Recall rj = importance of node j
rj = Si wij ri i,j = 1,….n
e.g., r2 = 1 r r r r4
= dot product of r vector with column 2 of W
Let r = n x 1 vector of importance values for the n nodes
Let W = n x n matrix of link weights
=> we can rewrite the importance equations as
r = WT r

4. Eigenvector Formulation
Need to solve the importance equations for unknown r, with known W
r = WT r
We recognize this as a standard eigenvalue problem, i.e.,
A r = l r (where A = WT)
with l = an eigenvalue = 1
and r = the eigenvector corresponding to l = 1

5. Eigenvector Formulation
Need to solve for r in
(WT – l I) r = 0
Note: W is a stochastic matrix, i.e., rows are non-negative and sum to 1
Results from linear algebra tell us that:
(a) Since W is a stochastic matrix, W and WT have the same eigenvectors/eigenvalues
(b) The largest of these eigenvalues l is always 1
(c) the vector r corresponds to the eigenvector corresponding to the largest eigenvector of W (or WT)

6. Solution for the Simple Example
Solving for the eigenvector of W we get
r = [ ]
Results are quite intuitive, e.g., 2 is “most important” 1 1 2 3
0.5
0.5
W =
0.5
0.5
0.5 1
0.5
0.5 4

7. Complexity of Page Rank
Complexity of eigenvector problem
Ax = lx
(A is an n*n matrix)
Complexity of fixed point update
x  A x
For the web, A is 10^9 x 10^9
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

8. CS 277: Data Mining Analysis of Web User Data
Padhraic Smyth
Department of Computer Science
University of California, Irvine

9. Data Sources for Web Mining
Web content
Text and HTML content on Web pages, e.g., categorization of content
Web connectivity
Hyperlink/directed-graph structure of the Web
e.g., using PageRank to infer importance of Web pages
e.g., using links to improve accuracy in classification of Web pages
Web user data (focus of this set of slides)
Data on how users interact with the Web
Navigation data, aka “clickstream” data
Search query data (keywords for users)
Online transaction data (e.g., purchases at an ecommerce store)
Volume of data?
Large portals (e.g., Yahoo!, MSN) report 100’s of millions of users per month
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

10. Introduction to Web Mining
Useful to study user behavior on the Web
e.g. search engine data can be used for
Exploration: e.g. # of queries per session?
Modeling: e.g. time of day dependence?
Prediction: e.g. which pages are relevant?
Applications
Understand social implications of Web usage
Design of better tools for information access
E-commerce applications
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

11. “Computational Advertising”
Revenue of many internet companies is driven by advertising
Key problem:
Given user data:
Pages browsed
Keywords used in search
Demographics
Determine the most relevant ads (in real-time)
About 50% of keyword searches can not be matched effectively to any ads
(other aspects include bidding/pricing of ads)
Another major problem: “click fraud”
Algorithms that can automatically detect when online advertisements are being manipulated (this is a major problem for Internet advertising)
New research area of “computational advertising”
See link to Stanford class by Andrei Broder on class Web site
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

12. Networks of Instant Messagers Leskovec and Horvitz, 2007
Network Data
240 IM users over 1 month
1 billion conversations per day
1.3 billion edges in the graph

13. Linking Demograpics with IM Usage from Leskovec and Horvitz, 2007

14. Login and Logout Durations for Instant Messenger Sessions
Login and Logout Durations for Instant Messenger Sessions from Leskovec and Horvitz, 2007
Regularities such as power-laws are abundant in Web data
Highly non-Gaussian
Aggregate behavior – very predictable
Individual behavior – much less so

15. Query Data (source: Dan Russell, Google)
Research Question:
Predict the age and gender of an individual given their query history
More difficult:
Predict how many people are using one account, and their ages and genders

16. Eye-Tracking: The Golden Triangle for Search
Eye-Tracking: The Golden Triangle for Search from Hotchkiss, Alston, Edwards, 2005; EnquiroResearch
Research Question:
If a user clicks on the 5th item on the list should we treat items 1 to 4 as “negatives” for learning?

17. Number of Terms per Query
(from Xie and O Halloran, 2002)
For both search engine, Distribution of query lengths is qualitatively similar
- Observed query Length Distributions (bars)
- Poisson Model (lines)
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

18. Power-law Characteristics of Common Queries
(from Xie and O Halloran, 2002)
Approximately Power-Law
Frequency f(r) of Queries with Rank r
queries from Vivisimo
1.9 Million queries from Excite
Presence of many rare queries makes prediction/ad-matching difficult
The easiest way to see a power law is to plot the data in log/log space. If there is a power law relationship, you get a straight line
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

19. Sample of 17 Million Unique Queries from eBay
From N. Parikh and N. Sundaresan
Inferring Semantic Query Relations from Collective User Behavior
Proceedings of CIKM, 2008
Log
Query
Frequency
Log Rank of Query in Sample
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

20. From: M. Richardson, Microsoft Research
Learning about the World through Long-Term Query Logs
ACM Transactions on the Web, October 2008
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

21. Case Study: Detecting Flu Outbreaks from Search Engine Data
J. Ginsberg, M. Mohebbi, R. Patel, L. Brammer, M. Smolinski, L. Brilliant
Detecting influenza epidemics using search engine query data
Nature, Februrary 2009
google for “google flu trends”
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

22. Detecting Flu Outbreaks
Problem:
Influenza epidemics cause 250k to 500k deaths per year (worldwide)
New strains can emerge quickly and be very dangerous
A key factor in reducing risk is quick detection of an outbreak
How are flu outbreaks currently detected?
Center for Disease Control (CDC) gathers counts of “influenza-like illness” (ILI) physician visits
Surveillance data published nationally and regionally, weekly basis
Problem: 1 to 2 week reporting lag
Other approaches
Monitor over-the-counter flu medication sales
Monitor calls to health advice lines
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

23. Using Search Queries Idea: Data:
Discover influenza related search queries and use these to predict ILI counts
Motivation: should be much faster than 1-2 week lag of CDC data
Data:
Look at all search queries in Google from 2003 to 2008
Several hundred billion individual searches in the United States
Keep track of only the 50 million most common queries
Keep a weekly count for each query
Also keep counts of each query by geographic region
(requires use of geo-location from IP addresses: >95% accurate)
So counts for 50 million queries x 170 weeks x 9 regions
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

24. Building a Predictive Model
Target variable to be predicted:
For each week, for each region
I(t) = percentage physician visits that are ILI (as compiled by CDC)
Input variable:
Q(t) = sum of top n highest correlated queries / total number of queries that week
“Model learning”:
log( I(t) / [1 – I(t)] ) = a log ( Q(t)/ [1 – Q(t) ] ) + noise
(This is really just calibrating log-odds Q(t) to predict log-odds I(t))
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

25. Addition of terms that generally correlate
with flu season, but not with specific outbreaks,
e.g., “high school basketball”
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

26. Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

27. Evaluating the Model Model fit to weekly data between 2003 and 2007
128 weeks
Predictions made on 42 weeks of unseen test data in
9 regions
Correlations per region of predicted ILI with actual ILI
Mean = 0.97
Min = 0.92
Max = 0.99
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

28. Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

29. Key point in these graphs is that the CDC data was lagging
Google predictions by
1 to 2 weeks
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

30. Measuring Web User data
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

31. How our Web navigation is recorded…
Web logs
Record activity between client browser and a specific Web server
Easily available
Can be augmented with cookies (provide notion of “state”)
Search engine records
Text in queries, which pages were viewed, which snippets were clicked on, etc
Client-side browsing records
Automatically recorded by client-side software
Harder to obtain, but much more accurate than server-side logs
Other sources
Web site registration, purchases, , etc
ISP recording of Web browsing
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

32. Web Server Log Files Server Transfer Log: Referrer Log: Agent Log:
transactions between a browser and server are logged
IP address, the time of the request
Method of the request (GET, HEAD, POST…)
Status code, a response from the server
Size in byte of the transaction
Referrer Log:
where the request originated
Agent Log:
browser software making the request (spider)
Error Log:
request resulted in errors (404)
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

33. W3C Extended Log File Format
cs = client-to-server actions
s = server actions
c = client actions
sc = server-to-client actions
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

34. Example of Web Log entries
Apache web log:
[29/Mar/2002:03:58: ] "GET /~sophal/whole5.gif HTTP/1.0" " "Mozilla/4.0 (compatible; MSIE 5.0; AOL 6.0; Windows 98; DigExt)"
[29/Mar/2002:03:59: ] "GET /~alexlam/resume.html HTTP/1.0" "-" "Mozilla/5.0 (Slurp/cat;
[29/Mar/2002:03:00: ] "GET /~tahir/indextop.html HTTP/1.1" " "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)“
[29/Mar/2002:03:00: ] "GET /~tahir/animate.js HTTP/1.1" " "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)“
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

35. Routine Server Log Analysis
Typical statistics/histograms that are computed
Most and least visited web pages
Entry and exit pages
Referrals from other sites or search engines
What are the searched keywords
How many clicks/page views a page received
Error reports, like broken links
Many software products that produce standard reports of this type of data
Very useful for Web site managers
But does not provide “deep” insights
e.g., are there clusters/groups of users that use the site in different ways?
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

36. Visualization of Web Log Data over Time
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

37. Descriptive Summary Statistics
Histograms, scatter plots, time-series plots
Very important!
Helps to understand the big picture
Provides “marginal” context for any model-building
models aggregate behavior, not individuals
Challenging for Web log data
Examples
Session lengths (e.g., power laws)
Click rates as a function of time, content
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

38. Web data measurement issues
Important to understand how data is collected
Web data is collected automatically via software logging tools
Advantage:
No manual supervision required
Disadvantage:
Data can be skewed (e.g. due to the presence of robot traffic)
Important to identify robots (also known as crawlers, spiders)
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

39. L = number of page requests in a single session
from visitors to
over 1 week in November 2002
(robots removed)
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

40. A time-series plot of ICS Website data
Number of page requests per hour as a function of time from page requests in the Web server logs during the first week of April 2002.
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

41. Example: Web Traffic from Commercial Site (slide from Ronny Kohavi)
Sept-11 Note significant drop in human traffic, not bot traffic
Weekends
Internal Perfor-mance bot
Registration at Search Engine sites
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

42. Robot / human identification
Removal of robot data is important preprocessing step before clickstream analysis
Robot page-requests often identified using a variety of heuristics
e.g. some robots self-identify themselves in the server logs
All robots in principle should visit robots.txt on the Web Server
Also, robots should identify themselves via the User Agent field in page requests
But other robots actively try to disguise that they are robots
Patterns of access
Robots explore the entire website in breadth first fashion
Humans access web-pages more typically in depth-first fashion
Timing between page-requests can be more regular for robots (e.g., every 5 seconds)
Duration of sessions, number of page-requests per day: often unusually large (e.g., 1000’s of page-requests per day) for robots.
Tan and Kumar (Journal of Data Mining and Knowledge Discovery, 2002) provide a detailed description of using classification techniques to learn how to detect robots
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

43. Fractions of Robot Data (from Tan and Kumar, 2002)
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

44. From Tan and Kumar, Overall accuracies of around 90% were obtained using decision tree classifiers, trained on sessions of lengths 1, 2, 3, 4,..
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

45. Page requests, caching, and proxy servers
In theory, requester browser requests a page from a Web server and the request is processed
In practice, there are
Other users
Browser caching
Dynamic addressing in local network
Proxy Server caching
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

46. Page requests, caching, and proxy servers
A graphical summary of how page requests from an individual user can be masked at various stages between the user’s local computer and the Web server.
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

47. Page requests, caching, and proxy servers
Web server logs are therefore not so ideal in terms of a complete and faithful representation of individual page views
There are heuristics to try to infer the true actions of the user: -
Path completion (Cooley et al. 1999)
e.g. If known B -> F and not C -> F, then session ABCF can be interpreted as ABCBF
Anderson et al for more heuristics
In general case, it is hard to know what exactly the user viewed
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

48. Identifying individual users from Web server logs
Useful to associate specific page requests to specific individual users
IP address most frequently used
Disadvantages
One IP address can belong to several users
Dynamic allocation of IP address
Better to use cookies (or login ID if available)
Information in the cookie can be accessed by the Web server to identify an individual user over time
Actions by the same user during different sessions can be linked together
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

49. Identifying individual users from Web server logs
Commercial websites use cookies extensively
97 % of users have cookies enabled permanently on their browsers
(source: Amazon.com, 2003)
However …
There are privacy issues – need implicit user cooperation
Cookies can be deleted / disabled
Another option is to enforce user registration
High reliability
But can discourage potential visitors
Large portals (such as Yahoo!) have high fraction of logged-in users
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

50. Sessionizing Time oriented (robust)
e.g., by gaps between requests
not more than 20 minutes between successive requests
this is a heuristic – but is a standard “rule” used in practice
Navigation oriented (good for short sessions and when timestamps unreliable)
Referrer is previous page in session, or
Referrer is undefined but request within 10 secs, or
Link from previous to current page in web site
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

51. Client-side data Advantages of collecting data at the client side:
Direct recording of page requests (eliminates ‘masking’ due to caching)
Recording of all browser-related actions by a user (including visits to multiple websites)
More-reliable identification of individual users (e.g. by login ID for multiple users on a single computer)
Preferred mode of data collection for studies of navigation behavior on the Web
Companies like ComScore and Nielsen use client-side software to track home computer users
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

52. Client-side data
Statistics like ‘Time per session’ and ‘Page-view duration’ are more reliable in client-side data
Some limitations
Still some statistics like ‘Page-view duration’ cannot be totally reliable e.g. user might go to fetch coffee
Need explicit user cooperation
Typically recorded on home computers – may not reflect a complete picture of Web browsing behavior
Web surfing data can be collected at intermediate points like ISPs, proxy servers
Can be used to create user profile and target advertise
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

53. Time (over a 24-hour period)
Example of client-side data from Alexa, each “dot” is a URL request in a browser
Time (over a 24-hour period)
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

54. Case Study: Clusters of Markov Chains for modeling user navigation on a Website
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

55. Markov Models for Modeling User Navigation
General approach is to use a finite-state Markov chain
Each state can be a specific Web page or a category of Web pages
If only interested in the order of visits (and not in time), each new request can be modeled as a transition of states
Issues
Self-transition
Time-independence
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

56. Modeling Web Page Requests with Markov chain mixtures
MSNBC Web logs (circa 2000)
Order of 2 million individual users per day
different session lengths per individual
difficult visualization and clustering problem
WebCanvas
uses mixtures of Markov chains to cluster individuals based on their observed sequences
software tool: EM mixture modeling + visualization
Next few slides are based on material in:
I. Cadez et al, Model-based clustering and visualization of navigation patterns on a Web site, Journal of Data Mining and Knowledge Discovery, 2003.
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

57. Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

58. From Web logs to sequences
, -, 3/22/00, 10:35:11, W3SVC, SRVR1, , 781, 363, 875, 200, 0, GET, /top.html, -,
, -, 3/22/00, 10:35:16, W3SVC, SRVR1, , 5288, 524, 414, 200, 0, POST, /spt/main.html, -,
, -, 3/22/00, 10:35:17, W3SVC, SRVR1, , 30, 280, 111, 404, 3, GET, /spt/images/bk1.jpg, -,
, -, 3/22/00, 16:18:50, W3SVC, SRVR1, , 60, 425, 72, 304, 0, GET, /top.html, -,
, -, 3/22/00, 16:18:58, W3SVC, SRVR1, , 8322, 527, 414, 200, 0, POST, /spt/main.html, -,
, -, 3/22/00, 16:18:59, W3SVC, SRVR1, , 0, 280, 111, 404, 3, GET, /spt/images/bk1.jpg, -,
, -, 3/22/00, 20:54:37, W3SVC, SRVR1, , 140, 199, 875, 200, 0, GET, /top.html, -,
, -, 3/22/00, 20:54:55, W3SVC, SRVR1, , 17766, 365, 414, 200, 0, POST, /spt/main.html, -,
, -, 3/22/00, 20:54:55, W3SVC, SRVR1, , 0, 258, 111, 404, 3, GET, /spt/images/bk1.jpg, -,
, -, 3/22/00, 20:55:07, W3SVC, SRVR1, , 0, 258, 111, 404, 3, GET, /spt/images/bk1.jpg, -,
, -, 3/22/00, 20:55:36, W3SVC, SRVR1, , 1061, 382, 414, 200, 0, POST, /spt/main.html, -,
, -, 3/22/00, 20:55:36, W3SVC, SRVR1, , 0, 258, 111, 404, 3, GET, /spt/images/bk1.jpg, -,
, -, 3/22/00, 20:55:39, W3SVC, SRVR1, , 0, 258, 111, 404, 3, GET, /spt/images/bk1.jpg, -,
, -, 3/22/00, 20:56:03, W3SVC, SRVR1, , 1081, 382, 414, 200, 0, POST, /spt/main.html, -,
, -, 3/22/00, 20:56:04, W3SVC, SRVR1, , 0, 258, 111, 404, 3, GET, /spt/images/bk1.jpg, -,
, -, 3/22/00, 20:56:33, W3SVC, SRVR1, , 0, 262, 72, 304, 0, GET, /top.html, -,
, -, 3/22/00, 20:56:52, W3SVC, SRVR1, , 19598, 382, 414, 200, 0, POST, /spt/main.html, -,
User 1 2 3 2 2 3 3 3 1 1 1 3 1 3 3 3 3
User 2 3 3 3 1 1 1
User 3 7 7 7 7 7 7 7 7
User 4 1 5 1 1 1 5 1 5 1 1 1 1 1 1
User 5 5 1 1 5 … …
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

59. Graphical Model for Markov Chainsci
Pages
HICSS Keynote Talk, Jan © Padhraic Smyth, UC Irvine:

60. Multiple Users…One Common Markov Chainci
Pages
Users
HICSS Keynote Talk, Jan © Padhraic Smyth, UC Irvine:

61. Multiple Users…One Chain per Userci
Pages
Users
HICSS Keynote Talk, Jan © Padhraic Smyth, UC Irvine:

62. One Chain per Cluster of Users
Cadez, Meek, Heckerman, Smyth, 2003
Clusterj A ci
Pages
Users
HICSS Keynote Talk, Jan © Padhraic Smyth, UC Irvine:

63. Likelihood of a Sequence in a Markov Chain Model
P(sequence) = P0(first state) x Pij ( aij )nij
Maximum likelihood estimation:
- given the nij’s, find the aij’s that maximize P(sequence)
Number of transitions from i to j
Transition probability from i to j
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

64. Probability Estimation in Markov Chains
p(state j | state i) = aij = nij / ni
With smoothing (or priors)
aij = (nij + aij ) / (ni + Sm aim )
where aij / Sm aim is the prior probability of transitioning from i to j
and where Sm aim is the strength (equivalent sample size) of the prior
Could set aij = aj (same priors for transitioning out of each state) or
aij = a (same priors for transitioning in and out of each state)
Number of times in state i
Number of transitions from i to j
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

65. Probability Estimation with Multiple Sequences
T sequences Assume sequences are independent nijt = number of times going from i to j in sequence t, t = 1,… T p(state j | state i) = aij = St nijt / St nit With smoothing: aij = (St nijt + aij ) / (St nit + Sm aim )
Number of transitions from i to j in sequence t
Number of times in state i in sequence t
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

66. Clusters of Markov Chains
Assume our data is generated by K different Markov chains
Each chain has its own parameters A, P0
This is a a mixture of Markov chains
T sequences – but we don’t know which sequence came from which cluster
Chicken-and-egg problem
If we knew which cluster each sequence belonged to, we could group sequences by cluster, and estimation of cluster parameters is easy
If we knew the parameters for each Markov chain, we could figure out (by Bayes rule) the most likely cluster for each sequence
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

67. Clusters of Probabilistic State Machines
Motivation:
approximate the heterogeneity of Web surfing behavior
HICSS Keynote Talk, Jan © Padhraic Smyth, UC Irvine:

68. EM Algorithm for Markov Clusters
EM = expectation-maximization
E-step
For each sequence, and given current parameter estimates for each cluster, estimate p(cluster k | sequence), k = 1, … K
M-step
Given p(cluster k | sequence), estimate Ak and Pk0 for each cluster
Algorithm:
Start with an initial random guess at parameters or p(cluster k | sequence)
Iteration = pair of EM steps
Halt iterations when parameters or p(cluster k | sequence) are not changing
Guaranteed to converge to a local maximum of the likelihood (under general conditions)
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

69. E Step of EM Algorithm T sequences P(cluster k | sequence t)
proportional to P(sequence t | cluster k) P(cluster k)
P(sequence t | cluster k)
= pkt = Pk0(first state) x Pij ( akij )nijt
Compute these “membership” probabilities for each sequence and each cluster k. Yields a T x K matrix of membership probabilities
Number of times going
from state i to state j in sequence t
Current parameter estimates for cluster k
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

70. M Step of EM Algorithm
For each transition probability parameter in each cluster k=1,..K
akij = ( St nijt pkt ) / ( St nit pkt )
Compute Pk0(first state) in a similar fashion
Also estimate P(cluster k) = (1/T) St pkt
Number of times in state i in sequence t
are “fractionally weighted”
by pkt, probability that sequence t
came from cluster k
Transition probability i->j
for cluster k
Transitions from i to j in sequence t
are “fractionally weighted”
by pkt, probability that sequence t
came from cluster k
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

71. Time Complexity E-Step M-step T sequences, average length L K clusters
Compute T x K matrix
Each entry takes O(L) time to compute
O(TKL) overall
M-step
For each of M2 transition probabilities
Sum over each sequence T
O(T M2 )
other parameters take less time
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

72. Experimental Methodology
Model Training:
fit 2 types of models
mixtures of histograms (multinomials)
mixtures of finite state machines
Train on a full day’s worth of MSNBC Web data
Model Evaluation:
“one-step-ahead” prediction on unseen test data
Test sequences from a different day of Web logs
compute log P(user’s next click | previous clicks, model)
Using equation on the previous slide
logP score:
Rewarded if next click was given high P by the model
Punished if next click was given low P by the model
negative average of logP scores ~ “predictive entropy”
Has a natural interpretation
Lower bounded by 0 bits (perfect prediction)
Upper bounded by log M bits, where M is the number of categories
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

73. Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

74. Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

75. Timing Results
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

76. WebCanvas Software tool for Web log visualization
uses Markov mixtures to cluster data for display
extensively used within Microsoft
also applied to non-Web data (e.g., how users navigate in Word, etc)
Algorithm and visualization are in SQLServer (the “sequence mining” tool)
Model-based visualization
random sample of actual sequences
interactive tiled windows displayed for visualization
more effective than
planar graphs
traffic-flow movie in Microsoft Site Server v3.0
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

77. HICSS Keynote Talk, Jan 2008 © Padhraic Smyth, UC Irvine:

78. Insights from WebCanvas for MSNBC data
From msnbc.com site adminstrators….
significant heterogeneity of behavior
relatively focused activity of many users
typically only 1 or 2 categories of pages
many individuals not entering via main page
detected problems with the weather page
missing transitions (e.g., tech <=> business)
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

79. Possible Extensions Adding time-dependence Uncategorized Web pages
adding time-between clicks, time of day effects
Uncategorized Web pages
coupling page content with sequence models
Modeling “switching” behaviors
allowing users to switch between behaviors
Could use a topic-style model: users = mixtures of behaviors
e.g., Girolami M & Kaban A., Sequential Activity Profiling: Latent Dirichlet Allocation of Markov Chains, Journal of Data Mining and Knowledge Discovery, Vol 10,
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

80. Related Work Mixtures of Markov chains
special case: Poulsen (1990)
general case: Ridgeway (1997), Smyth (1997)
Clustering of Web page sequences
non-probabilistic approaches (Fu et al, 1999)
Markov models for prediction
Anderson et al (IJCAI, 2001):
mixtures of Markov outperform other sequential models for page-request prediction
Sen and Hansen 2003
Zukerman et al. 1999
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

81. Further Reading
Modeling the Internet and the Web, P. Baldi, P. Frasconi, P. Smyth, Wiley, 2003.
ACM Transactions on Internet Technology (ACM TOIT)
Annual WebKDD workshops at the ACM SIGKDD conferences.
Papers on user-navigation models
Selective Markov models for predicting Web page accesses, M. Deshpande, G. Karypis, ACM Transactions on Internet Technology, May 2004.
Model-based clustering and visualization of navigation patterns on a Web site, Cadez et al, Journal of Data Mining and Knowledge Discovery, 2003.
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

82. Backup Slides (can be ignored)
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

83. Prediction with Markov mixtures
P(st+1 | s[1,t] ) =
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

84. Prediction with Markov mixtures
P(st+1 | s[1,t] ) = S P(st+1 , k | s[1,t] )
= S P(st+1 | k , s[1,t] ) P(k | s[1,t] )
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

85. Prediction with Markov mixtures
P(st+1 | s[1,t] ) = S P(st+1 , k | s[1,t] )
= S P(st+1 | k , s[1,t] ) P(k | s[1,t] )
= S P(st+1 | k , st ) P(k | s[1,t] )
Prediction of
kth component
Membership, based
on sequence history
=> Predictions are a convex combination of
K different component transition matrices,
with weights based on sequence history
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

86. Analysis of Search Engine Query Logs
# of Queries
Source
Time Period
Lau & Horvitz
4690
Excite
Sep 1997
Silverstein et al
1 Billion
AltaVista
6 weeks in Aug & Sep 1998
Spink et al
(series of studies)
1 Million for each time period
Sep 1997 Dec 1999 May 2001
Xie & O’Hallaron
110,000
Vivisimo
35 days Jan & Feb 2001
1.9 Million
8 hrs in a day, Dec 1999
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

87. Main Results
Average number of terms in a query ranges from a low of 2.2 to a high of 2.6
The most common number of terms in a query was 2
The majority of users don’t refine their query
The number of users who viewed only a single page increased from 29% (1997) to 51% (2001) (Excite)
85% of users viewed only first page of search results (AltaVista)
45% (2001) of queries are about Commerce, Travel, Economy, People (was 20% in 1997)
All four studies produced a generally consistent set of findings about user behavior in a search engine context
most users view relatively few pages per query
most users don’t use advanced search features
For both search engine, Distribution of query lengths is qualitatively similar
Adult content, 1/ to 1/
Entertainment or recreation, 20% 1997 to 7% 2001
Business 20% 1997 to 45% 2001
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

88. Markov models for page prediction
For simplicity, consider order-dependent, time-independent finite-state Markov chain with M states
Let s be a sequence of observed states of length L. e.g. s = ABBCAABBCCBBAA with three states A, B and C. st is state at position t (1<=t<=L). In general,
first-order Markov assumption
This provides a simple generative model to produce sequential data
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

89. Markov models for page prediction
First-order Markov model assumes that the next state is based only on the current state
Limitations
Doesn’t consider ‘long-term memory’
We can try to capture more memory with kth-order Markov chain
Inordinate amount of training data O(Mk+1)
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine

90. Mixtures of Markov Chains
Cadez et al. (2003) and Sen and Hansen (2003) replace the first-order Markov chain:
with a mixture of first-order Markov chains
where c is a discrete-value hidden variable taking K values Sk P(c = k) = 1 and
P(st | st-1, c = k) is the transition matrix for the kth mixture component
One interpretation of this is user behavior consists of K different navigation behaviors described by the K Markov chains
Data Mining Lectures Analysis of Web User Data Padhraic Smyth, UC Irvine