Temporal Diversity in Recommender Systems Neal Lathia, Stephen Hailes, Licia Capra, and Xavier Amatriain SIGIR 2010 April 6, 2011 Hyunwoo Kim

Outline  Introduction  Why Temporal Diversity?  Evaluating for Diversity  Promoting Temporal Diversity  Conclusion 2

Introduction  Collaborative Filtering [Kim, ECRA2010] 3

Introduction 4 in 2006 in 2011 Alice  User’s interest changes over time [Zheng, ESWC2011] baby healtheducation

Introduction  A problem with current evaluation techniques –No temporal characteristics of the produced recommendations  In this work, –Diversity of top-N lists over time 5

Why Temporal Diversity?  Two perspectives –Changes that CF data undergoes over time –How surveyed users respond to recommendations with varying levels of diversity  Changes over time –Continuous rating of content –Recommender systems have to make decisions based on INCOMPLETE and CHANGING data –A list at any particular time is likely to be different with previous list –Do these changes translate into different recommendations over time? 6

Why Temporal Diversity?  User survey –Popular movies from 7

Why Temporal Diversity?  User survey –S1: popular movies with no diversity –S2: popular movies with diversity –S3: randomly selected movies 8 In S3, some users commented: “appeared to very random” “varied widely” “avoided box office hits” … In S1, some users commented: “lack of diversity persisted” “too naïve” “not working” “decreased interest” … Users are responding to the impression of the recommender system!!

Why Temporal Diversity?  Qualities in recommendations –ACCURATE recommendations –CHANGE OVER TIME –NEW recommendations 9

Evaluating for Diversity  How diverse CF algorithms are over time –Baseline: item’s mean rating –Item-based k-Nearest Neighbor (kNN) –Matrix factorization approach based on Singular Value Decomposition (SVD)  Dataset –Netflix prize dataset  To improve the accuracy of predictions about how much someone is going to enjoy a movie based on their movie preferences  $1,000,000 grand prize on September 21,

Evaluating for Diversity  Diversity and novelty 11 Last week’s list This week’s list Diversity = 1/5

Evaluating for Diversity  Diversity and novelty 12 Previous recommendations This week’s list Novelty = 2/5

Evaluating for Diversity  Diversity results and analysis –Baseline produces little to no diversity –Factorization and nearest neighbor approaches increment diversity 13

Evaluating for Diversity  Novelty results and analysis –Novelty values are lower than diversity values –When different a recommendation appears, it is a recommendation at some point in the past 14

Evaluating for Diversity  How diversity relates to accuracy –RMSE: Root Mean Squared Error –Different algorithms often overlap and kNN CF is sometimes less accurate than the baseline 15

Promoting Temporal Diversity  Diversity comes at the cost of accuracy  When promoting diversity, we must continue to take into account users’ preferences  Three methods –Temporal switching –Temporal user-based switching –Re-ranking frequent visitors’ lists 16

Promoting Temporal Diversity  Temporal switching  Temporal user-based switching 17 kNN SVD kNN SVD kNN user login 1 st 2 nd 3 rd 4 th 5 th

Promoting Temporal Diversity  Temporal switching from a system 18

Promoting Temporal Diversity  Temporal user-based switching 19

Promoting Temporal Diversity  Re-ranking frequent visitors’ lists 20 Full listTop-5 listRe-ranking list Diversity 40%

Promoting Temporal Diversity  Re-ranking frequent visitors’ lists –Only a single CF algorithm is used 21

Conclusion 22  What we found –State-of-the-art CF algorithms produce low temporal diversity –They repeatedly recommend the same top-N items to users  What we did –A metric to measure temporal diversity –A fine-grained analysis of the factors that may influence diversity  Future work –How novel items find their way into recommendations –How user rating patterns can be used to improve recommender system’s resilience to attack