1 Collaborative Filtering Rong Jin Department of Computer Science and Engineering Michigan State University

2 Outline  Brief introduction information filtering  Collaborative filtering Major issues in collaborative filtering Main methods for collaborative filtering Flexible mixture model for collaborative filtering Decoupling model for collaborative filtering

3 Short vs. Long Term Info. Need  Short-term information need (Ad hoc retrieval) “Temporary need”, e.g., info about used cars Information source is relatively static User “pulls” information Application example: library search, Web search  Long-term information need (Filtering) “Stable need”, e.g., new data mining algorithms Information source is dynamic System “pushes” information to user Applications: news filter

4 Examples of Information Filtering  News filtering  filtering  Movie/book/product recommenders  Literature recommenders  And many others …

5 Information Filtering  Basic filtering question: Will user U like item X?  Two different ways of answering it Look at what U likes  characterize X  content-based filtering Look at who likes X  characterize U  collaborative filtering  Combine content-based filtering and collaborative filtering

6 Other Names for Information Filtering  Content-based filtering is also called “Adaptive Information Filtering” in TREC “Selective Dissemination of Information” (SDI) in Library & Information Science  Collaborative filtering is also called Recommender systems

7 Example: Content-based Filtering Description: A homicide detective and a fire marshall must stop a pair of murderers who commit videotaped crimes to become media darlings Rating: Description: Benjamin Martin is drawn into the American revolutionary war against his will when a brutal British commander kills his son. Rating: Description: A biography of sports legend, Muhammad Ali, from his early days to his days in the ring Rating: History What to Recommend? Description: A high-school boy is given the chance to write a story about an up-and-coming rock band as he accompanies it on their concert tour. Recommend: ? Description: A young adventurer named Milo Thatch joins an intrepid group of explorers to find the mysterious lost continent of Atlantis. Recommend: ? No Yes

8 Example: Collaborative Filtering User User User 32?354 User 3 is more similar to user 1 than user 2  5 for movie “15 minutes” for user 3 5

9 Collaborative Filtering (CF) vs. Content-based Filtering (CBF)  CF do not need content of items while CBF relies the content of items  CF is useful when content of items are not available or difficult to acquire are brief and insufficient  Example: movie recommendation  A movie is preferred may because its actor its director its popularity

10 Application of Collaborative Filtering

11 ? Collaborative Filtering  Goal: Making filtering decisions for an individual user based on the judgments of other users u1u2…umu1u2…um Users: U Objects: O o 1 o 2 … o j o j+1 … o n 3 1 …. … 4 2 ? 2 5 ? 4 3 ? 3 ? 1 2 u test 3 4…… 1

12 Collaborative Filtering  Goal: Making filtering decisions for an individual user based on the judgments of other users  General idea Given a user u, find similar users {u 1, …, u m } Predict u’s rating based on the ratings of u 1, …, u m

13 Example: Collaborative Filtering User User User 32?354 User 3 is more similar to user 2 than user 1  5 for movie “15 minutes” for user 3 5

14 Memory-based Approaches for CF  The key is to find users that are similar to the test user  Traditional approach Measure the similarity in rating patterns between different users Example: Pearson Correlation Coefficient

15 Pearson Correlation Coefficient for CF  Similarity between a training user y and a test user y 0 : Remove the rating bias from each training user

16 Pearson Correlation Coefficient for CF  Estimate ratings for the test user Weighted vote of normalized rates

17 Example User Normalized Rate User Normalized Rate User 32?354 Normalize Rate

18 Example User Normalized Rate User Normalized Rate User 32?354 Normalize Rate

19 Example User Normalized Rate User Normalized Rate User 32?354 Normalize Rate

20 Problems with Memory-based Approaches User 1?5342 User 2415?5 User 35?425 User 41535?  Most users only rate a few items Two similar users can may not rate the same set of items  Clustering users and items

21 Flexible Mixture Model (FMM) Cluster both users and items simultaneously User 1?5342 User 2415?5 User 35?425 User 41535? User clustering and item clustering are correlated !

22 Flexible Mixture Model (FMM) Cluster both users and items simultaneously User Class I1p(4)=1/4 p(5)=3/4 3 User Class IIp(4)=1/4 p(5)=3/4 p(1)=1/2 p(2)=1/2 p(4)=1/2 p(5)=1/2 Movie Type I Movie Type II Movie Type III Unknown ratings are gone!

23 Flexible Mixture Model (FMM) Z o Z u O U R P(o|Z o ) P(u|Z u ) P(Z o ) P(Z u ) P(r|Z o,Z u ) Z u : user class Z o : item class U: user O: item R: rating Hidden variable Observed variable

24  Annealed Expectation Maximization (AEM) algorithm  E-step: calculate posterior probability for hidden variables z u and Z o b: temperature for Annealed EM algorithm  M-step: updated parameters Flexible Mixture Model: Estimation

25 Flexible Mixture Model: Predication  Fold-in process Repeat the EM algorithm including ratings from the test user Fix all the parameters except for P(u t |z u ) Key issue: What user class does the test user belong to ?

26 Another Prob. with Memory-based Approaches User User User User Users with similar interests can have different rating patterns  Decoupling preference patterns from rating patterns

27 Decoupling Model (DM) Z o Z u O U Hidden variable Observed variable Z u : user class Z o : item class U: user O: item R: rating

28 Decoupling Model (DM) Z u : user class Z o : item class U: user O: item R: rating Z pref : whether users like items Z o Z u O U Z pref Hidden variable Observed variable

29 Decoupling Model (DM) Z u : user class Z o : item class U: user O: item R: rating Z pref : whether users like items Z R : rating class Z o Z u O U R Z pref Z R  Separating preference and rating patterns  User class + Rating class  rating R Z u  Z pref and Z R +Z pref  r Hidden variable Observed variable

30 Experiment  Datasets: EachMovie and MovieRating  Evaluation: Mean Absolute Error (MAE): average absolute deviation of the predicted ratings to the actual ratings on items. The smaller MAE, the better the performance MovieRatingEachMovie Number of Users Number of Items Avg. # of rated items/User Number of ratings56

31 Experiment Protocol  Test the sensitivity of the proposed model to the amount of training data Vary the number of training users MovieRating dataset: 100 and 200 training users EachMovie dataset: 200 and 400 training users  Test the sensitivity of the proposed model to the information needed for the test user Vary the number of rated items provided by the test user  5, 10, and 20 items are given with ratings

32 Experimental Results: FMM and other baseline algorithms Movie Rating, 100 Training Users Movie Rating, 200 Training Users Each Movie, 400 Training Users Each Movie, 200 Training Users Given: Given: Given: Given: MAEMAE MAEMAE MAEMAE MAEMAE A smaller MAE indicates better performance

33 FMM vs. DM Results on Movie Rating Results on Each Movie Training Users Size Algorithms 5 Items Given 10 Items Given 20 Items Given 100 FMM DM FMM DM Training Users Size Algorithms 5 Items Given 10 Items Given 20 Items Given 200 FMM DM FMM DM Smaller value indicates better performance