1. Hybrid Web Recommender Systems
Robin Burke
Presentation by Jae-wook Ahn
10/04/05

2. Hybrid Web Recommender Systems
References
Entrée system & dataset
Burke, R. (2002). Semantic ratings and heuristic similarity for collaborative filtering. AAAI Workshop on Knowledge-based Electronic Markets 2000.
Feature augmentation, mixed hybrid example
Torres, R., McNee, S., Abel, M., Konstan J., & Riedl J. (2004). Enhancing Digital Libraries with TechLens+. Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries.
Hybrid recommender system UI issue
Schafer, J. (2005). DynamicLens: A Dynamic User-Interface for a Meta-Recommendation System. Workshop: Beyond Personalization 2005, IUI’05.
Collaborative filtering algorithm
Sarwar, B., Karypis, G., Konstan, J., & Riedl, J. (2001). Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web.
10/5/05
Hybrid Web Recommender Systems

3. Concepts and Techniques

4. Hybrid Recommender Systems
Mix of recommender systems
Recommender system classification – knowledge source
Collaborative (CF)
User’s ratings “only”
Content-based (CN)
Product features, user’s ratings
Classifications of user’s likes/dislikes
Demographic
User’s ratings, user’s demographics
Knowledge-based (KB)
Domain knowledge, product features, user’s need/query
Inferences about a use’s needs and preferences
10/5/05
Hybrid Web Recommender Systems

5. Hybrid Web Recommender Systems
CF vs. CN
User-based CF
Searches for similar users in user-item “rating” matrix
Item-based CF
Searches for similar items in user-item “rating” matrix CN
Searches for similar items in item-feature matrix
Example – TF*IDF term weight vector for news recommendation
Items
Ratings
Users
10/5/05
Hybrid Web Recommender Systems

6. Recommender System Problems
Cold-start problem
Learning based techniques
Collaborative, content-based, demographic
 Hybrid techniques
Stability vs. plasticity problem
Difficulty to change established user’s profile
Temporal discount – older rating with less influence
KB – fewer cold start problem (no need of historical data)
CF/Demographic – cross-genre niches, jump outside of the familiar (novelty, serendipity)
10/5/05
Hybrid Web Recommender Systems

7. Strategies for Hybrid Recommendation
Combination of multiple recommendation techniques together for producing output
Different techniques of different types
Most common implementations
Most promise to resolve cold-start problem
Different techniques of the same type
Ex) NewsDude – naïve Bayes + kNN
10/5/05
Hybrid Web Recommender Systems

8. Seven Types of Recommender Systems
Taxonomy by Burke (2002)
Weighted
Switching
Mixed
Feature combination
Feature augmentation
Cascade
Meta-level
10/5/05
Hybrid Web Recommender Systems

9. Hybrid Web Recommender Systems
Weighted Hybrid
Concept
Each component of the hybrid scores a given item and the scores are combined using a linear formula
When recommenders have consistent relative accuracy across the product space
Uniform performance among recommenders (otherwise  other hybrids)
10/5/05
Hybrid Web Recommender Systems

10. Weighted Hybrid Procedure
Training
Joint rating
Intersection – candidates shared between the candidates
Union – case with no possible rating  neutral score (neither liked nor disliked)
Linear combination
10/5/05
Hybrid Web Recommender Systems

11. Hybrid Web Recommender Systems
Mixed Hybrid
Concepts
Presentation of different components side-by-side in a combined list
If lists are to be combined, how are rankings to be integrated?
Merging based on predicted rating or on recommender confidence
Not fit with retrospective data
Cannot use actual ratings to test if right items ranked highly
Example
CF_rank(3) + CN_rank(2)  Mixed_rank(5)
10/5/05
Hybrid Web Recommender Systems

12. Mixed Hybrid Procedure
Candidate generation
Multiple ranked lists
Combined display
10/5/05
Hybrid Web Recommender Systems

13. Hybrid Web Recommender Systems
Switching Hybrid
Concepts
Selects a single recommender among components based on recommendation situation
Different profile  different recommendation
Components with different performance for some types of users
Existence of criterion for switching decision
Ex) confidence value, external criteria
10/5/05
Hybrid Web Recommender Systems

14. Switching Hybrid Procedure
Switching decision
Candidate generation
Scoring
No role for unchosen recommender
10/5/05
Hybrid Web Recommender Systems

15. Feature Combination Hybrid
Concepts
Inject features of one source into a different source for processing different data
Features of “contributing recommender” are used as a part of the “actual recommender”
Adding new features into the mix
Not combining components, just combining knowledge source
10/5/05
Hybrid Web Recommender Systems

16. Feature Combination Hybrid Procedure
 In training stage
Candidate generation
Scoring
10/5/05
Hybrid Web Recommender Systems

17. Feature Augmentation Hybrid
Concepts
Similar to Feature Combination
Generates new features for each item by contributing domain
Augmentation/combination – done offline
Comparison with Feature Combination
Not raw features (FC), but the result of computation from contribution (FA)
More flexible to apply
Adds smaller dimension
10/5/05
Hybrid Web Recommender Systems

18. Feature Augmentation Hybrid Procedure
10/5/05
Hybrid Web Recommender Systems

19. Hybrid Web Recommender Systems
Cascade Hybrid
Concepts
Tie breaker
Secondary recommender
Just tie breaker
Do refinements
Primary recommender
Integer-valued scores – higher probability for ties
Real-valued scores – low probability for ties
Precision reduction
Score:  0.83
10/5/05
Hybrid Web Recommender Systems

20. Cascade Hybrid Procedure
Primary recommender
Ranks
Break ties by secondary recommender
10/5/05
Hybrid Web Recommender Systems

21. Hybrid Web Recommender Systems
Meta-level Hybrid
Concepts
A model learned by contributing recommender
 input for actual recommender
Contributing recommender completely replaces the original knowledge source with a learned model
Not all recommenders can produce the intermediary model
10/5/05
Hybrid Web Recommender Systems

22. Meta-level Hybrid Procedure
Contributing recommender
 Learned model
Knowledge Source Replacement
Actual Recommender
10/5/05
Hybrid Web Recommender Systems

23. Experiments

24. Testbed – Entrée Restaurant Recommender
Entrée System
Case-based reasoning
Interactive critiquing dialog
Ex) Entry  Candidates  “Cheaper”  Candidates  “Nicer”  Candidates  Exit
Not “narrowing” the search by adding constrains, but changing the focus in the feature space
10/5/05
Hybrid Web Recommender Systems

25. Testbed – Entrée Restaurant Recommender (cont’d)
Entrée Dataset
Rating
Entry, ending point – “positive” rating
Critiques – “negative” rating
Mostly negative ratings
Validity test for positive ending point assumption – strong correlation between original vs. modified (entry points with positive ratings)
Small in size
10/5/05
Hybrid Web Recommender Systems

26. Evaluation Methodology
Measures
ARC (Average Rank of the Correct recommendations)
Accuracy of retrieval
At different size retrieval set
Fraction of the candidate set (0 ~ 1.0)
Training & Test set
5 fold cross validation – random partition of training/test set
“Leave one out” methodology – randomly remove one item and check whether the system can recommend it
Sessions Sizes
Single visit profiles – 5S, 10S, 15S
Multiple visit profiles – 10M, 20M, 30M
10/5/05
Hybrid Web Recommender Systems

27. Hybrid Web Recommender Systems
Baseline Algorithms
Collaborative Pearson (CFP)
Pearson’s correlation coefficient for similarity
Collaborative Heuristic (CFH)
Heuristics for calculating distances between critiques
“nicer” and “cheaper”  dissimilar
“nicer” & “quieter”  similar
Content-based (CN)
Naïve Bayes algorithm – compute probability that a item is “liked” / “disliked”
Too few “liked” items  modified candidate generation
Retrieve items with common features with the “liked” vector of the naïve Bayes profile
Knowledge-based (KB)
Knowledge-based comparison metrics of Entrée
Nationality, price, atmosphere, etc.
10/5/05
Hybrid Web Recommender Systems

28. Hybrid Web Recommender Systems
Baseline Evaluations
Techniques vary in performance on the Entrée data
Content-based (CN) – weak
Knowledge-based (KB) – better on single-session than multi-session
Heuristic collaborative (CFH) – better than correlation-based (CFP) for short profiles
Room for improvement
Multi-session profiles
10/5/05
Hybrid Web Recommender Systems

29. Hybrid Web Recommender Systems
Baseline Evaluations
10/5/05
Hybrid Web Recommender Systems

30. Hybrid Comparative Study
Missing components
Mixed hybrid
Not possible with retrospective data
Demographic recommender
No demographic data
10/5/05
Hybrid Web Recommender Systems

31. Hybrid Web Recommender Systems
Results – Weighted
Hybrid performance better in only 10 of 30
CN/CFP – consistent synergy (5 of 6)
Lacks uniform performance
KB, CFH
Linear weighting scheme assumption – fault
10/5/05
Hybrid Web Recommender Systems

32. Hybrid Web Recommender Systems
Results – Switching
KB hybrids – best switching hybrids
10/5/05
Hybrid Web Recommender Systems

33. Results – Feature Combination
CN/CFH, CN/CFP
Contributing CN
Identical to CFH, CFP
CFH maintains accuracy with reduced dataset
CF/CN Winnow – modest improvement
10/5/05
Hybrid Web Recommender Systems

34. Results – Feature Augmentation
Best performance so far
Particularly CN*/CF*
Good for multi-session profiles
10/5/05
Hybrid Web Recommender Systems

35. Hybrid Web Recommender Systems
Results – Cascade
CFP/KB, CFP/CN
Great improvement
Also good for multi-profile sessions
10/5/05
Hybrid Web Recommender Systems

36. Results – Meta-level Hybrids
CN/CF, CN/KB, CF/KB, CF/CN
Not effective
No synergy
Weakness of KB/CN in Entrée dataset
Both components should be strong
10/5/05
Hybrid Web Recommender Systems

37. Hybrid Web Recommender Systems
Discussion
Dominance of the hybrids over basic recommenders
Synergy was found under
Smaller profile size
Sparse recommendation density
 hybridization conquers cold start problem
10/5/05
Hybrid Web Recommender Systems

38. Hybrid Web Recommender Systems
Discussion (cont’d)
Best hybrids
Feature augmentation, cascade
FA allows a contributing recommender to make a positive impact
without interfering with the performance of the better algorithm
10/5/05
Hybrid Web Recommender Systems

39. Hybrid Web Recommender Systems
Conclusions
Knowledge-based recommendation is not limited
Numerously combined to build hybrids
Good for secondary or contributing components
Cascade hybrids are effective
Though rare in literatures
Effective for combining recommender with different strengths
Different performance characteristics
Six hybridization techniques
Relative accuracy & consistency of hybrid components
10/5/05
Hybrid Web Recommender Systems

40. System Example & Related Issues

41. System Example – TechLens+
Hybrid recommender system
Recommenders – CF, CN
Hybrid algorithms – CF/CN FA, CN/CF FA, Fusion (Mixed)
Corpus
CiteSeer
Title, abstract (CN), citations (CF)
Methodology
Offline experiment, Online user study with questionnaire (by asking satisfaction on the recommendation)
Results
Fusion was the best
Some FA were not good due the their sequential natures
Different algorithms should be used for recommending different papers
Users with different levels of experiences perceive recommendations differently
10/5/05
Hybrid Web Recommender Systems

42. Meta-recommender – DynamicLens
Can user provided information improve hybrid recommender system output?
Meta-recommender
Provide users with personalized control over the generation of a recommendation list from hybrid recommender system
MetaLens
IF (Information Filtering), CF
10/5/05
Hybrid Web Recommender Systems

43. Meta-recommender – DynamicLens (cont’d)
Dynamic query
Merges preference & recommendation interfaces
Immediate feedback
Discover why a given set of ranking recommendations were made
10/5/05
Hybrid Web Recommender Systems

44. Questions & Comments