1. Hierarchical Probabilistic Relational Models for Collaborative Filtering Jack Newton (newton@cs.ualberta.ca) and Russ Greiner (greiner@cs.ualberta.ca) Introduction, Problem Set UpOur ApproachExperiments and Results Introduction Recommender Systems PRMs Hierarchical PRMs Results Contributions Personalized Recommender Systems recommend specific products For example: Amazon.com’s book recommender; Yahoo!’s LAUNCHcast music recommender Very popular! We designed/built a recommender system – “tadpole” – using Probabilistic Relational Models (PRMs) [KP98] Hierarchical PRMs (hPRMs) [Get02] applied to EachMovie dataset Content-based recommenders use only facts about products and individual (potential) purchaser Eg: a movie recommender system: just People  Movies database Each tuple  25, Male, Calif, … ,  Action, Budget, …, 4  lists facts about a person, facts about a movie, a vote  {1,.., 5} Use dataset to learn a classifier, that predicts vote for novel person/movie pairs. Collaborative Filtering-based recommenders base recommendations on ratings other “similar” users have assigned to similar products. If person P1 appears similar to person P2 (perhaps based on their previous “liked movies”) and P2 liked X, perhaps P1 will like X. Our goal: a cohesive framework for combining all types of information: properties of product properties of user, voting patterns of all users, As well voting patterns of a given user to make accurate recommendations. Probabilistic Relational Models, and an extension to PRMs called Hierarchical PRMs (hPRMs), offer a probabilistic framework we can apply to the recommender system problem domain [Get02]. Evaluation: applying the PRM framework to the EachMovie dataset. A PRM encodes class-Level dependencies used to make inferences about a particular instance of a class. Learning Must first learn PRM from the data [FGK99] algorithm for learning a legal structure for a PRM estimating parameters for that PRM. Education Age Gender PersonMovie Vote Score Theater Status Video Status Inference Given PRM encoding the class-level dependencies, Generate a Ground Bayesian Network for each specific object Use same structure/parameters for each instance of class Use standard Bayesian Network inference algorithm Different results for child nodes as different data for parents, … John.Education John.Age John.Gender Vote JohnOnStarWars.Score StarWars.TheaterStatus StarWars.VideoStatus Two limitation of PRMs (which motivate hPRMs): 1. Vote.Score can depend on attributes of related objects, such as Person.Age, but Vote.Score can NOT depend on itself in any way. BAD: want John’s Vote on Star Wars to help predict John’s Vote on T3 Fred’s Vote on Star Wars … (Why? PRM’s class-level dependency structure must be DAG) 2. Restricted to one dependency graph for Vote.Score However, you could may want one dependency graph for movies of the Comedy genre, and another for the Action genre hPRMs [Get02] address both problems: hPRMs use a class hierarchy such as that in Figure 3a, to learn the hPRM in Figure 3b: Action Movie ComedyThriller Romantic Comedy Slapstick Comedy Education Age Gender Person Action-Movie Thriller-Vote Score Action-Vote Score Romantic- Comedy-Vote Score Slapstick- Comedy-Vote Score Romantic- Comedy-Movie Theater Status Video Status Slapstick- Comedy-Movie Theater Status Video Status Thriller-Movie Theater Status Video Status Theater Status Video Status AVG Compared to Correlation (CR), Bayesian Clustering (BC), Bayesian Network (BN), Vector Similarity (VSIM) as presented in [BHK98] Metric: Mean Absolute Error (MAE) [BHK98] 5-fold cross-validation Built PRM and hPRM models – learning, inference algorithms Show that (h)PRMs can apply to recommender systems in general Evaluated in context of EachMovie database, demonstrated competitive results against existing algorithms Demonstrate superiority of hPRMs over standard PRMs. Algorithm Absolute Deviation CR1.257 BC1.127 BN1.143 VSIM2.113 PRM1.26 Algorithm Absolute Deviation CR0.994 BC1.103 BN1.066 VSIM2.136 hPRM1.060 References [BHK98] John S. Breese, David Heckerman, and Carl Kadie. Empirical analysis of predictive algorithms for collaborative filtering. In UAI98, pages 43–52, 1998. [FGKP99] Nir Friedman, Lise Getoor, Daphne Koller, and Avi Pfeffer. Learning probabilistic relational models. In IJCAI-99, pages 1300–1309, 1999. [Get02] L. Getoor. Learning Statistical Models from Relational Data. PhD thesis, Stanford University, 2002. [KP98] D. Koller and A. Pfeffer. Probabilistic frame based systems. In AAAI-98,pages 580–587, Madison, WI, 1998. Figure 1: A standard PRM Figure 2: A ground Bayesian Network Figure 3a: A class hierarchyFigure 3b: An hPRM learned on the EachMovie dataset The EachMovie Dataset Acknowledgements Lise Getoor, for useful discussion, encouragement to pursue this line of work, and access to software and data that aided us in building our tadpole system. Alberta Ingenuity, NSERC, and iCORE for funding. Often used to test recommender systems 72,916 users, 1,628 movies, 2,811,983 votes Composed of three tables: Person: describes people; fields: age, gender, zip code, … Movie: describes movies; fields: genre, … Vote: user’s rating on movie; {0,1,2,3,4,5}