Evaluation of Recommender Algorithms for an Internet Information Broker based on Simple Association Rules and on the Repeat-Buying Theory WEBKDD 2002 Edmonton,

Slides:

Advertisements

Similar presentations

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for 1 Lecture Notes for E Alpaydın 2010.

Advertisements

Relationship Mining Association Rule Mining Week 5 Video 3.

1 Evaluation Rong Jin. 2 Evaluation  Evaluation is key to building effective and efficient search engines usually carried out in controlled experiments.

LEARNING INFLUENCE PROBABILITIES IN SOCIAL NETWORKS Amit Goyal Francesco Bonchi Laks V. S. Lakshmanan University of British Columbia Yahoo! Research University.

A Graph-based Recommender System Zan Huang, Wingyan Chung, Thian-Huat Ong, Hsinchun Chen Artificial Intelligence Lab The University of Arizona 07/15/2002.

1 1 Slide © 2011 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole.

Association Rule Mining. 2 The Task Two ways of defining the task General –Input: A collection of instances –Output: rules to predict the values of any.

MIS2502: Data Analytics Association Rule Mining. Uses What products are bought together? Amazon’s recommendation engine Telephone calling patterns Association.

COLLABORATIVE FILTERING Mustafa Cavdar Neslihan Bulut.

Chapter 8 Interval Estimation Population Mean:  Known Population Mean:  Known Population Mean:  Unknown Population Mean:  Unknown n Determining the.

Active Learning and Collaborative Filtering

The Wisdom of the Few A Collaborative Filtering Approach Based on Expert Opinions from the Web Xavier Amatriain Telefonica Research Nuria Oliver Telefonica.

Lecture 14: Collaborative Filtering Based on Breese, J., Heckerman, D., and Kadie, C. (1998). Empirical analysis of predictive algorithms for collaborative.

LinkSelector: A Web Mining Approach to Hyperlink Selection for Web Portals Xiao Fang University of Arizona 10/18/2002.

Discovery of Aggregate Usage Profiles for Web Personalization

ForecastingOMS 335 Welcome to Forecasting Summer Semester 2002 Introduction.

Sparsity, Scalability and Distribution in Recommender Systems

Recommender systems Ram Akella November 26 th 2008.

Algorithms for Efficient Collaborative Filtering Vreixo Formoso Fidel Cacheda Víctor Carneiro University of A Coruña (Spain)

Collaborative Recommendation via Adaptive Association Rule Mining KDD-2000 Workshop on Web Mining for E-Commerce (WebKDD-2000) Weiyang Lin Sergio A. Alvarez.

Today Evaluation Measures Accuracy Significance Testing

FALL 2012 DSCI5240 Graduate Presentation By Xxxxxxx.

Performance of Recommender Algorithms on Top-N Recommendation Tasks

An Integrated Approach to Extracting Ontological Structures from Folksonomies Huairen Lin, Joseph Davis, Ying Zhou ESWC 2009 Hyewon Lim October 9 th, 2009.

Distributed Networks & Systems Lab. Introduction Collaborative filtering Characteristics and challenges Memory-based CF Model-based CF Hybrid CF Recent.

The paired sample experiment The paired t test. Frequently one is interested in comparing the effects of two treatments (drugs, etc…) on a response variable.

1 1 Slide IS 310 – Business Statistics IS 310 Business Statistics CSU Long Beach.

1 A Theoretical Framework for Association Mining based on the Boolean Retrieval Model on the Boolean Retrieval Model Peter Bollmann-Sdorra.

ASSOCIATION RULE DISCOVERY (MARKET BASKET-ANALYSIS) MIS2502 Data Analytics Adapted from Tan, Steinbach, and Kumar (2004). Introduction to Data Mining.

A Recommendation System for Software Function Discovery Naoki Ohsugi Software Engineering Laboratory, Graduate School of Information Science, Nara Institute.

Experimental Evaluation of Learning Algorithms Part 1.

1 Recommender Systems Collaborative Filtering & Content-Based Recommending.

Collaborative Filtering versus Personal Log based Filtering: Experimental Comparison for Hotel Room Selection Ryosuke Saga and Hiroshi Tsuji Osaka Prefecture.

M. Sulaiman Khan Dept. of Computer Science University of Liverpool 2009 COMP527: Data Mining Association Rule Mining March 5, 2009.

EigenRank: A Ranking-Oriented Approach to Collaborative Filtering IDS Lab. Seminar Spring 2009 강 민 석강 민 석 May 21 st, 2009 Nathan.

EXAM REVIEW MIS2502 Data Analytics. Exam What Tool to Use? Evaluating Decision Trees Association Rules Clustering.

Objectives Objectives Recommendz: A Multi-feature Recommendation System Matthew Garden, Gregory Dudek, Center for Intelligent Machines, McGill University.

Computational Intelligence: Methods and Applications Lecture 16 Model evaluation and ROC Włodzisław Duch Dept. of Informatics, UMK Google: W Duch.

Association Rule Mining

A Content-Based Approach to Collaborative Filtering Brandon Douthit-Wood CS 470 – Final Presentation.

ASSOCIATION RULES (MARKET BASKET-ANALYSIS) MIS2502 Data Analytics Adapted from Tan, Steinbach, and Kumar (2004). Introduction to Data Mining.

Case studies in recommender systems  The MovieLens data set, others –focus on improving the Mean Absolute Error …  What about the business.

Social Tag Prediction Paul Heymann, Daniel Ramage, and Hector Garcia- Molina Stanford University SIGIR 2008.

Charles Tappert Seidenberg School of CSIS, Pace University

A Clustering-based QoS Prediction Approach for Web Service Recommendation Shenzhen, China April 12, 2012 Jieming Zhu, Yu Kang, Zibin Zheng and Michael.

© 2012 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.

Identifying “Best Bet” Web Search Results by Mining Past User Behavior Author: Eugene Agichtein, Zijian Zheng (Microsoft Research) Source: KDD2006 Reporter:

Online Evolutionary Collaborative Filtering RECSYS 2010 Intelligent Database Systems Lab. School of Computer Science & Engineering Seoul National University.

Web Analytics Xuejiao Liu INF 385F: WIRED Fall 2004.

An Efficient Rigorous Approach for Identifying Statistically Significant Frequent Itemsets Adam Kirsch, Michael Mitzenmacher, Havard University Andrea.

Data Mining Association Rules Mining Frequent Itemset Mining Support and Confidence Apriori Approach.

Improvement of Apriori Algorithm in Log mining Junghee Jaeho Information and Communications University,

MIS2502: Data Analytics Association Rule Mining David Schuff

Collaborative Filtering With Decoupled Models for Preferences and Ratings Rong Jin 1, Luo Si 1, ChengXiang Zhai 2 and Jamie Callan 1 Language Technology.

Mining Dependent Patterns

Recommender Systems Session I

WSRec: A Collaborative Filtering Based Web Service Recommender System

Preface to the special issue on context-aware recommender systems

Methods and Metrics for Cold-Start Recommendations

Martin Rajman, Martin Vesely

Association Rule Mining

Chapter 8 Interval Estimation

Frequent patterns and Association Rules

MIS2502: Data Analytics Association Rule Mining

MIS2502: Data Analytics Association Rule Mining

Data Science in Industry

Fusing Rating-based and Hitting-based Algorithms in Recommender Systems Xin Xin

DESIGN OF EXPERIMENT (DOE)

MIS2502: Data Analytics Association Rule Learning

Charles Tappert Seidenberg School of CSIS, Pace University

Presentation transcript:

Evaluation of Recommender Algorithms for an Internet Information Broker based on Simple Association Rules and on the Repeat-Buying Theory WEBKDD 2002 Edmonton, Alberta, Canada Michael Hahsler Andreas Geyer-Schulz

Page 2 Michael Hahsler WEBKDD 2002, Edmonton, Canada Introduction Research questions – Are frequent itemsets (association rules) useful to build recommender systems for information brokers? – How does a recommender based on association rules perform compared to a recommender based on the repeat-buying theory known from marketing research? Framework for evaluation Item-to-item correlation type recommender Association rules / Repeat-buying theory Experimental setup Results and conclusion

Page 3 Michael Hahsler WEBKDD 2002, Edmonton, Canada Framework for Evaluation added (Fayyad et al. 1996)

Page 4 Michael Hahsler WEBKDD 2002, Edmonton, Canada Performance Measures for Recommender Algorithms From Machine Learning – Accuracy – Coverage – MAE From Information Retrieval – Precision / recall – F-measure (F1) / E-measure Other measures – Receiver Operating Characteristic (ROC) – Mobasher's R

Page 5 Michael Hahsler WEBKDD 2002, Edmonton, Canada Item-to-item Correlation Type of Recommender Recommendations can be based on many sources of information (e.g. Ansari et al 2000): – Expert evaluations – Expressed preferences – Individual characteristics – Item characteristics (content based) – Expressed preferences of other people (collaborative) We analyze non-personalized transactions where items are used together. explicit, observations

Page 6 Michael Hahsler WEBKDD 2002, Edmonton, Canada Association Rules (Agrawal et al, 1993) Support-Confidence Framework Model assumptions: none We use only Rules with 1 item in the antecedent for recommendations Criticism: confidence ignores the frequency of the consequent. Other measures of interestingness are – Lift - symmetric measure of co-occurrence (deviation from independence) – Conviction - deviation of the implication from independence

Page 7 Michael Hahsler WEBKDD 2002, Edmonton, Canada Repeat-buying Theory (Ehrenberg 1988) Model assumptions: – Stationary (in the analyzed time period) – "Purchases" of items follow independent Poisson processes – Distribution of the parameters follows a Gamma- distribution – Used for panel data for consumer products (scanner data) LSD (NBD) gives the probability that an item (combination of items) is "purchased" 1,2,...,r times by pure chance.

Page 8 Michael Hahsler WEBKDD 2002, Edmonton, Canada Repeat-buying Theory (Recommender) Users "purchase" some items more often together than predicted Filter outliers as potential recommendations Outliers Co-occurence of 2 Web pages from out Dept.'s home page in the Web Server's log

Page 9 Michael Hahsler WEBKDD 2002, Edmonton, Canada Experimental Setup (Info. Broker) Educational Internet Information Broker for lecture notes, research material, enrollment information,...

Page 10 Michael Hahsler WEBKDD 2002, Edmonton, Canada Experimental Setup (Data Set) Available Data: 6 months of transaction data (January - June 2002) Transactions: Items: 3774 Co-occurrences: Data Set: co-occurrence lists for 300 items randomly selected 9 users from the target group evaluated 1661 co- occurrences 561 "recommend" "don't recommend"

Page 11 Michael Hahsler WEBKDD 2002, Edmonton, Canada Experimental Setup (Evaluated Data)

Page 12 Michael Hahsler WEBKDD 2002, Edmonton, Canada Results (Support for Association Rules) Low support necessary for reasonable recall

Page 13 Michael Hahsler WEBKDD 2002, Edmonton, Canada Results (Lift / Conviction) Lift / conviction don't perform sig. better than confidence.

Page 14 Michael Hahsler WEBKDD 2002, Edmonton, Canada Results (Precision, Accuracy) Perform better than a random choice Support-Confidence sensitive to misspecification (conf)

Page 15 Michael Hahsler WEBKDD 2002, Edmonton, Canada Results (MAE) Support-Confidence sensitive to misspecification (conf) Repeat-buying seems more robust

Page 16 Michael Hahsler WEBKDD 2002, Edmonton, Canada Results (Parameter of the Repeat-buying Recommender) Expected error II rate is below the avg. rate (exception thresholds 0, 0.1, 0.2)

Page 17 Michael Hahsler WEBKDD 2002, Edmonton, Canada Conclusion Frequent itemsets represent useful recommendations Accuracy >70%, Precision 60-90% Both algorithms perform very similar with adequate parameters. Repeat-buying algorithm seems more robust to misspecification Model assumptions of the repeat-buying theory need to be checked. Datasets from other applications are needed to confirm findings