Reputation Systems Guest Lecture Paul Resnick Associate Professor Univ. of Michigan School of Information

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Learning Objectives n Understand –What a reputation system is –Theory about when and why it should work –Open research questions n Participate in design –Recognize situations when it might be helpful –Raise some of the difficult design challenges

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Outline n What is a reputation system? n Theory: when/why they should work n Empirical results n Design space n Case study: NPAssist recommender

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Definition n A Reputation System… –Collects –Distributes –Aggregates n …information about behavior

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Examples n BBB n Bizrate n eBay n Expertise sites –Epinions “top reviewers” –Slashdot karma system

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Why Reputation Systems n Interacting with strangers n Sellers (Exchange Partners) Vary –Skill –Effort –Ethics

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Other Trust-Inducing Mechanisms in E-commerce n Insurance n Escrow n Fraud Prosecution

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu How Reputation Systems Should Work n Information n Incentive n Self-selection

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Some Issues n Anonymity n Name changes n Name trades n Lending reputations n Eliciting evaluation n Honesty of evaluations

Anonymity Analysis

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu 1L Pseudonyms n Third-party issues pseudonyms –No cost –Not replaceable –Reveal name to third party –Don’t reveal mapping of name to pseudonym

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Empirical Results: eBay n Feedback is provided n It’s almost all positive n Reputations are informative n Reputation benefits –Effect on probability of sale –Effect on price

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Provision of Feedback n Negatives: paid but did not receive; seller cancelled; not as advertised; communication n Neutrals: slow shipping, not as advertised, communication

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Feedback Profiles of Buyers and Sellers

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Predicting Problematic Transactions n Logistic Regression f(0,0) = 1.91% f(100,0) =.18% f(100,3) =.53% N = Beginning Block Number 0. Initial Log Likelihood Function -2 Log Likelihood Log Likelihood Dependent Variable.. NEGNEUT Variables in the Equation Variable B S.E. Wald df Sig R Exp(B) LNNPOS LNPOS Constant

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Predictive Value

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Some Recently Completed Work n Experiment: does reputation affect profit? –Many positives: Yes, but only a little (8.1%) –One or two negatives: No n Incentives for quality feedback provision –Can pay based on agreement among raters

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Studies Currently Underway n Feedback provision (empirical) –Reciprocation, altruism, and free riding n Dynamics: learning and selection (empirical) n Geography: trust and trustworthiness by state

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Design Space n Rating scales n Aggregation of ratings n Who rates? n Incentives for raters n Identification/Anonymity –Exchange partners –Evaluation providers

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Case Study n Goal: help Michigan non-profits select consultants and other service providers n Is this a good candidate for a reputation system?

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Case Study n Goal: help Michigan non-profits select consultants and other service providers n Is this a good candidate for a reputation system?  Interacting with strangers  Sellers (Exchange Partners) Vary  Skill  Effort  Ethics

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Case Study Design Choices n Rating scales n Aggregation of ratings n Who rates? n Incentives for raters n Identification/Anonymity –Exchange partners –Evaluation providers

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Summary n RS inform, incent, select n Opportunity for RS: interactions with strangers n Design space –Scales, aggregation, raters, incentives, anonymity