Computer-Mediated Communication Reputation (Part II)

Distilling reputation from various sources of information What is the difference between reputation and opinion? Some implied notions of reputation: predicting future behavior? Public versus private reputation? Subjectivity Issue: interpretation of the “same facts” still up for interpretation 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Computer-Mediated Communication — Cheshire & Fiore “Reputation” systems Explicit Implicit Behavior Ratings by others Derived from behavior Join Date: Mar 2004 Posts: 22 What are some problems with implicit “reputation” information? Hard to be sure what it means, whether it tells you anything useful. Direct experience 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Computer-Mediated Communication — Cheshire & Fiore Resnick et al. 2006 Effect of strong reputations on revenues compared to those without reputation Effect of “negatives” in a brief reputation on outcome of revenue 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Matched Pairs by Different Sellers 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Computer-Mediated Communication — Cheshire & Fiore Reputation Effects Do strong reputations matter? What is the impact of negative reputation marks in a mixed reputation system such as eBay? 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Critiquing Methodological Approaches to Similar Problems 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Two approaches to studying online reputation system (Ebay) Why study it in a lab? Why study it in a field experiment? Assignment: maybe random, maybe deliberate, but controlled by the experimenter. 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Computer-Mediated Communication — Cheshire & Fiore The Validity Problem Internal Validity External Validity Internal Validity: It is clear what is being measured and we try hard to account for other factors of influence External Validity: Generalizability…is this generalizable to other contexts other than the lab? 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

External Validity: Generalizing From Experiments ? 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Ecological Validity: Approximation of Real-Life Activity Ecological Validity is often viewed as a “Threat” to external validity; even a subset of the larger ‘external validity problem’. Resnick et al. Yamagishi et al. 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Computer-Mediated Communication — Cheshire & Fiore Resnick et al. 2006 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Computer-Mediated Communication — Cheshire & Fiore “Reputation” systems Explicit Implicit Experience Ratings by others Derived from behavior Join Date: Mar 2004 Posts: 22 What are some problems with implicit “reputation” information? Hard to be sure what it means, whether it tells you anything useful. Direct experience 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Computer-Mediated Communication — Cheshire & Fiore 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

How does a “good” poster look? More active posters are more interactive Regular, active (but not overactive) participation: good Posting in too many groups or dominating threads: bad More posts, more threads <==> greater fraction are replies Regular, active associated with wanting to read more in the future, respect, trust Too many groups, dominating threads associated with not wanting to read more in the future. (Spam; being a jerk) 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Computer-Mediated Communication — Cheshire & Fiore Why do this? Predicting how real people would have rated if we had explicit ratings on a huge scale. 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Other Implicit ‘Reputation’ Information in CMC? 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Another side of reputation… “ Internet markets also have significant advantages in establishing reputations … any information that is gleaned can be near costlessly tallied on a continuing basis … [and] that information can be near costlessly transmitted to millions of potential customers. — Resnick et al. 2006, p. 80 ” 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

Computer-Mediated Communication — Cheshire & Fiore The CMC and ‘Offline’ Reputation Link: Emergent Reputation Systems and Identity 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

So, Is ‘Reputation’ Obsolete? 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore

As designers, what can we learn from all of this? What kind of community do you have (or are you trying to foster)? When and Why to use Pos/Neg/Mixed/Hybrid Reputation Systems? What behavior(s) do you want to encourage, reward, punish? Consider the “unintended consequences” of implicit information Just because you build a system to be interpreted a certain way doesn’t mean that the user will agree… Others? 9/17/2018 Computer-Mediated Communication — Cheshire & Fiore