1. School of Information University of Michigan Expertise Sharing Dynamics in Online Forums Lada Adamic joint work with Jun Zhang, Mark Ackerman, Eytan Bakshy, Jiang Yang School of Information, University of Michigan

2. Outline of talk Knows Knowledge iN

3. Oozing out knowledge Knowledge In ``Knowledge search is like oozing out knowledge in human brains to the Internet. People who know something better than others can present their know-how, skills or knowledge'' NHN CEO Chae Hwi-young “(It is) the next generation of search… (it) is a kind of collective brain -- a searchable database of everything everyone knows. It's a culture of generosity. The fundamental belief is that everyone knows something.” -- Eckart Walther (Yahoo Research)

4. Limitations of Current Systems The Response Time Gap The Expertise Gap Difficult to infer reliability of answers Automatically ranking expertise may be helpful.

5. Related work Analysis of online communities NetScan (Smith, Fisher, et al. at Microsoft) Social network analysis (LiveJournal, blog communities) Motivations of online participation (Lakhani & Hippel, Kraut) Graph-based ranking algorithms PageRank, HITS, etc. Expertise sharing studies Expertise recommenders ContactFinder (Krulwich et al.), Answer Garden (Ackerman) Small Blue (Lin) Automatic evaluation of expertise levels Using different text resources (Kautz, et al, and a lot of others) Using email networks (Campbell et al.) Best answers correspond to best questions (Agichtein et al.)

6. Java Forum 87 sub-forums 1,438,053 messages community expertise network constructed: 196,191 users 796,270 edges

7. Constructing a community expertise network A BC Thread 1 Thread 2 Thread 1: Large Data, binary search or hashtable? user A Re: Large... user B Re: Large... user C Thread 2: Binary file with ASCII data user A Re: File with... user C A B C 1 1 A BC 1 2 A BC 1/2 1+1//2 A B C 0.9 0.1 unweighted weighted by # threads weighted by shared credit weighted with backflow

8. Uneven participation number of people one replied to ‘answer people’ may reply to thousands of others ‘question people’ are also uneven in the number of repliers to their posts, but to a lesser extent

9. Not Everyone Asks/Replies Core: A strongly connected component, in which everyone asks and answers IN: Mostly askers. OUT: Mostly Helpers The Web is a bow tieThe Java Forum network is an uneven bow tie

10. fragment of the Java Forum

11. Relating network structure to Java expertise Human-rated expertise levels 2 raters 135 JavaForum users with >= 10 posts inter-rater agreement (  = 0.74,  = 0.83) for evaluation of algorithms, omit users where raters disagreed by more than 1 level (  = 0.80,  = 0.83) LCategoryDescription 5Top Java expertKnows the core Java theory and related advanced topics deeply. 4Java professionalCan answer all or most of Java concept questions. Also knows one or some sub topics very well, 3Java userKnows advanced Java concepts. Can program relatively well. 2Java learnerKnows basic concepts and can program, but is not good at advanced topics of Java. 1NewbieJust starting to learn java.

12. Algorithm Rankings vs. Human Ratings simple local measures do as well (and better) than measures incorporating the wider network topology Top K Kendall’s  Spearman’s  # answers z-score # answers indegree z-score indegree PageRank HITS authority 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

13. automated vs. human ratings # answers human rating automated ranking z # answers HITS authority indegree z indegree PageRank

14. Modeling community structure to explain algorithm performance Control Parameters: Distribution of expertise Who asks questions most often? Who answers questions most often? best expert most likely someone a bit more expert ExpertiseNet Simulator

15. Simulating probability of expertise pairing suppose: expertise is uniformly distributed probability of posing a question is inversely proportional to expertise p ij = probability a user with expertise j replies to a user with expertise i 2 models: ‘best’ preferred‘just better’ preferred j>i

16. Visualization Best “preferred”just better

17. Degree correlation profiles best preferred (simulation)just better (simulation) Java Forum Network asker indegree

18. It can tell us when to use which algorithms Preferred Helper: ‘just better’ Preferred Helper: ‘best available’

19. Different ranking algorithms perform differently In the ‘just better’ model, a node is correctly ranked by PageRank but not by HITS

20. Summary for a focused expertise sharing forum Expertise Networks have interesting characteristics A set of useful metrics Ranking algorithms are affected by network structures Simulation as an analysis tool There are rich design opportunities Find experts with the help of structural information (and content analysis) Predict good answers Re-order questions/answers to match expertise UIST2007: “Expertise-Level based Interface Personalization for Online Help-seeking Communities”

21. Looking at diverse sets of question-answer forums (Yahoo Answers) Expertise across different topics Everyone is not a Java expert, but everyone knows something... cars & transportation maintenance & repairs beauty & style hair w/ Jun Zhang, Eytan Bakshy, Mark Ackerman

22. Yahoo! Answers A community-driven knowledge market site, allowing users to ask and answer questions About 80 million answers since launch in Dec 2005 Similar sites: Naver, Baidu-knows our sample 1 month 8,452,337 answers 1,178,983 questions unique repliers: 433,402 unique askers: 495,414 users who are both askers and helpers: 211,372

26. length of answer vs. # of answers ign Jokes & Riddles Parenting Polls &Surveys Baby Names Wrestling Small business Genealogy thread length content length

27. network structure of a technical forum programming & design

28. network structure of a discussion forum politics

29. Clustering categories: technical, advice/support, discussion

30. Breadth of user’s post patterns Sum entropies at each level cars & transportation maintenance & repairs beauty & style hair 0.3 0.7 0.1 0.2 L=1 L=2 car audio thanks to Mark Newman

31. user with entropy = 0 all answers are in the Pets > Dogs subcategory

32. user with medium entropy _Travel *1* _Asia_Pacific *1* _Pets *1* __General___Pets *1* _Sports *18* __Cricket *18* _Society___Culture *19* _Cultures___Groups *1* __Religion___Spirituality *18* _Arts___Humanities *1* __Philosophy *1* ===== entropy ====== L1 entropy: 0.99 L2 entropy: 1.09 total: 2.08

33. user with medium entropy _Beauty___Style *32* __Makeup *7* __General___Beauty___Style *3* __Fashion___Accessories *12* __Hair *7* _Skin___Body *3* _Family___Relationships *8* __Singles___Dating *6* __Friends *1* __Family *1* ===== entropy ====== L1 entropy: 0.50 L2 entropy: 1.83 total: 2.33 focused on a few top level categories, during the sampled period

34. high entropy user ===== entropy ====== L1 entropy: 2.64 L2 entropy: 3.11 total entropy: 5.75 user with high entropy

35. Entropy and “expertise” consider 1 st level categories and 2 nd level entropies (H L=2 ) within them individually

36. Predicting best answers  lengthier replies

37. Predicting best answers ProgrammingMarriagewrestling answer length+++ thread length--- replier best answers +++ replier answers--- R2R2 0.7290.6930.692

38. people who answer about X, ask about Y

39. Yahoo Answers Summary Everyone knows something Differing network characteristics by category type Focus corresponds to performance, primarily for “factual categories” Predicting best answers easiest for those categories

40. Competing to share expertise on Witkey sites

41. when money is given to best answers previously (Java Forum): asker -> replier in Task CN: submitter -> winner

42. task prestige network If winners of other tasks lose in this task, this task is more prestigious...

43. network of task participants Same two users compete twice: same winner 77% of the time (compared to 1/2 chance) Same two users compete 3x: same winner all 3 times in 56% of the cases (compared to 1/4 chance)

44. Taskcn knowledge sharing community Askers offer cash reward for best “solution” to task w/ Jiang Yang and Mark Ackerman

45. does money matter? more views not more submissions on average small negative correlation with task PageRank task may be a bit more difficult average prestige negatively correlated with number of participants but winner’s prestige positively correlated does the number of participants matter?

46. predicting who will win Competing against few other and having a good track record is predictive of future wins Some users “learn” to improve their odds of winning by selecting less popular tasks

47. users learn to choose less competitive tasks

48. successful users shorten interval between wins

49. summary of Taskcn findings Amount of reward does not correlate with # submissions expertise level It does correlate with the number of views Can infer expertise from new formulation of expertise networks Can predict future probability of winning Overall, users don’t increase their winning rate over time successful users choose less popular tasks decrease intervals between wins

50. for more info Competing to share expertise Yang, J., Adamic, L., Ackerman, M.S., ICWSM2008 Examining knowledge sharing on Yahoo Answers Adamic, L., Zhang, J., Ackerman, M.S., Knowledge Sharing and Yahoo Answers: Everyone knows something, WWW’08 ExpertiseRank algorithms and evaluations Zhang, J., Ackerman, M.S., Adamic, L., Expertise Networks in Online Communities: Structure and Algorithms, WWW’07 Simulations of expertise networks Zhang, J., Ackerman, M.S., Adamic, L., CommunityNetSimulator: Using Simulations to Study Online Community Network Formation and Implications, C&T2007 QuME: A Mechanism to Support Expertise Finding In Online Help- seeking Communities J. Zhang, M. S. Ackerman, and L. A. Adamic, UIST2007, Newport, RI, 2007. Lada Adamic ladamic@umich.eduladamic@umich.edu http://www-personal.umich.edu/~ladamic

51. thanks! Jun Zhang junzh@umich.edujunzh@umich.edu http://www-personal.si.umich.edu/~junzh Mark Ackerman ackerm@eecs.umich.eduackerm@eecs.umich.edu http://www.eecs.umich.edu/~ackerm/ Jiang Yang yangjian@umich.edu http://www.yangjiang.us/ Eytan Bakshy ebakshy@umich.edu http://www-personal.umich.edu/~ebakshy Thanks to: ARI, Intel, NSF 0325347