Presentation is loading. Please wait.

Presentation is loading. Please wait.

WINLAB Narayan B. Mandayam Collaborators: S. Anand*, O. Arazy †, and O. Nov* *NYU Poly †University of Haifa & Alberta School of Business Acknowledgment:

Similar presentations

Presentation on theme: "WINLAB Narayan B. Mandayam Collaborators: S. Anand*, O. Arazy †, and O. Nov* *NYU Poly †University of Haifa & Alberta School of Business Acknowledgment:"— Presentation transcript:

1 WINLAB Narayan B. Mandayam Collaborators: S. Anand*, O. Arazy †, and O. Nov* *NYU Poly †University of Haifa & Alberta School of Business Acknowledgment: NAKFI Game Theoretic Modeling of Online Knowledge Creation in Wikipedia

2 WINLAB DNA of Silicon Brains?  Peer Production Large number of individuals co-create knowledge Examples- Citizen science projects, Citizen journalism, Open source software (Linux Kernel Development), Wikipedia  From Individual Informed Brains to a Society Scale Informed Brain Uncovering the DNA of Social Knowledge Creation  Inspired by similar efforts in the life sciences, such as the Human Genome Project, we seek to explore basic patterns or “building blocks” of the process through which individual human brains co-create society-scale silicon brains the relationship between sequential patterns of these building blocks and attributes of the resulting silicon brains 2

3 WINLAB Wikipedia as a Silicon Brain  Among the most popular information repositories on the web 18B page views (500M unique visitors/month) and counting 6 th most popular after Google, Microsoft, Facebook and Yahoo  The largest “collaborative effort” in human history Content: ~34M articles (~5M in English) Editors: ~23M editors/contributors (130K active in last month, 30K with at least 5 edits) Administrators: ~1400  Wikipedia is a silicon-based “brain” - a large scale knowledge repository created collaboratively  people contribute their knowledge, expertise and energy  a common pool (information good) accessible to everybody 3

4 WINLAB DNA of Wikipedia  Wikipedia is built on Wiki technology A web-based collaborative authoring tool  A contributor can add content, add to or delete existing content Similar to a Google doc or multi-user editing with ‘Track Changes’ in MS-Word  Each “edit” made by a user, creates a new version of the wiki page All versions are tracked in the “History” page (adopting version control principles from software development) 4 WikiDNA

5 WINLAB DNA of Wikipedia 5

6 WINLAB Characteristics of Collaboration in Wikipedia 6  Contributors Motivations Reputation enhancement, Ego, Express one’s opinions Contributors’ goal is to increase the content that they “own” “Competitive” in nature, “Refactoring” of others’ contributions  Comprehensive Array of Governance Mechanisms Automated tools Social norms and procedures Quality management, Conflict Management, Status management Act to ensure information credibility, unintentional biases, intentional biases (“Wiki lobbying”), vandalism  Product Quality Information Quality, Accuracy, Completeness, Objectivity, Representation “5 pillars”

7 WINLAB Modeling Contributor Activity and Governance Mechanisms in Wikipedia  Noncooperative Game for Modeling Contributor Interactions  Stackelberg Game for Modeling Governance Mechanism Interactions with Contributor Actions 7  Derive insights from model  Validate model with data from Wikipedia

8 WINLAB Non-cooperative game amongst contributors 8

9 WINLAB Nash equilibrium of the noncooperative game 9

10 WINLAB Solution and Implications of Nash equilibrium 10

11 WINLAB How to measure quantities in model from data? 11

12 WINLAB How to model governance? 12

13 WINLAB Sample Data and Quality Assessment  Representative sample of 89 Wikipedia articles used in (Arazy et al., 2011, 2013) Stratified sampling by topic (e.g., culture, geography) For each article, details of every edit made (and the contributor making it) from article’s inception to January 2007; the average article: # of edits = 91 # of unique contributors = 49  Set includes measures of information credibility (7-point Likert scale) 5-6 students independently analyze each page and produce detailed reports by comparing to external resources Senior university librarians independently analyze articles (employ students’ report + other sources) and rate: information quality, accuracy, completeness, objectivity, and representation Senior librarians sit together, argue differences, and arrive on consensus 13

14 WINLAB  1956 Trans-Canada Air Lines accident, Abreu Camp, Alcohol 120% Alpha Iota Omicron Ancient DNA Anime Vegas Antonio Inoki vs Renzo Gracie Arms sales to Iraq 1973-1990 Art Finley Ashton, Illinois Australian contribution to the 1991 Gulf War Battle of Magdhaba Belle Glade, Florida Bess of Hardwick Biphenyl Blue and white (porcelain) BMW 3 Series Briccriu Cameron Bright Canadian federal election, 1930 Chandrashekarendra Saraswati Chikkamagaluru district Commonwealth Scientific and Industrial Research Organisation Construction Core mantle boundary Dhol Dianthus Wiki pages for Quality Assessment (1/2) 14  Dragoon Sniper Rifle Dzhezkazgan Edouard Pingret Electronic lock Eleonore Duplay Empire Theatres Fawsley Ferdinand III, Grand Duke of Tuscany Fiat 1300/1500 Flying car Frunzik Mkrtchyan Gneisenau class battlecruiser Graz Great Dismal Swamp Greenup, Illinois Hero of the Soviet Union High pressure area High-end audio cables Ier arrondissement In a Fix Irving Kanarek Jacobi identity Jay Fiedler JE Khopesh

15 WINLAB Wiki pages for Quality Assessment (2/2) 15  K-pop Ludwigsfelde Magnetohydrodynamic drive Medieval churches of York Meow Wars Merrimac Ferry Mr. Potato Head Multitrack recording Myles Brand Newtonian telescope Nueces massacre Operation Osprey Orange Revolution Otis, Colorado Perfect game (bowling) Peter Jackson Philippe, comte de Paris Pine Lawn, Missouri Pledge of Allegiance Poisson algebra Politics of Alberta  R. Lee Ermey RBC Center Sandkings (novelette) Self-service password reset Shoulder strap Smelly Cat Spoiler effect Stephen Goldsmith Student unionism in Australia System Requirements Specification The Bad Plus The Real World: Austin Thrombosis Timesplitters Tippmann A-5 Treaty of Mutual Cooperation and Security between the United States and Japan Uncle Vanya Unidentified submerged object Urbanization in Africa WDC 65C02 Well-Tempered Clavier William Holborne Winged bean

16 WINLAB Data: Governance, Quality & Unbiasedness  Average quality is correlated to reducing difference between max and min fractional ownerships (bias) 16 Quality vs Difference between maximum and minimum fractional ownership Average Quality “5 Pillars” (ensuring no one group takes over ownership of content)

17 WINLAB A Stackelberg Model for Wikipedia 17 Governance Model Contributor Model

18 WINLAB  1000 pages from January 2012 dump of English Wikipedia  219,811 distinct contributors  Lifespan: 129 – 4078 days, Avg. duration: 2681 days (~7.35 years)  25 Topical Categories: Agriculture, Arts, Business, Chronology, Concepts, Culture, Education, Environment, Geography, Health, History, Humanities, Humans, Language, Law, Life, Mathematics, Medicine, Nature, People, Politics, Science, Society, Sports, Technology  4 Maturity Strata (Number of Revisions): 1-10, 11-100, 101-1000, 1000+  40 articles in each topical category, 250 articles in each maturity stratum Wikipedia Articles for Empirical Data Validation 18

19 WINLAB Validating Analysis against Data-1 19 Abreu Camp Paris

20 WINLAB Validating Analysis against Data -2 20

21 WINLAB Validating Model: Estimation Error & Significance 21 Error 11-15%, Significance Test p =0.03

22 WINLAB  Mathematical Models for Collaborative Knowledge Creation  Wikipedia as an example of a silicon brain  Developed a Game Theoretic Model for Knowledge Creation in Wikipedia  Non cooperative game for contributor interactions  Users noncooperatively maximize their content ownership  Effort measured as function of Levenshtein distance of edits  Stackelberg model for influence of governance mechanisms on editors  Governance factor is the implicit outcome of emphasis on “5 pillars” and “product quality”  Reducing difference between maximum and minimum fractional ownership subject to objectivity/quality constraints Conclusions and Future Directions-1 22

23 WINLAB  Nash Equilibrium Implications  Only users who expend less than “average effort” have non- zero content ownership  Seems counterintuitive but a consequence of governance  Unintended consequences of governance: Wiki-bureaucracy, Difficult to navigate rules, Driving away contributors  Model can offer guidelines on how to moderate governance  Dynamic models that track sequential actions and interactions  Dynamic games  Evolutionary game theory  Model improvements and validation with larger data sets  Improved modeling/analysis of governance; Other user metrics Conclusions and Future Directions-2 23

24 WINLAB References-1 24  Anand S., Arazy O., Mandayam N.B., Nov O. 2013, A Game Theoretic Analysis of Wikipedia, Proceedings of the 4th International Conference on Decision and Game Theory for Security (GameSec 2013), LNCS 8252, pp. 29-44, Springer 2013  Arazy O., Stroulia E., Ruecker S., Arias C., Fiorentino C., Ganev V., and Yau T., 2010, Recognizing Contributions in Wikis: Authorship Categories, Algorithms, and Visualizations, Journal of the American Society for Information Science and Technology (JASIST), 61(6), pp. 1166-1179.  Arazy O. and Kopak R., 2011, On the Measurability of Information Quality, Journal of the American Society for Information Science and Technology (JASIST), 62(1), pp. 89-99.  Arazy, O., Nov, O., Patterson, R., & Yeo, L. (2011). Information quality in Wikipedia: The effects of group composition and task conflict. Journal of Management Information Systems, 27(4), 71-98.  Arazy, O., Yeo, L., & Nov, O. (2013). Stay on the Wikipedia task: When task‐related disagreements slip into personal and procedural conflicts. Journal of the American Society for Information Science and Technology.  Arazy O., Nov O. and Ortega F., The [Wikipedia] World is Not Flat: on the organizational structure of online production communities, the 22nd European Conference on Information Systems (ECIS’2014), Tel-Aviv, Israel, June 9-11, 2014.  Auray, N., 2012, Online Communities and Governance Mechanisms, in Brousseau E., Marzouki M., and Adel C. (eds), Governance, Regulation and Powers on the Internet, pp. 211-231  Butler, B., Joyce, E., & Pike, J. (2008, April). Don't look now, but we've created a bureaucracy: the nature and roles of policies and rules in Wikipedia. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1101-1110). ACM.  Butler, B., Sproull, S., Kiesler, S. & Kraut, R. E. 2007. Community effort in online communities: who does the work and why. In: WEISBAND, S. (ed.) Leadership at a Distance Research in Technologically Supported Work. London UK: Lawrence Erlbaum Associates  Forte, A., Larco, V., & Bruckman, A. (2009). Decentralization in Wikipedia governance. Journal of Management Information Systems, 26(1), 49-72.

25 WINLAB References-2 25  Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2013). The rise and decline of an open collaboration system how wikipedia’s reaction to popularity is causing its decline. American Behavioral Scientist, 57(5), 664-688.  Hilligoss, B., & Rieh, S.Y. (2008). Developing a unifying framework of credibility assessment: Construct, heuristics, and interaction in context. Information Processing & Management, 44(4), 1467–1484  Kittur, A., Suh, B., Pendleton, B. A., & Chi, E. H. (2007, April). He says, she says: conflict and coordination in Wikipedia. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 453-462). ACM.  Krieger, M., Stark, E. M., & Klemmer, S. R. (2009). Coordinating tasks on the commons: designing for personal goals, expertise and serendipity. In CHI ’09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1485-1494). New York, NY:  Kriplean, T., Beschastnikh, I., McDonald, D. W., & Golder, S. A. (2007, November). Community, consensus, coercion, control: cs* w or how policy mediates mass participation. In Proceedings of the 2007 international ACM conference on Supporting group work (pp. 167-176). ACM.  Navarro, G. (2001). A guided tour to approximate string matching. ACM computing surveys (CSUR), 33(1), 31-88.  Niederer, S., & Van Dijck, J. (2010). Wisdom of the crowd or technicity of content? Wikipedia as a sociotechnical system. New Media & Society, 12(8), 1368-1387  Nov, O. (2007). What motivates wikipedians?. Communications of the ACM,50(11), 60-64.  O'neil, M. 2009. Cyberchiefs. Autonomy and Authority in Online Tribes, London, UK, Pluto Press  Rafaeli, S., & Ariel, Y. (2008). 11 Online Motivational Factors: Incentives for Participation and Contribution in Wikipedia.  Schroer, J., & Hertel, G. (2009). Voluntary engagement in an open web-based encyclopedia: Wikipedians and why they do it. Media Psychology, 12(1), 96-120.  Stvilia, B., Twidale, M., Smith, L. & Gasser, L. 2008. Information quality work organization in Wikipedia. Journal of the American Society for Information Science and Technology, 59, 983-1001  Viégas, F. B., Wattenberg, M., & McKeon, M. M. (2007). The hidden order of Wikipedia. In Online communities and social computing (pp. 445-454). Springer Berlin Heidelberg.

Download ppt "WINLAB Narayan B. Mandayam Collaborators: S. Anand*, O. Arazy †, and O. Nov* *NYU Poly †University of Haifa & Alberta School of Business Acknowledgment:"

Similar presentations

Ads by Google