How Wikipedia Really Works, and What This Means for the Nature of "Truth" Amy Bruckman Associate Professor

Great Uncle Oscars Wikipedia Page Oscar Brodney –Born February 18th, 1907 –Hollywood screenwriter –Nominated for an Oscar Screenplay for The Glenn Miller Story –Other credits: Harvey Tammy and the Bachelor Francis the Talking Mule Abbott & Costellos Mexican Hayride Etc. –I have edited his Wikipedia page On my watchlist

Outline Epistemology overview –The social construction of knowledge How reliable is Wikipedia? Who contributes to Wikipedia & why? Wikis for writing-to-learn –Science Online (Andrea Forte) Conclusion: a teachable moment

Understanding Wikipedia How many people have ever used Wikipedia? Have you edited Wikipedia? Do you have a watch list? The problem with Wikipedia is that it only works in practice. In theory, it can never work. (New York Times, 4/23/07)

What Makes Wikis Unique? English Wikipedia is the #8 most popular website (as of 4/18/11) –Ten Wikipedias have over 500,000 articles In order: English, German, French, Italian, Polish, Dutch, Portugese, Russian, Spanish, Japanese Constructionist learning (Papert, Resnick) –Learning by working on personally meaningful projects –Learning through design and construction activities –Low barrier to entry –Easy learning curve –No ceiling Extremely light weight –Small differences in accessibility change user behavior Collaboration on a large-scale

The Source People Use First Thanks to Googles page rank algorithm, Wikipedia often pops up first Source most people use first Is this a good thing?

Crisis in Epistemology People dont know what to believe They are asking critical questions about their sources … sometimes

What is Truth? Epistemology is the study or a theory of the nature and grounds of knowledge especially with reference to its limits and validity (Websters Online) Spectrum from: –Objectivism The world exists and is knowable through our senses. –Subjectivism We are limited by our subjective perceptions. –You may not really be there.

The Sane Middle Ground Pragmatic Realism (Hilary Putnam) –The would is only accessible through our subjective perceptions –But theres a strange correlation between our perceptions –Thats because the world is actually there

How Do We Resolve Conflict about What is True? Sociology of science (Woolgar, Latour) says: –Knowledge is socially constructed –Our best guess at reality is what we agree is true The more people agree, the more sure we are –Peer review expresses this process Scientists form a knowledge-building community

Wikipedia Supports Knowledge-Building Discourse How reliable is an article? –How many people have edited it? –How many people watch it? –Spans a spectrum from total trash to much more reliable than refereed journal articles Postive example: biography articles of new supreme court nominee, new pope Negative example: the Massachusetts State House

Evaluations Comparison of Wikipedia and Brittanica by Jim Giles (Nature) –Average Wikipedia article has 4 errors –Average Brittanica article has 3 errors Most vandalism is reverted in second (Viegas et al) Stable versions in use in German Wikipedia, being tested on English Wikipedia –Changes not visible to unlogged-in users til screened by a human for obvious vandalism

WikiTrust How can users know what is reliable? Research by Adler, Chatterjee, de Alfaro, et. al. Orange text is less reliable –Prime ministers name is changed. Fjog is Danish for fool. Assigning Trust to Wikipedia Content, Proceedings of WikiSym 2008

Historical Biography Historian Roy Rosenzweig compares Wikipedia, Encarta, American National Biography Online (scholarly) –Sample of 52 people in ANBO 50% in Wikipedia 20% in Encarta –Articles are longer in ANBO Wikipedia articles are 25% as long as ANBO Encarta articles are 25% as long as Wikipedia Errors: –ANBO: 1; Wikipedia: 4; Encarta: 4 Source: Can History Be Open Source? by Roy Rosenzweig. The Journal of American History Volume 93, Number 1 (June, 2006): (Available online.)

Differences of Style All three show biases in how much coverage each subject gets –Wikipedia on Woodrow Wilson: 3200 words; Isaac Asimov: 3500 Compare ANBO and Wikipedia on Lincoln –Both accurate –ANBO has elegant prose, richer contextualization, confident judgment –Wikipedias neutral point of view policy (NPOV) leads to waffling –Wikipedia is more factualist –Wikipedia has fun, colorful details

Emphasis on References Statements without references are deleted quickly Online references are checked more closely than print ones Loops implying a reference where there is none do happen –Wikipedia info has no cite –Journalist cites Wikipedia article –Someone fixing Wikipedia cites journalist Reference syntax is awkward –We created a tool to help: ProveIt

ProveIt

Why Do People Contribute? Interviews with 21 Wikipedians Becoming a part of Wikipedia is a process of: –Legitimate peripheral participation (Lave & Wenger), in a –Knowledge-building community (Scardamalia & Bereiter) Becoming Wikipedian, Andrea Forte, Susan Bryant et. al. (Group 2005 )

Legitimate Peripheral Participation (LPP) Lave and Wenger (1991) –Example: tailors in West Africa –Start by sweeping floor Legitimate: floor needs to be swept Peripheral: watching activity around them –When someone finally says sew this seam, theyre ready Seen it over and over Real-world learning is often more like this than like school

Becoming Wikipedian Typical progression: –Start with one edit –Get a watch list –Begin to care about the site as a whole LPP in a knowledge-building community Activity Theory –Transformation of tools, rules, division of labor, object, community

Power and Authority on Wikipedia It is NOT a free for all –How it really works matters Interview study with 11 people in administrative roles on Wikipedia –Nature of power and authority –How conflicts are resolved Brett Favre: Profootballreference.com lists Farve as having 8224 career passing attempts, while the official Packers website and NFL.com list him as having An edit war ensues over the 1 attempt leading to an editor getting indefinitely banned. Sockpuppeting followed, including "aging" accounts to circumvent semi-protection. All over 1 passing attempt... In a 16 year hall of fame career. His name is still spelled weird. Forte & Bruckman Scaling Consensus: Increasing Decentralization in Wikipedia Governance (HICSS 2008)

Increasing Decentralization Policy –Creation Main policy creation slowing Moving into WikiProjects –Interpretation & enforcement Jimmy --> ArbCom --> Admins 1785 admins (as of 4/18/11) Complicated process –Example: British climatologist William Connelley »Broke rules »Penalty from ArbCom: limited to one revert per day »Penalty not enforced by Admins Becoming an Admin –Differs by language –Criteria getting harder

How is Wikipedia Organized? Wikipedia governance is accomplished by articulating social norms as policies and guidelines Processes for defining and enforcing policies and guidelines are becoming increasingly decentralized Wikipedia cant function as one communitydiscrete sub-communities have emerged: WikiProjects

Decentralization: WikiProjects Group that focuses on a particular content area Case Study: WikiProject Military History –Gather subject-matter experts in one place for networking and collaboration –Establish subject-specific guidelines –Support review processes that arent possible site- wide –Encourage contributions People know their stubs will be nurtured, their articles protected –Does not provide mediation for disputes Lack of local enforcement a potential problem

Deviant Behavior: The Experiences of Banned Users Measurements of author contributions suggest that banned Wikipedia users sometimes have a history of good contributions (Adler et. al., 2008) Whats going on here? Ongoing study examines the experiences of people who have been banned from the site

Emergent Themes Perception among banned users is that the experience of editing Wikipedia has changed –Early on, contributing content was sufficient to be a valued community member –Increasingly, community members goals need to include negotiating complicated social structures –Maintaining social ties is now an important component of being a Wikipedia contributor

Science Online: Motivation What if we created a version of Wikipedia written by high-school students? –Focus is on science PhD work of Andrea Forte –MLIS UT Austin –Assistant Professor, Drexel University

Wiki as a Construction Kit Constructing text is a powerful learning activity –Writing-to-learn (Scardamalia and Bereiter, Emig, Britton, etc.) We can design environments that support specific writing activities Design challenges –Support critical citation – media literacy skills –Make it fit in the classroom

Three Classroom Studies AY 2005: Public Policy AY 2007: Environmental Science (public school) AY 2008: Biology (private school) Software improvements to MediaWiki, based on pilot study findings: –Add support for citations MediaWiki mods and Firefox plugin –Teacher tools

Constructionist Learning at Work

Good News & Bad News Constructionism predicts precisely the kind of engagement and learning that we saw when students wrote on the wiki –The creation and sharing of a personally meaningful public artifact as a learning activity leads to deep engagement Problematic fit with AP exam Theyre student-made so there could be a lot more information on the wiki than we actually need to know for the test. – Sylvia The level of thinking that I guess I had them do and work on some of those is probably deeper than the curriculum requires for the assessment. – Mr. Grant Contrast public vs. private school students

Crisis in Epistemology How do we know if something is true? One answer: degree of review –Highly edited Wikipedia article is arguably more reliable than a refereed journal article Experts dont agree Students are confused Real answer is multi-dimensional –How accurate is Wikipedia? Well, which article?

A Teachable Moment Role for librarians in bringing clarity Critical moment where the nature of proof is at issue –Its always been an issue –Now everyone is listening

Acknowledgments ELC students: –Betsy DiSalvo, Casey Fiesler, Andrea Forte (Drexel University), Kurt Luther, Sarita Yardi –Jordan Patton, Vanessa Larco ELC sponsors: –IBM, Intel, Microsoft, Pitney Bowes, Ricoh –The National Science Foundation –US Department of Education For more info: –My research group: –ProveIt: –My blog: –Wikimedia Foundation FAQ for librarians: