1. 1 When Information Technology “Goes Social” Marti Hearst UCB SIMS SLA Meeting Oct 19, 2000

2. 2 The SLA Hearst

3. 3 Standard Information Retrieval Ranking l Problem: there is a lot of useful information available. Which pieces in particular does the user want to view right now? l Procedure: –match the words that reside in documents against the words users state in their query –combinations of words are proxies for the underlying meaning of the documents

4. 4 Standard Web Search Engine Architecture crawl the web create an inverted index Inverted index user query Show results To user Search engine servers Check for duplicates, store the documents DocIds

5. Figure from Baeza-Yates & Ribeiro-Neto Document Processing Steps

6. 6 Spam l Email Spam: –Undesired content l Web Spam: –Content is disguised as something it is not, in order to »Be retrieved more often than it otherwise would »Be retrieved in contexts that it otherwise would not be retrieved in

7. 7 Web Spam l What are the types of Web spam? –Add extra terms to get a higher ranking »Repeat “cars” thousands of times –Add irrelevant terms to get more hits »Put a dictionary in the comments field »Put extra terms in the same color as the background of the web page –Add irrelevant terms to get different types of hits »Put “free beer” in the title field in sites that are selling cars –Add irrelevant links to boost your link analysis ranking l There is a constant “arms race” between web search companies and spammers

8. 8 Information Retrieval Goes Social l A new way of –using words as weapons? –using words to mean other than what they say? –subliminal authoring? l Ranking algorithms must now adopt a posture of “defensive searching”.

9. 9 Information Retrieval Goes Social Thirty years of research on ranking algorithms had never remotely considered what happens with IR goes social. For the underlying assumptions of IR, these problems are almost absurd.

10. 10 Information Retrieval Goes Social The IR side isn’t all innocent –issues of ranking sites higher in return for payoffs

11. 11 Traditional Information Filtering l At least 20 years of research on information filtering –A stream of information flows by, filter out those not of interest, or retain those of interest l Focus: how to identify which documents about a particular topic –financial news, terrorist activity l A classification problem l Usually single-user judgements only

12. 12 Information Filtering Goes Social l ABC news call ~1993. They’ve heard about categorization software. They want to identify: –news programming about sex and violence l With the WWW, rapid commercial adoption of filtering software for: – adult content. l This was not on the research radar screen. –Major use of filtering now: taste alignment. –Major technique: pooled judgements »Examples: Ringo, DirectHit

13. 13 Why Does this Happen? l Computer Scientists are not trained to think of the social interactions in the use of their systems l There wasn’t good reason to see this happening soon. –The PARC Tapestry project (CACM 35 (12), 1992) –Collaborative Filtering, but ahead of its time

14. 14 Domain Names

15. 15 Hypertext Then l Proceedings of ACM Hypertext 89 –28 papers: »Navigation, engineering, knowledge representation, implementation & interfaces, applications, IR, usability of links, fiction and writing –9 panels: »Interchanging hypertexts »Narrative and Consciousness »Lessons from ACM hypertext project »Indexing »Expert Systems »Higher Education: A Reality Check »Software Engineering »Cognitive Aspects »Confessions: What’s Wrong with our Systems

16. 16 Hypertext Then l Much discussion on –semantics of link types –navigation paths –not getting lost (still an issue!) –how to author documents l What about social implications?

17. 17 Hypertext Then l Two papers are relevant. –Amy Pearl, Sun’s Link Service: A Protocol for Open Linking »discusses use of a separate link repository to allow linking between objects that reside on different systems »simply assumes bidirectional linking »concerned with technical difficulties –Bob Glushko, Design Issues for Multi-Document Hypertexts »considers the question of whether links should be allowed outside of documents »concludes they should, but in a cautionary manner

18. 18 Course Gedanken Experiment l What happens if bi-directional links are possible? Required? l My naïve pre-social CS-y thoughts: –easier to link footnotes and their citations –easier to link papers, lectures, to author’s home page –easier to find related information

19. 19 Course Gedanken Experiment l What the socially-savvy SIMS students said about bi-directional links l Basically, overall a negative thing. –link “spamming” »people who hate microsoft overburdening them with links »sexual harrassment –use for false endorsements –alliances, negotiation for cross-linking, a link market –inability to hide confidential information –advertisers would be affected –redirect unwanted links to another page

20. 20 What’s Going On? l Before going social, most hypertext was –within a single “document” or user group –incompatible with outside hypertext –seen as useful as a new way for reading acomplex documents l After going social, hypertext is –seen as useful for linking information in quite farflung places, assembled by people who don’t have know each other or have access to each other’s systems –social issues follow l Without appropriate safeguards, pages might also have to adopt “defensive linking”

21. 21 CS and the Social Sciences l A subset of CS has long engaged with social sciences and humanities l Artificial Intelligence (since early 60’s) »psychology (cognitive science) »linguistics »philosophy l More recently, HCI »human-computer interaction »psychology (cognitive science, human factors) »ethnography l But … sociology … NOT

22. 22 What is this leading to? l I might be suggesting the topic: how should CS research be changed? l Instead, I think these effects are interesting in their own right.

23. 23 Turning the Tables l The standard way to incorporate a field (like sociology) into a CS project would be for the purposes of building better systems. –recommend information better –filter information better l Instead, what if the goal is to build systems to better understand society?

24. 24 Talk Re-Cap l When information processing systems “go social” they are used in radical, often unexpected ways l Now that information processing systems have gone social, it is time to use them to help us better understand society l Let’s Turn the Tables –Create technology to aid study of society