1. Individualized Knowledge Access David Karger Lynn Andrea Stein Mark Ackerman Ralph Swick

2. Information Access A key task in Oxygen: help people manage and retrieve information Three overlapping projects: l Haystack:  information storage and retrieval  application clients l Semantic Web: next-generation metadata l Volt: collaborative access

3. Presentation Overview Motivation l Information access behavior and goals System Design & Architecture l Data Model l Interacting data and UI components Working applications l Base haystack l Frontpage l Volt

4. Motivation

5. Problem Scenario I try solving problems using my data: l Information gathered personally l High quality, easy for me to understand l Not limited to publicly available content My organization: l Personal annotations and meta-data l Choose own subject arrangement l Optimize for my kind of searching Adapts to my needs

6. Then Turn to a Friend Leverage l They organize information for their own use l Let them find things for me too Shared vocabulary l They know me and what I want Personal expertise l They know things not in any library Trust l Their recommendations are good

7. Last to Library/web Answer usually there l But hard to find l Wish: rearrange to suit my needs l Wish: help from my friends in looking

8. Lessons Individualized access l Best tools adapt to individual ways of organizing and seeking data Individualized knowledge l People know more than they publish l That knowledge is useful to them and others Collaborative use l Right incentives lead to sharing and joint use

9. Haystack Individualized access l My data collection, organization l Search tools tuned for me Collaborate to leverage individual knowledge l Access unpublished information in others’ haystacks l Self interest public benefit Lens to personalize access to the world library l Rearrange presentation to suit my personal needs

10. Example Info on probabilistic models in data mining l My haystack doesn’t know, but “probability” is in lots of email I got from Tommi Jaakola l Tommi told his haystack that “Bayesian” refers to “probability models” l Tommi has read several papers on Bayesian methods in data mining l Some are by Daphne Koller l I read/liked other work by Koller l My Haystack queries “Daphne Koller Bayes” on Yahoo l Tommi’s haystack can rank the results for me…

11. System Design

12. Gathering Data Haystack archives anything l Web pages browsed, email sent and received, address book, documents written And any properties, relationships l Text of object (for text search) l Author, title, color, citations, quotations, annotations, quality, last usage Users freely add types, relationships

13. Semantic Web Arbitrary objects, connected by named links No fixed schema l User extensible Sharable by any application l A new “file system”? DocD. KargerHaystack title author Outstanding quality says HTML type

14. Gathering Data Active user input l Interfaces let user add data, note relationships Mining data from prior data l Plug-in services opportunistically extract data Passive observation of user l Plug-ins to other interfaces record user actions Other Users

15. Data Extraction Services Web Observer Proxy Triple Store Mail Observer Proxy Machine Learning Services Web Viewer Volt Viewer/ Editor Spider

16. Sample Applications

17. Because everything uses the Semantic Web constructions, a variety of application clients can share information l Web Browser---data viewer l FrontPage---personalized information filter l Volt---collaboration tool

18. Haystack via Web Web server interface Basic operations: l Insert objects l View objects l Queries

19. Haystack via Web

20. Viewer shows one node and associated arrows Service notices we’ve archived a directory; so archives the objects it contains (and so on…)

21. Haystack via Web Services detect document type, extract relevant metadata Output can specialize by type of object

22. Mediation Haystack can be a lens for viewing data from the rest of the world l Stored content shows what user knows/likes l Selectively spider “good” sites l Filter results coming back  Compare to objects user has liked in the past l Can learn over time Example - personalized news service

23. News Service

24. Scavenges articles from your favorite news sources l Html parsing/extracting services Over time, learns types of articles that interest you l Prioritizes those for display Content provider no longer controls viewing experience l No more ads

25. Personalized News Service

26. Collaborative Access Want to leverage others’ work in organizing information l No need to “publish” expertise l Exposed automatically---without effort l Self interest helps others

27. Volt Volt is about collaboration between people l The Haystack architecture allows easy collaboration among individuals  semantic web references to Haystack objects l Individuals share parts of their Haystack l Group spaces and shared notebooks

28. Volt

29. Collaborators Those I interact with l Frequent mail contact l Frequent visits to their home page Those with shared content l And who have same opinions about content l Collaborative filtering techniques Referrals Expertise search engine

30. Expertise Beacon

31. Volt Expertise Beacons Group spaces and shared notebooks l Create individual and group profiles Profiles can be used to find other people l Allows targeted search l “Who else is working on this project?” User controls visibility/privacy

32. Summary Next generation information access Semantic Web l provides a language and capabilities for meta-data Haystack l teases out individual knowledge, l stores it in a coherent fashion, and l allows a variety of application clients to leverage individual meta-data Volt l turns individual knowledge into a community resource

33. More Info http://haystack.lcs.mit.edu/ http://www.w3c.org/2001/sw karger@mit.edu las@ai.mit.edu ackerman@lcs.mit.edu swick@w3.org