MyLifeBits Jim Gemmell & Gordon Bell SDForum Distinguished Speaker Series February 19, 2004

Outline What is MyLifeBits What is MyLifeBits History/Motivation History/Motivation MyLifeBits system outline MyLifeBits system outline Demo Demo Future work Future work

MyLifeBits is: An experiment in lifetime storage An experiment in lifetime storage Digitizing Gordon Bell’s past Digitizing Gordon Bell’s past Capturing more of his future Capturing more of his future A software system A software system Capture Capture Storage & retrieval Storage & retrieval Organization & annotation Organization & annotation Minimum requirement: fulfill Vannevar Bush’s 1945 “Memex” vision Minimum requirement: fulfill Vannevar Bush’s 1945 “Memex” vision

Memex As We May Think, Vannevar Bush, 1945 “A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility” Full-text search, text & audio annotations, and hyperlinks Full-text search, text & audio annotations, and hyperlinks

I am data

History Started in telepresence & telework Started in telepresence & telework Remote worker needs everything in notebook Remote worker needs everything in notebook Rediscovery of the PC Rediscovery of the PC Started scanning… Started scanning…

Our guinea pig,Gordon Bell, has digitized a lifetime’s worth of: Books written (and read when possible) Books written (and read when possible) Personal documents (letters, bills, legal, papers written, memos, …) Personal documents (letters, bills, legal, papers written, memos, …) Photos Photos CD collection CD collection Posters, paintings Posters, paintings Home movies Home movies And, of course, all files on his PC And, of course, all files on his PC Now recording: telephone calls, more video, all web pages visited, usage logging, radio, TV… Now recording: telephone calls, more video, all web pages visited, usage logging, radio, TV…

Lifetime storage Suppose you began keeping: Suppose you began keeping: 100 messages a day (5KB each) 100 messages a day (5KB each) 100 web pages day (50KB each) 100 web pages day (50KB each) 5 scanned pages a day (100KB each) 5 scanned pages a day (100KB each) 1 book every 10 days (1 MB each) 1 book every 10 days (1 MB each) 10 photos per day (400 KB JPEG each) 10 photos per day (400 KB JPEG each) 8 hours per day of sound - e.g. telephone, voice annotations, and meeting recordings (8 Kb/s) 8 hours per day of sound - e.g. telephone, voice annotations, and meeting recordings (8 Kb/s) 1 new music CD every 10 days (45 min each at 128 Kb/s) 1 new music CD every 10 days (45 min each at 128 Kb/s) It will take you 5 years to fill up your 80 GB drive It will take you 5 years to fill up your 80 GB drive By that time, you can buy a terabyte hard drive – which will take more than 60 years to fill By that time, you can buy a terabyte hard drive – which will take more than 60 years to fill

Trying to fill a terabyte in a year Gordon’s lifetime collection < 30 GB (12 GB is music CDs) Gordon’s lifetime collection < 30 GB (12 GB is music CDs) Item Per TB Per day Photo (400 KB JPEG) 2.7M photos 7254 photos 1 MB document 1.0M docs 2872 docs 128 kb/s audio 18.6K hours 51 hours 256 kb/s video 9.3K hours 26 hours 1.5 Mb/s video 290 hours 4 hours

“yet if the user inserted 5000 pages of material a day it would take him hundreds of years to fill the repository, so that he can be profligate and enter material freely” -Vannevar Bush, 1945

Why bother? An existence proof. The following exist in abundance: Shoeboxes full of photos Shoeboxes full of photos Photo albums & framed photos Photo albums & framed photos Creative Memories is a thriving business selling resources for created high-end photo albums that are well laid out and highly annotated, using long-lasting materials. Creative Memories is a thriving business selling resources for created high-end photo albums that are well laid out and highly annotated, using long-lasting materials. Home videos Home videos Bookshelves and filing cabinets Bookshelves and filing cabinets Old bundles of letters Old bundles of letters Professional video/photo companies do capture at kids’ sports events and sell content like hotcakes Professional video/photo companies do capture at kids’ sports events and sell content like hotcakes Probably not accessed very often but TREASURED (what’s the one thing you would save in a fire?) Probably not accessed very often but TREASURED (what’s the one thing you would save in a fire?)

Why bother?..more reasons To eliminate physical storage (paper, CDs…) To eliminate physical storage (paper, CDs…) It costs more (in time) to delete than the cost the storage It costs more (in time) to delete than the cost the storage You may only want to retrieve one of many items in the future, but cannot predict which one (which is why you file many things now) You may only want to retrieve one of many items in the future, but cannot predict which one (which is why you file many things now) For posterity and nostalgia For posterity and nostalgia For memory enhancement & faster search (search your LifeBits rather than the web … a single source to look for anything you have ever seen) For memory enhancement & faster search (search your LifeBits rather than the web … a single source to look for anything you have ever seen) Let content analysis and data mining discover trends and correlations in your life Let content analysis and data mining discover trends and correlations in your life

So you’ve got it – now what do you do with it? Can you find anything? Can you find anything? Can you organize that many objects? Can you organize that many objects? Once you find it will you know what it is? Once you find it will you know what it is? Once you’ve found it once, could you find it again? Once you’ve found it once, could you find it again?

“A record if it is to be useful … must be continuously extended, it must be stored, and above all it must be consulted” “The difficulty seems to be, not so much that we publish unduly … but rather that publication has been extended far beyond our present ability to make real use of the record” - Vannevar Bush

MyLifeBits Software MyLifeBits store database Voice annotation tool Text annotation tool Telephone capture tool TV capture tool TV EPG download tool Radio capture tool Radio EPG tool PocketPC transfer tool PocketRadio player Import files MyLifeBits Shell files Legacy applications Browser tool Internet IM capture MAPI interface Legacy client

Making the store useful Encourage annotations Encourage annotations Alternatives to hierarchy for organization Alternatives to hierarchy for organization Exploiting database features Exploiting database features

Everything goes in a database You need all the features of a database (Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, replication) You need all the features of a database (Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, replication) If you don’t use one, you will find yourself creating one! If you don’t use one, you will find yourself creating one! Files as blobs, also sync with file system for legacy apps Files as blobs, also sync with file system for legacy apps SQL

Entities & Links Annotates Caller in Phone Call Photo of Event Transcludes

MyLifeBits Schema (simplified) Images Music Phone calls Resources Relation- ships Relation- ship types Entity types Resource entities Event types Event log Events Tasks People Notes Messages Saved searches

DEMO

Summary We have entered an era of virtually unlimited storage, enabling the lifetime store (“Memex”) We have entered an era of virtually unlimited storage, enabling the lifetime store (“Memex”) To make the store useful we need annotation, typed links, and database features To make the store useful we need annotation, typed links, and database features

Future work: Content analysis & Data Mining “Creative thought and essentially repetitive thought are very different things. For the latter there are, and may be, powerful mechanical aids” – Vannevar Bush Is MyLifeBits just enough rope to hang yourself with? Is MyLifeBits just enough rope to hang yourself with? MyLifeBits must become MyPersonalAssistant MyLifeBits must become MyPersonalAssistant Content analysis and data mining Content analysis and data mining Doc similarity & “clean living” Doc similarity & “clean living” Document meta-data extraction Document meta-data extraction

Future work: UI Focus to date on functionality – now we need to make it friendlier Focus to date on functionality – now we need to make it friendlier User studies with SFSU User studies with SFSU

Future work: Visualizations Don't give me a little card image and say, "That's all you've got, because that's what I thought you should want for your virtual shoebox." There have got to be multiple modalities and the designers have to be able to deal with that. … don't metaphor me in, don't give me only one way of looking at things. Don't give me a little card image and say, "That's all you've got, because that's what I thought you should want for your virtual shoebox." There have got to be multiple modalities and the designers have to be able to deal with that. … don't metaphor me in, don't give me only one way of looking at things. -Andy van Dam, Hypertext '87 Keynote Address Next Media Web Scout U. Maryland IN-SPIRE

Future work: new capture modes/devices

BONUS SLIDES

Dear Appy, How committed are you? Signed, Lost and Forgotten Data Dear Appy, I'm having trouble with long-term commitment -- not on my end, heaven knows, but from the apps that created me and with whom I like to associate. Over time, these pesky apps evolve and they simply don't recognize the data that they once helped create! But, we data progeny -- and there are lots of us -- feel that as our creators, these apps should be responsible for eternal support. But the little problem with recognition isn't the worst of it – sometimes the apps even disappear altogether. I ask you, is it expecting too much for 20-something year old data like me to be interpretable by my app (e.g. Acrobat, DB2, Draw, Eudora, Office, Quicken, or RealNetworks), or am I just associating with irresponsible apps? If things continue on their current path, it seems I will be completely un-interpretable within 20 to 50 years! My apps will move to other platforms, or evolve to be more Internet- or Next-Big-Thing-centric... By Gordon Bell