1. Lost in Memories Interacting With Large Photo Collections on PDAs Susumu Harada, Mor Naaman*, Yee Jiun Song, QianYing Wang, Andreas Paepcke Digital Library Project Stanford University

2. JCDL 20042 Motivation Small devices ubiquitous Storage, bandwidth: cheap The new photo wallet: PDA/Cellphone Can have access to all my photos Co-present sharing

3. JCDL 20043 Challenges Acquisition rate Small screen Find photos now! In addition… !

4. JCDL 20044 Personal Photo Collections Searching/browsing very difficult Little discernible structure to photo collections Warning: repeat!

5. JCDL 20045 Content-based retrieval –Basic, primitive (far from semantic) Manual labeling –Improved, yet cumbersome Visual methods for fast scanning (Zoom) –Don’t scale well, utilize unavailable screen space Managing Personal Photos Warning: repeat!

6. JCDL 20046 Our Approach Absolutely no human effort required Utilize automatically-captured, easy-to- get metadata, like time [and location] Warning: repeat!

7. JCDL 2004 Oy Vey! Hmm… I can do this! [JCDL 2002] Big Mama

8. JCDL 20048 Outline Timeline Browsing –What are “Events” –Timeline user interface Time clustering (detecting events) Experiment

9. JCDL 20049 Existing PDA Photo Browsers Require manual organization Even then, is it good enough?

10. JCDL 200410 Instead Use natural notion of time and event: –Photos that were taken in the same occasion and context.

11. JCDL 200411 Events: Time-based Structure Birthday Europe July 4th Feb Jul Sep Shoot bursts 2003 …

12. JCDL 200412 Events Birthday Europe July 4th Feb Jul Sep 2pm 5pm 6pm Feb 12 Cake Gifts Sub events

13. 13 Timeline PDA Opening Game Automatically created “meaningful” events “Drill in”: Click on any box, or use timeline

14. Number of Photos Stack icon Time range Back button Scroll Negative space Go to Thumbnail view Dates

15. 15 Zoom in time

16. 16 Flipping Through a Pile in Place

17. 17 That photo of the DB group members from WWW2003…

18. 18 That photo of the DB group members from WWW2003…

19. JCDL 200419 Outline Timeline Browsing Time clustering (detecting events) Experiment

20. JCDL 200420 Event Detection in Personal Collections Graham et al (our project). Time as essence for photo browsing through personal digital libraries. JCDL 2002. U. Gargi. Time-based analysis and event clustering. HP Tech Report (2003). Platt et al. Phototoc: Automatic clustering for browsing personal photographs. MSR Tech Report (2003). Stent and Loui. Using event segmentation to improve indexing of consumer photographs. SIGIR 2001. Cooper et al. Automatically Organizing Digital Photographs Using Time and Content IEEE Image Processing (2004). More…

21. JCDL 200421 Events According to Graham (et al) Time 1 day, 11 hours –Detect threshold gaps (6-24 hours) –Within each segment Find outliers Split at outliers –Repeat recursively PDA: Merge at any level – limit to 10 events 10, 15, 13, 26, 400, 21, 55,…

22. JCDL 200422 Event Tree Days Months Years

23. JCDL 200423 Outline Timeline Browsing Time clustering (events) Experiment

24. JCDL 200424 Experiment: Basic vs. Timeline TL thumbTL time view Basic Timeline 15 subjects Average 1200 personal photos

25. JCDL 200425 Tasks Search: find randomly chosen photo in own collection as quickly as possible Browsing: build a collage of –Friends –Family –Trip –Special events “Make time span and set of occasions broad”

26. JCDL 200426 The Interface or the Organization? Controlled using an additional condition Use automatic organization (events) with basic interface

27. JCDL 200427 Even with basic interface, “automatic” matches “manual” After learning, Timeline improves search time by 29% over basic. Timeline success rate better Least backtracking in Timeline (after learning) Results

28. JCDL 200428 Time (sec) 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 Number of Subject-Trials 70 60 50 40 30 20 10 Year Month Week Day Thumbs Full View Dwelling Times

29. JCDL 200429 At least as good as manual organization Novel interface beneficial Learning effects considerable (even better) Conclusions

30. JCDL 200430 Geo-photos, of course Utilizing “off screen space” (classify, share) Data collection Current Work

31. JCDL 200431 Off Screen Space

32. JCDL 200432 Thank You! More details: Proceedings Google: Mor Naaman mor@cs.stanford.edu http://www-db.stanford.edu/~mor/

33. JCDL 200433 Additional Slides

34. JCDL 200434 Three Phases of Photo Search Opening game: –Large-granularity search strategies Roughly when? [Where was it? Who was in the picture?] Middle game: –Check search assumptions Is this really the right year? No, Bobby wasn’t born yet. End game: –Look at individual photos in detail

35. JCDL 200435 Experiment Setup Browser Interface Baseline Timeline Manual organization Automatic organization BM BA TL1 TL2