1. 1 Flexible Search and Navigation using Faceted Metadata Prof. Marti Hearst Dr. Rashmi Sinha, Ame Elliott, Jennifer English, Kirsten Swearingen, Ping Yee February, 2002 University of California, Berkeley http://bailando.sims.berkeley.edu/flamenco.html Research funded by NSF CAREER Grant, NSF9984741

2. 2 Outline 1.Motivation 2.Approach Integrate Search into Information Architecture via Faceted Metadata 3.Definitions: Information Architecture Faceted Metadata 4.Recipe Interface and Usability Study 5.Image Interfaces and Usability Studies 6.Conclusions

3. 3 Motivation and Background

4. 4 Claims Web Search is OK –Gets people to the right starting points Web SITE search is NOT ok The best way to improve site search is –NOT to make new fancy algorithms –Instead … improve the interface

5. 5 The Philosophy Information architecture should be designed to integrate search throughout Search results should reflect the information architecture. This supports an interplay between navigation and search This supports the most common human search strategies.

6. 6 An Important Search Strategy Do a simple, general search –Gets results in the generally correct area Look around in the local space of those results If that space looks wrong, start over –Akin to Shneiderman’s overview + details Our approach supports this strategy –Integrate navigation with search

7. 7 Following Hyperlinks Works great when it is clear where to go next Frustrating when the desired directions are undetectable or unavailable

8. 8 An Analogy text search hypertext

9. 9 Main Idea Use metadata to show where to go next –More flexible than canned hyperlinks –Less complex than full search –Help users see and return to what happened previously

10. Search Usability Design Goals 1.Strive for Consistency 2.Provide Shortcuts 3.Offer Informative Feedback 4.Design for Closure 5.Provide Simple Error Handling 6.Permit Easy Reversal of Actions 7.Support User Control 8.Reduce Short-term Memory Load From Shneiderman, Byrd, & Croft, Clarifying Search, DLIB Magazine, Jan 1997. www.dlib.org

11. 11 Information Architecture

12. 12 A Taxonomy of WebSites low high Complexity of Applications Complexity of Data From: The (Short) Araneus Guide to Website development, by Mecca, et al, Proceedings of WebDB’99, http://www-rocq.inria.fr/~cluet/WEBDB/procwebdb99.html Catalog Sites Web-based Information Systems Web- Presence Sites Service- Oriented Sites

13. 13 An Important IA Trend Generating web pages from databases Implications: –Web sites can adapt to user actions –Web sites can be instrumented

14. 14 Faceted Metadata

15. 15 Metadata: data about data Facets: orthogonal categories Time/DateTopicGeoRegion 

16. 16 Faceted Metadata: Biomedical MeSH (Medical Subject Headings) www.nlm.nih.org/mesh

17. 17 Mesh Facets (one level expanded)

18. 18 Questions we are trying to answer How many facets are allowable? Should facets be mixed and matched? How much is too much? Should hierarchies be progressively revealed, tabbed, some combination? How should free-text search be integrated?

19. 19 How NOT to do it Yahoo uses faceted metadata poorly in both their search results and in their top-level directory They combine region + other hierarchical facets in awkward ways

20. 20 Yahoo’s use of facets

21. 21 Yahoo’s use of facets

22. 22 Yahoo’s use of facets

23. 23 Yahoo’s use of facets Where is Berkeley? College and University > Colleges and Universities >United States > U > University of California > Campuses > Berkeley U.S. States > California > Cities >Berkeley > Education > College and University > Public > UC Berkeley

24. 24 Problem with Metadata Previews as Currently Used –Hand edited, predefined –Not tailored to task as it develops –Not personalized –Often not systematically integrated with search, or within the information architecture in general

25. 25 Recipe Collection Examples

26. 26 From soar.berkeley.edu (a poor example)

27. 27

28. 28 From www.epicurious.com (a good example)

29. 29

30. 30

31. 31

32. 32 Epicurious Metadata Usage Advantages –Creates combinations of metadata on the fly –Different metadata choices show the same information in different ways –Previews show how many recipes will result –Easy to back up –Supports several task types “Help me find a summer pasta,'' (ingredient type + event type), “How can I use an avocado in a salad?'' (ingredient type + dish type), “How can I bake sea-bass'' (preparation type + ingredient type)

33. 33 Metadata usage in Epicurious PrepareCuisineIngredientDish Recipe

34. 34 Metadata usage in Epicurious PrepareCuisineIngredientDish PrepareCuisineDish I Select

35. 35 Metadata usage in Epicurious PrepareCuisineIngredientDish I > Group by PrepareCuisineDish

36. 36 Metadata usage in Epicurious PrepareCuisineIngredientDish PrepareCuisineDish I > Group by

37. 37 Metadata usage in Epicurious PrepareCuisineIngredientDish PrepareCuisineDish I > Group by PrepareCuisine I Select

38. 38 Recipe Information Architecture Information design –Recipes have five types of metadata categories Cuisine, Preparation, Ingredients, Dish, Occasion Each category has one level of subcategories

39. 39 Recipe Information Architecture Navigation design –Home page: show top level of all categories –Other pages: A link on an attribute ANDS that attribute to the current query; results are shown according to a category that is not yet part of the query A change-view link does not change the query, but does change which category’s metadata organizes the results

40. 40 Epicurious Basic Search Lacks integration with metadata

41. 41

42. 42 Usability Study: epicurious

43. 43 Epicurious Usability Study 9 participants Three interfaces –Simple search form –Enhanced search form –Browse Two task types –known-item search –browsing for inspiration

44. 44 Epicurious Usability Study 9 participants –Independent Variables: 1) Epicurious Interface (Basic vs. Enhanced vs. Browse) 2) Task type (known-item search vs. browsing for inspiration) 3) Degree of constraint of query 4) Number of results required (1 vs. many) –Dependent Variables: 1) Time to find satisfactory recipe(s) 2) Navigation path (backtracking, starting over, revising queries) 3) Satisfaction with results of search 4) Satisfaction with individual system features (e.g. breadcrumbs, query previews, refine by hyperlinks) 5) Likelihood of using each interface in the future.

45. 45 Epicurious Usability Study Participants were asked to: –Do 3 pre-specified searches in advance –In the lab: Specify a cooking scenario of interest to them –Search for 3 recipes for this recipe –Search for each recipe using each of the interfaces Complete several structured tasks –Along the way, answer questions about Getting closer or farther away from goal Satisfaction with search results Satisfaction with the interface

46. 46 Epicurious Usability Study: Preference Data

47. 47 Epicurious Usability Study Interface Preference

48. 48 Epicurious Usability Study Feature Preference

49. 49 Epicurious Usability Study Constraint-based Preferences

50. 50 Usability Study Results: Summary People liked the browsing-style metadata- based search and found it helpful People sometimes preferred the metadata search when the task was more constrained –But zero results are frustrating –This can be alleviated with query previews People dis-prefer the standard simple search

51. 51 Missing From Epicurious How to scale? –Hierarchical facets –Larger collection How to integrate search? How to allow expansion in addition to refinement?

52. 52 Application to Image Search

53. 53 Current Approaches to Image Search Visual Content and Cues, e.g., QBIC (Flickner et al. ‘95) Blobworld (Carson et al. ‘99) Body Plans (Forsyth & Fleck ‘00) –Color, texture, shape –Move through a similarity space Keyword based –Piction (Srihari ’91) –WebSeek (Smith and Jain ’97) –Google image search

54. 54 A Commonality Among Current Content-based Approaches: Emphasis on similarity Little work on analyzing the search needs

55. 55 The Users Architects and City Planners

56. 56 The Collection ~40,000 images from the UCB architecture slide library The current database and interface is called SPIRO Very rich, faceted, hierarchical metadata

57. 57 Architects’ Image Use Common activitie: –Use images for inspiration Browsing during early stages of design –Collage making, sketching, pinning up on walls –This is different than illustrating powerpoint Maintain sketchbooks & shoeboxes of images –Young professionals have ~500, older ~5k No formal organization scheme –None of 10 architects interviewed about their image collections used indexes Do not like to use computers to find images

58. 58 Development Timeline Needs assessment. –Interviewed architects and conducted contextual inquiries. Lo-fi prototyping. –Showed paper prototype to 3 professional architects. Design / Study Round 1. –Simple interactive version. Users liked metadata idea. Design / Study Round 2: –Developed 4 different detailed versions; evaluated with 11 architects; results somewhat positive but many problems identified. Matrix emerged as a good idea. Metadata revision. –Compressed and simplified the metadata hierarchies Design / Study Round 3. –New version based on results of Round 2 –Highly positive user response

59. 59 The Interface Nine hierarchical facets –Matrix –SingleTree Chess metaphor –Opening –Middlegame –Endgame Tightly Integrated Search Expand as well as Refine Intermediate pages for large categories

60. 60

61. 61

62. 62

63. 63

64. 64

65. 65

66. 66

67. 67

68. 68 Usability Study on Round 3 19 participants –Architecture/City Planning background Two versions of the interface –Tree (one hierarchical facet at a time) –Matrix (multiple hierarchical facets) Several tasks Subjective responses –All highly positive –Very strong desire to use the interface in future –Will replace the current SPIRO interface

69. 69 Study Tasks 1.High Constraint Search: Find images with metadata assigned from 3 facets (e.g., exterior views of temples in Lebanon) 1.1) Start by using a Keyword Search 1.2) Start by Browsing (clicking a hyperlink) 1.3) Start by using method of choice 2.Low Constraint Search: Find a low-constraint set of images (metadata in one facet) 3.Specific Image Search: Given a photograph and no other info, find the same image in the collection 4.Browse for Images of Interest

70. 70 Interface Evaluation Users rated Matrix more highly for: –Usefulness for design work –Seeing relationships between images –Flexibility –Power On all except “find this image” task, users also rated the Matrix higher for: –Feeling “on track” during search –Feeling confident about having found all relevant images

71. 71 Overall Preferences: Matrix vs. Tree Simple search (e.g. images of deserts) Complex search (e.g. exteriors of temples in Lebanon) Find images like this one OVERALL PREFERENCE Matrix 131416 Tree 5433

72. 72 User Comments - Matrix “Easier to pursue other queries from each individual page” “Powerful at limiting and expanding result sets. Easy to shift between searches.” “Keep better track of where I am located as well as possible places to go from there.” “Left margin menu made it easy to view other possible search queries, helped in trouble-shooting research problems.” “Interface was friendlier, easier, more helpful.” “I understood the hierarchical relationships better.”

73. 73 User Comments – Tree Pro –“Simple” –“More typical of other search engines I’d use” –“Visually simpler and more intuitive…Matrix a bit overwhelming with choices.” Con –“I found SingleTree difficult to use when I had to refine my search on a search topic which I was not familiar with. I found myself guessing.” –“SingleTree required more thought to use and to find specific images.” –“I do not trust my typng and spelling skills. I like having categories.”

74. 74 Task Completion Times (Find Image is an artificial task: given a photo and no other info, find it in the collection.)

75. 75 When Given A Choice … For each interface, one task allowed the user to start with either a keyword search or the hyperlinks. 3 chose to search in both interfaces 11 chose to browse in both interfaces 4 chose to search in Matrix, browse in Tree 1 chose to browse in Matrix, search in Tree

76. 76 Precision and Recall Computed for tasks 1.1-1.3 Pooling used for determining relevant set Precision based on what was visible on screen

77. 77 Feature Usage Percentages (Dark bars show subtotals)

78. 78 Feature Usage (%) Types of Actions

79. 79 Feature Usage (%) Refining

80. 80 Feature Usage – Expanding / Starting Over

81. 81 Interface Evaluation Users rated Matrix more highly for: –Usefulness for design work –Seeing relationships between images –Flexibility –Power On all except “find this image” task, users also rated the Matrix higher for: –Feeling “on track” during search –Feeling confident about having found all relevant images

82. 82 Application to Medline

83. 83 Summary and Conclusions

84. 84 Summary A new approach to web site search –Use hierarchical faceted metadata dynamically, integrated with search Many difficult design decisions –Iterating and testing was key

85. 85 Summary Two Usability Studies Completed –Recipes: 13,000 items –Architecture Images: 40,000 items Conclusions: –Users like and are successful with the dynamic faceted hierarchical metadata, especially for browsing tasks –Very positive results, in contrast with studies on earlier iterations –Note: it seems you have to care about the contents of the collection to like the interface

86. 86 Summary We have addressed several interface problems: –How to seamlessly integrate metadata previews with search Show search results in metadata context “Disambiguate” search terms –How to show hierarchical metadata from several facets The “matrix” view Show one level of depth in the “matrix” view –How to handle large metadata categories Use intermediate pages –How to support expanding as well as refining Still working on it to some extent

87. 87 Advantages of the Approach Supports different search types –Highly constrained known-item searches –Open-ended, browsing tasks –Can easily switch from one mode to the other midstream –Can both expand and refine

88. 88 Advantages of the Approach Honors many of the most important usability design goals –User control –Provides context for results –Reduces short term memory load –Allows easy reversal of actions –Provides consistent view

89. 89 Advantages of the Approach Allows different people to add content without breaking things Can make use of standard technology

90. 90 Some Unanswered Questions How to integrate with relevance feedback (more like this)? –Would like to use blobworld-like features How to incorporate user preferences and past behavior? How to combine facets to reflect tasks?

91. 91 Thank you! bailando.sims.berkeley.edu/flamenco.html For more information: