1. Users and Metasearch Applications: New Challenges for Usability Assessment William E. Moen, Ph.D. Texas Center for Digital Knowledge University of North Texas Denton, Texas Presented at Access 2003: Extending Our Abilities October 3, 2003 Vancouver, British Columbia, Canada

2. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia2 Access personal journey 1999: The Z Texas Profile: Improving Z39.50 Interoperability with a Focus on User Requirements 2000: Z39.50 Update: Profiles and Semantic Interoperability 2001: Bath Profile Update: An International Z39.50 Specification for Library Applications and Resource Discovery 2001: Z-Interop: The Z39.50 Interoperability Testbed 2002: Z39.50 for Finding It All 2003: Users and Metasearch Applications

3. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia3 Standards in service of applications Preparing technical infrastructure for broadcast searching / metasearch Profiles specifying Z39.50 implementation Defining searches Solving interoperability problems

4. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia4 Where are we now? Some support among library technology vendors for Z39.50 profiles Many candidate search targets don’t support Z39.50 profiles or Z39.50 at all Commercial metasearch products and applications emerging in the past several years NISO Metasearch Initiative Assessing what we’ve got and how to improve

5. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia5 Metasearch Applications with: –Single search interface –Broadcast searches to multiple targets –Search targets may be Relatively homogeneous (e.g., library catalogs) Very diverse (e.g., format, topical coverage, etc,) Applications are faced with search targets –That are accessible via standard protocols (e.g., Z39.50) or proprietary APIs or screen-scraping –That support few or many common access points –That support various record formats for exchange

6. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia6 The challenge If our users make it to the library's web site at all, chances are they are confronted with library terminology they don't understand and a long list of databases they have to decipher and choose among. The result? Libraries are losing potential users. Librarians license valuable and costly full-text databases that we know contain the information researchers are seeking. But in a three-click world, each vendor's database remains a separate silo of information that our users don't find. Even if patrons are familiar with searching the OPAC, that won't help them retrieve articles. Library services that require training or require the user to come to the library undermine the advantages of licensing electronic content. Judy Luther -- 10/1/2003, Trumping Google? Metasearching's Promise, Library Journal

7. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia7 Metasearch issues User interface Collection selection (search targets) Reliable access and connection Search functionality Presenting results Navigating and subsequent searching Performance

8. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia8 Library of Texas A Virtual Library Reaching Across Boundaries to Extend Access to Information Resources Collection Boundaries Organizational Boundaries Format Boundaries

9. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia9 ZLOT project Z Implementation Component of the Library of Texas (ZLOT) –Develop standards-based approach for the virtual library Project Focus –Identify requirements for resource discovery service –Demonstrate feasibility and utility of resource discovery tools –Assist in implementation and testing Phase 1 – 2002: Design and specifications Phase 2 – 2003: Implementation

10. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia10 Service-centric virtual library Virtual Library Services User Group User Group User Group User Group Library Resources Library Technology

11. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia11 Virtual library services Standard Information Interchange Among Services FinancialAuthorization Resource Sharing Patron Account Library Information Education Resource Discovery ReferencePersonalization

12. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia12 Resource discovery: Supporting user tasks Browse Select Collection Evaluate Search Common User Interface Personalize Save Acquire

13. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia13 Resource discovery: Operational view Content OAI Repository Standards-based Search & Retrieval Protocols Proprietary or Non-standard Search & Retrieval Protocols Content Common Search Interface User Interface Texas Library Directory Database Relational Database To Power the RDS Application

14. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia14 LOT RDS: Summary of key requirements Customizable interface –For local library presence –For personalization Common interface for search and retrieval –Simple and advanced searching –Single database searching –Broadcast searching across similar/dissimilar databases Tools to help users select search targets Common display for results Indications of availability and “how to get”

15. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia15 Database-driven customization Texas Library Directory Database (TLDD) –Multi-use relational database RDS functionality driven by TLDD –Association of user with home library –Access/authentication mechanisms –Preliminary set of search targets Geographic proximity Strength of collection –Interlibrary loan request initiation

16. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia16 TLDD overview General Organization Details Person Details Address Details Collection Details –Collection – describes Collections of resource items or resource item descriptions –Collection Strength – describes the strength of a Collection Service Details –Service – contains information about the Services delivered by an Organization –Access Policy – describes Access Policies for a given Service –Z Protocol Information - records information about the level of support of a given Z39.50 Service delivery process Common Use –Alias – Supports the recording of names, abbreviations, acronyms, standard numbers, and symbols used to identify an entity

17. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia17 Optimizing the RDS Assumptions –Multiple user groups –Diversity of information needs, behaviors, and expectations –Users may not understand distributed search applications –User interface presents complex functionality Strategy to optimize –Usability assessment –Helping users develop new mental model

18. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia18 Informing the RDS product Heuristic evaluation by usability expert Acceptance testing by ZLOT staff –Use of original technical specifications –Identified usability issues User testing Ongoing testing by developers, ZLOT and TSLAC staff

19. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia19 Usability considerations Users –Categorization of users based on demographics, skills, needs, behaviors –Current library users vs. new users Interface is really about functionality –Presents application and its components –Components support various user tasks Usability criteria, measures, and procedures

20. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia20 Usability criteria Jakob Nielsen –Learnability –Efficiency –Memorability –Errors –Satisfaction –Other quality attributes (e.g., utility) Whitney Quesenbery –Effective –Efficient –Engaging –Error tolerant –Easy to learn Nielson, J. (2003). Usability 101. http://www.useit.com/alertbox/20030825.html Quesenbery, W. (2002). Getting Started: Using the 5Es to understand users. http://www.wqusability.com/articles/getting-started.html

21. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia21 Application & components Technical components Interoperability components User interface components Task components –Collection selection components –Searching components –Retrieval components –Personalization components

22. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia22 Usability framework

23. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia23 Priority usability issues Do user understand the service? Are user tasks supported? –Tasks represented by application functionality –Selection of search targets –Searching Simple Advanced Refining searches –Reviewing results –Navigating –Accessing/acquiring Is system performance adequate?

24. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia24 5 Es of usability revisited Resource Discovery Service Effective Efficient Easy to Learn Engaging Error Tolerant Adapted from Quesenbery, W. (2002). Getting Started: Using the 5Es to understand users.

25. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia25 Next steps Prepare a formal usability assessment Develop measures based on 5 Es to assess RDS components Develop scenarios for users to exercise all functionality Conduct user testing Inform enhancements to the product Generalize usability assessment methodology other for metasearch applications, resource discovery services, and virtual libraries

26. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia26 RDS example

27. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia27 RDS example

28. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia28 RDS example

29. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia29 RDS example

30. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia30 RDS example

31. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia31 RDS example

32. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia32 RDS example

33. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia33 RDS example

34. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia34 RDS example

35. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia35 RDS example

36. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia36 DC proceedings/Seamark

37. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia37 DC proceedings/Seamark

38. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia38 Metasearch usability Selecting appropriate search targets –Default set of targets –User selection from target list –Getting contextual info from users for search –Intermediate search against representations of targets Making sense of search results –Linear lists of results –Using metadata to categorize results and show relationships –Hierarchical faceted metadata

39. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia39 References NISO Metasearch Initiative Library of Texas Initiative Hierarchical Faceted Metadata and Search –The Flamenco Search Interface Project –Faceted Metadata Search and Browse

40. MoenAccess 2003 -- October 3, 2003 -- Vancouver, British Columbia40 William E. Moen wemoen@unt.edu Library of Texas ZLOT Project Texas Center for Digital Knowledge