YOU ONLY THINK YOU’RE LIKE GOOGLE : COMPARATIVE USER EXPERIENCE OF DISCOVERY PLATFORMS Rice Majors Faculty Director of Libraries Information Technology University of Colorado, Boulder AMICAL Conference – April 2012 – American University of Sharjah

Purpose(s) of study  Create a “common good” set of data  Vendors lack access/resources for consistent & comparative studies  Many libraries lack access/resources to carry out studies (especially comparative)  Working on a campus, I have relatively easy access to study participants  Pursuing this research justified acquiring usability resources we can use for other purposes

Study design & methodology  Task based testing with undergraduate students  Usability software (Morae)  Video capture of actions taken  Video/audio capture of “thinking aloud”  Survey instrument  Tested against five discovery tools / next-gens

Partner libraries  James Madison University (EBSCO Discovery Service)  University of Colorado Boulder (Encore Synergy)  Vanderbilt University (Primo Central)  Arizona State University (Summon)  Auraria Library, Denver (WorldCat Local)

Participants  Undergraduate students only  Library student employees were excluded  28 participants in total  January 26-March 17, 2011  Each participant tested one interface only (to avoid “learning”)

Participant tasks 1. Find three books on a particular topic toward writing a paper. 2. Find three articles on a particular topic toward writing a paper – two articles must be peer- reviewed. 3. Find recordings by a particular artist. Remind yourself to look at these again later. 4. The library doesn’t own the book “[title].” Have the library get this book for you.

Survey Instrument  7 Likert items assessing perceptions of ease of use:  I was able to find what I need for these tasks using this discovery platform  If I were doing my own research, I would be able to find what I needed using this discovery platform [etc]  Short answer questions:  What is easy to use about this platform?  What is hard to use about this platform?  What one change would make the biggest improvement to this discovery platform?

Perceptions of ease of use

Features appreciated – all  Finding different types of resources & narrowing search (12)  Ability to books & articles (7)  Found relevant articles/results (2)  Ability to save records for later  Smooth interface to library website  Simple interface  Integration with consortial borrowing  Clearly labeling peer- reviewed articles

Comments specific to each product  Summary recommendations for each product are based on consensus of study participants only  Where there are fewer recommendations, there was simply less consensus on what issues to address  Note that some vendors have meanwhile addressed some of the issues documented here  Testing was done January 26-March 17, 2011  Plausible that vendors heard similar feedback from existing and potential customers

Product-specific feedback  EBSCO Discovery  Call it something else  Connect users to ILL / resource-sharing more easily  Make it clear[er] how to use the structured search boxes  Address the need to login just to perform searches  Encore Synergy  Simplify / streamline the interface  Add an advanced search option  Primo Central  Make it easier to find options in “additional services”  Connect users to ILL / resource-sharing more easily

Product-specific feedback  Summon  Make it easier to find & use option  Connect users to ILL / resource-sharing more easily  Address misconception of “add results beyond your library collection”  Rely less on link resolver technology  Make it clear[er] when/why options reset  WorldCat Local  Make it easier (in several ways) to use option  Connect users to ILL / resource-sharing more easily  Make it clear[er] when/why options reset  Improve layout (e.g. so it’s clear[er] when an abstract isn’t available)

Feedback for all products 1. Introduce further enrichment options to allow patrons to evaluate resources (c.f. Amazon) 2. Make it possible/intuitive to add all results to a list & export all results 3. Provide context so that it’s clear what has been searched (& what is not included) 4. Partner (better) with libraries on who does what 5. Make it easy to get help (“ask a librarian” etc.)

Inadvertent findings  Jargon & library practices  Do know what “interlibrary loan” is  Do not know what “government publications” or “electronic resources” are, nor the catalog nickname  Musings about “book” qua “book”  Want FRBR solutions (maybe)  Participant behavior  Will type anything into the search box  Will look for expected features

Issues with study design  User defined success  Inescapable testing of information literacy  Students already have multiple frameworks for using library catalog software  Library implementation choices  Library website design choices

More information & intentions  “Highlights” video footage to demonstrate findings (available on request)  Article forthcoming in Library Trends  Follow up studies  Involve students from several campuses, with different article database providers and different discovery tools  Use multiple implementations of each interface  More tasks (with randomization of order)

THANK YOU! Rice Majors Faculty Director of Libraries Information Technology University of Colorado, Boulder AMICAL Conference – April 2012 – American University of Sharjah