Usability Assessment, Round Two: Re-Examining the UW Libraries Information Gateway Jennifer Ward University of Washington

Presentation Overview Background and History Current environment Hardware, personnel Efforts to date Lessons learned and futures

Looking Back Move to Web interface for most services – early 1998 Rapid design, testing, and prototyping of website Only formal usability testing was by students as a class assignment

Current Environment Web Steering Committee (WSC) established August, 2000 Decentralized website management Approximately 170 staff in 40 units with write permissions No testing on current interface

Lab and Support Usability lab –Other labs on campus –Currently in a shared conference room; will have dedicated space soon –PC with Camtasia, video camera Half-time graduate student dedicated to usability efforts –Brought a fresh perspective to the process Work with WSC to determine priorities System-wide support for assessment

Summer, 2001 Online Survey July 23 – August 13, unique responses; almost 1/3 each faculty, students, staff Goal was to get a sense of what users thought of the Gateway Based on a previous survey Used WebQ survey software, linked from home page

Summer, 2001 Survey Results Asked a lot of open-ended questions about navigation and features Nearly 80% successfully found what they wanted –“Once one becomes familiar with the site, it’s easy to navigate.” Library Catalog most important resource –Most requested new features focused on catalog functionality

Points to Ponder Timing is everything! –Low response rate Data quality –Many open-ended questions went unanswered –Too focused on navigation and features; need to broaden it

Autumn, 2001 Online Survey September 21 – October 20, 2001 Same goal as before, but do a better job this time Plugged in online campus newsletter Incentive was provided, but not advertised 131 unique responses –10% faculty, 28% staff, 27% grad students, 27% undergrads, 5% alumni, and 2% non-affiliates

Autumn, 2001 Survey Structure Restructured summer survey so we’d get more answers Ran past a couple of local experts for feedback 15 questions, multiple choice or open- ended Users could remain anonymous –Didn’t get affiliation, which would have been nice for data analysis

Autumn, 2001 Survey Results Participants for follow up studies Part of site that drew the most comments was our catalog (!!) Overwhelming response – it’s not “broken” –Almost 80% very satisfied or satisfied –Some areas need work and we’re addressing them –“Micro” level changes

Autumn, 2001 Survey Results Asked what new feature users wanted: –Some things were already on our site; looked at making them more prominent –Database selection wizard/helper –Online tutorial for Gateway and catalog –Better integration with campus authentication –$100 put into their bank account

Autumn, 2001 Survey Results Data clumping Wording and Navigation Databases and Journals –Show vendors on database list –Need an “all-purpose” database instead of having so many –No clear winner on hierarchical vs alphabetical listing of databases and e- journals

Card Sorting November 13 – December 6, 2001 –Developed and started recruiting while we were waiting for online survey to conclude To determine the significance of features of the Gateway and how users would organize and name these features Standardized procedures and documentation –Consent forms, scripts, task lists Participants –one undergrad, one grad, one faculty, and two staff members (not what we wanted)

Card Sorting – Tasks Pilot Test Think aloud protocol, videotape, index cards Two sets of tasks with 46 cards Rank according to perceived importance –Students more flexible than faculty/staff Group according to their own navigational structure –Change wording if they wanted

Card Sorting – Results Statistical analysis (dendrogram) –More significant with more participants Items in upper right are more important Confusing wording (i.e., Cascade, Connecting, Starting Points) Didn’t understand News and Current Events Have not used data in redesign

“By Subject” Pages One of the “broken” items from the survey and WSC’s perspective Politically hot topic How to help beginning researcher get started, also try to address the layout Hierarchical vs alphabetical presentation Recruitment –Targeted , undergrads from library units

“By Subject” Pages – Tasks Think aloud protocol, captured screen activity with software Write answers to questions as they go along Mockups on a test server Undergrad tasks varied – did they use “Core Resources”? Graduate student and faculty tasks were targeted to their discipline

“By Subject” Pages – Results Undergraduates 1.Wanted listing by class name, searched Yahoo! for everything (more comfortable) –Wanted ranking system (4 stars) 2.Expert researcher who never left the test pages; completed tasks in 2-3 minutes –Understood and liked “Core Resources” 3.Stayed with test pages except when he felt lost, then returned to his favorite search tool – Google –Drawn to resources with the folder icon

“By Subject” Pages – Results Graduate Students 1.Went there once, wasn’t satisfied and never came back 2.Not inclined to leave the pages; weakest researcher Faculty 1.Resources were already bookmarked

“By Subject” – Other Efforts We’re working on adding a survey to the bottom of these pages Search box on each page Simplify the design by removing explanatory blurbs at top

Lessons Learned However long you think something will take, multiply the time by 3 and you might be close Standardize the process as much as possible; use templates and set guidelines Refine tests before the involving users – run pilot tests or talk with experts Timing, timing, timing Try not to put something online unless you know it works. If users don’t find what they’re looking for the first time, they don’t come back.

Questions? The website: Contact us: or