Presentation on theme: "By Jason Russell Phone: (907)590-1984 MLIS Graduate: U.S.M. Hattiesburg Campus LinkedIn Profile:"— Presentation transcript:
By Jason Russell Phone: (907) MLIS Graduate: U.S.M. Hattiesburg Campus LinkedIn Profile:
What is Website Usability? Web site usability is the practice of ensuring that a Web site is: Findable Intuitive Up to date
What we will cover This presentation will consist of 4 sections: Section 1: Project Management Section 2: Develop Your Project Plan Section 3: Performing your usability study Section 4: Reviewing your data and presenting results
Section1: Project Management: Can I afford to utilize outside consultants? Utilizing the time of usability experts unfortunately does not come cheap. Various prices of top companies are listed below: Nielsen Norman Group, evaluations/, anywhere between $10,000 to $40,000 or higher evaluations/ Vaught Usability anywhere between $2500 to $4,000 for expert reviewshttp://www.vaughtusability.com/ Due to the varied nature of usability projects, project quotes range considerably.
Section 1: Project Management: Building your usability team. If your institution lacks sufficient funds to pay someone else to do the work, then the project will need to be accomplished internally. Remember, time is money. If 2 employees at $22 an hour, and one volunteer, each devote 50 hours to this project, it equates to around $2,200 in total staff wages. Utilize interns when available
Section 1: Project Management: Searching for low cost help by going digital. What if the library is in a small or rural community, and volunteers are hard to come by? Do not be afraid to go digital with your search. Library and Informational Sciences graduate students, at colleges that offer an MLIS program, are a great resource to turn to: ALA has a comprehensive database, listing all institutions that offer such programs at
Section 2: Develop your Project Plan This Section will cover: Figuring out what to test Choosing your test participants And the most cost effective testing methods to utilize
Section 2: Develop your Project Plan: Figuring out what to test Define the parameters of your project Are you creating a new website, updating an entire existing website, or updating a portion of a website? What are your libraries mission and vision statements, if any, for your website, as well as it’s major short, mid, and long term goals?
Section 2: Develop your Project Plan Contd. Figuring out what to test It is a good idea to include your home page with any testing. Imagine your homepage as an airport, and the destinations offered are the different parts of your site that you want users to go to. The airplane being the path that the user has to take to get there.
Section 2: Develop your Project Plan Contd. Figuring out what to test Utilize preliminary surveys, comments from staff, and feedback from users to help limit your study to a manageable size. It is far better to perform a limited study of a portion of your site well, then a broad study of the entire site poorly, or worse, have to stop mid way through due to lack of resources. Over 90% of assessed studies stated as a basis for their methodology the work done by Jakob Nielsen. Specifically, the concept of diminishing returns on data, and increasing costs and time spent, after utilizing five users.
Section 2: Develop your Project Plan Contd. Choosing test participants Know your users Once you have a good idea what needs to be tested the next important step is to gather test participants. Utilizing prior internal reports such as community analysis documents or user growth projections save time and resources. These sources of information all serve to help you choose the correct participants that are representative of your current and potential user base.
Section 2: Develop your Project Plan Contd. Chosen Test Participant Demographics Testing sessions that used card sort methodology used 10 participants most frequently, and the user group 'Faculty/Staff' most often. Testing sessions that used task analysis methodology used between one and seven users most often, with two users being tested the most. The user group 'Undergraduate' was tested most frequently.
Section 2: Develop your Project Plan Contd. What the data says
Section 2: Develop your Project Plan Contd. Chosen Test Participant Demographics For the testing type Interview, the number of participants that were utilized per testing round ranged between two and 48 test participants The user groups Faculty/Staff and Undergraduate were utilized most often, being utilized four times each. For the testing type Survey, the number of participants that were utilized ranged the most per testing round, between five and 1,235 individuals.
Section 2: Develop your Project Plan Contd. Manner of Recruitment Target user groups in the surrounding community for selection as volunteers. Attracting those users to be test participants can be a challenge. The methods used should be considered carefully: Should I offer cash incentives vs. simple refreshments? Should I have door prizes? To what length do I put up notices around my institution? To what degree is staff willing to help market this study to patrons? Are my user groups in a geographically constrained location, as in the case of students and teachers living in housing?
Section 2: Develop your Project Plan Contd. Manner of Recruitment The most utilized methods for recruitment were , gift certificates, and flyers, being utilized in 20, 19, and 16 studies respectively. Many studies included a monetary reward, or gift certificate to the local book shop or university store. The most frequent amount being $10.00.
Section 2: Develop your Project Plan Contd. Manner of Recruitment Other methods utilized include: Phone calls, blog posts, SMS messaging, or simply setting up a display table at a known locale where a targeted user group is likely to be. Using a combination of passive and energetic methods will increase the likelihood that adequate test participants volunteer in sufficient numbers.
Section 2: Develop your Project Plan Contd. Choosing test participants Choosing the optimum number of test participants. Once you have the correct ratio of users and user groups it is now necessary to figure out how many users to test. This is dependent upon several factors: Time frame of testing available (the more the users to be tested the longer the testing will take). Number of dry runs (rounds of testing, and different types of testing to be performed.
Section 2: Develop your Project Plan Contd. Choosing your testing methods There are a number of testing methods that are available for use. The methods that we will go over are: Eye Tracking studies Remote Testing Task Analysis studies Card Sorts
Section 2: Develop your Project Plan Contd. Choosing your testing methods Eye Tracking Technology: Eye tracking technology is utilized to help determine where users look on Web pages when they are performing information queries. This technology is normally utilized in conjunction with task analysis studies. This method has a potentially high initial technology cost, ranging widely depending upon the company used. Eye Tech DS offers systems ranging from 7,500-10,000. Freeware products such as Labs Media use click tracking software to attain a similar result. It can be found at
Section 2: Develop your Project Plan Contd. Choosing your testing methods Remote Testing In many cases, you may want to perform testing remotely, especially if your user base comes from widely spread geographic locations. This method does come with some potential problems Skill level with technology of test participant. If the test is proctored distantly then extra time should be allowed to ensure the tester can use the technology. Not being present, the test proctor can miss subtle body cues that happen outside of view. Software can be expensive to purchase.
Section 2: Develop your Project Plan Contd. Choosing your testing methods Task Analysis studies This method of testing assesses how users perform search queries. Tasks are designed generally to assess major services/sections offered on the site. Test length should be limited to 60 minutes to avoid user test- fatigue. Number of tasks vary depending upon complexity of task, and perceived user browsing skills. (Our study participants completed around 8 tasks in one hour.)
Section 2: Develop your Project Plan Contd. Choosing your testing methods Card Sorts Card Sorts help to determine the best informational layout of your site. Card Sorts can be performed on line or in person. Usabilitest offers a free 3-day subscription if you are able to schedule all testing in that time period. Usabilitest
Section 2: Develop your Project Plan Contd. Choosing your testing methods Each of the above mentioned methods do not all have to be completed to ensure testing success. At the very minimum, performing 5 sessions of card sort testing and task analysis studies with an optional retest of between 3-5 users after redesign is an ideal plan. Jakob Nielsen (the acclaimed Usability Guru) reports that testing 5 users will reveal about 85% of the problems within the tested section of the Web site. (2013)85%
Section 2: Develop your Project Plan Contd. Choosing your testing methods The most common methods used were: Web based task analysis studies, open card sorts, focus group interviews, and Web based surveys. Approximately 63% of the time Web based task analysis studies were utilized. Approximately 10% of the time open card sorts were utilized. Approximately 8% of the time focus groups were utilized. Approximately 17% of the time Web based surveys were utilized.
Section 3: Performing your usability study This section will cover: choosing a proper testing location, how to ensure adequate collection of data, and how to limit testing bias.
Section 3: Performing your usability study contd. Choosing a Proper Test Location Choosing a proper location helps to ensure testing success. The location should generally have the following characteristics: Quiet, Clean, Available for the entire testing phase, If the testing is performed off site ensure the above qualities that relate to the tester are present. If the testing is performed at a tester’s work station ensure that his or her manager is aware of when the testing is occurring, and notifies staff so that distractions are limited.
Section 3: Performing your usability study contd. Ensuring adequate collection of data Protect your data! Whatever recording software you use to capture data you should be prepared for the worst Always record utilizing two different methods. Further, all notes, digital or otherwise, and recordings should be copied and saved to a separate computer or external HDD.
Section 3: Performing your usability study contd. Ensuring adequate collection of data With whatever recording technology you use ensure that it functions and is reliable well in advance of your testing. Ensure that when you are testing you limit introductions to 5 minutes or less for a 60 minute test.
Section 3: Performing your usability study Limiting Testing Bias Testing bias can quite simply ruin data, and invalidate the entire project. Limiting testing bias in a critical item to consider. Fortunately, limiting this bias can be fairly simple, simply by: Standardizing everything in the test that can be standardized Letting the data dictate generation of results, not personal opinion.
Section 3: Performing your usability study Limiting Testing Bias For each test performed, a pre-session demographic questionnaire and a post-session qualitative questionnaire, or some combination of both, were generally utilized. Utilization of both the pre and post questionnaires during a single test happened the most frequently at 37% of the time.
Section 4: Reviewing data and presenting results This section will cover: how to compile your raw data into a form that your supervisors can utilize easily, and how to craft a report that summarizes that data effectively.
Section 4: Reviewing data and presenting results contd. Making sense of your raw data Microsoft Excel can be utilized to help organize and represent your data, with statistical functionality should it become necessary. Within Microsoft is a function called PivotTable that allows the data to be organized and queried in multiple different ways. PivotTables is especially useful in attaining statistical data of usability results so long as test population size, is in the upper range of between individual tests per study, and 39 individual tests when performing eye tracking. (Nielsen, 2012) offers a well rounded guide to PivotTable creation and utilization.
Section 4: Reviewing data and presenting results contd. Crafting a professional document Crafting a document that not only presents the collected data and generated results in a logical manner, but also illustrates how the study was designed and performed, can be done so in several ways: utilizing internal reporting requirements already in existence, utilizing layout guidelines detailed at authoritative sites such as or utilizing layout guidelines for academically published papers.
Conclusion There are numerous methods for conducting usability studies. This presentation touched only on how to perform a basic low cost usability study. Keeping costs down can include: Hiring free help, Focusing effort on targeting specific user groups when attaining test participants, Utilizing free trial software for testing and recording, Utilizing program such as Excel to organize and assess the data, Et cetera
Works Cited Nielsen, J. (2000). Why You Only Need to Test with 5 Users. In AlertBox. Retrieved January 15, 2015, from the Nielsen Norman Group Web site: test-with-5-users/ test-with-5-users/ Nielsen, J. (2005). Time Budgets for Usability Sessions. In AlertBox. Retrieved January 15, 2015, from the Nielsen Norman Group Web site: usability-sessions/ usability-sessions/ Nielsen, J. (2012). How Many Test Users in a Usability Study? In AlertBox. Retrieved February 10, 2015, from the Nielsen Norman Group Web site: