1. U.S. Department of Agriculture eGovernment Program eGovernment Working Group Meeting Chris Niedermayer, USDA eGovernment Executive January 14, 2004

2. U.S. Department of Agriculture eGovernment Program 2  Welcome  Introduction: Usability Studies  Usability Studies at USDA:  USDA.gov  FSIS  ERS  Q&A and Next Steps  Appendix Agenda

3. U.S. Department of Agriculture eGovernment Program 3 What is Usability? Usability is the overall measurement of the satisfaction of interactions a user has with a product or system such as a Web site, software application, mobile technology, or user-operated device.  A high level of satisfaction occurs when users find a Web site that is useful and usable; this is termed an “engaging user experience”.  Quality of the user’s experience on a Web site can be measured using the following usability principles, or heuristics:  Use a simple page design  Know your users and speak their language  Ensure consistency throughout the site  Design your site for efficiency and ease of use  Provide an engaging experience  Support the user and provide feedback  Use a clear navigation structure

4. U.S. Department of Agriculture eGovernment Program 4 Why is Usability Important? An engaging user experience on a particular Web site results in:  Increased usage of the Web site  Improved user productivity  Reduced training, support, and development costs

5. U.S. Department of Agriculture eGovernment Program 5  Welcome  Introduction: Usability Studies  Usability Studies at USDA:  USDA.gov  FSIS  ERS  Q&A and Next Steps  Appendix Agenda

6. U.S. Department of Agriculture eGovernment Program 6 USDA.gov: Usability Approach The USDA.gov team conducted usability testing with a sample user community to test the effectiveness and ease of use of the redesigned site.  Test participants were selected to represent segments of USDA’s vast customer base. Participants included Agribusiness and Cooperatives, Consumers and Families, Educators. Producers, and Researchers, to name a few.  Feedback obtained from participants during usability testing focused on the following: Is the page design simple and easy to follow, even for inexperienced Web users? Does the site provide users with content written for them, using wording/language they understand? Is the site consistent throughout? Is the site designed to promote efficiency and ease of use, enabling users to achieve their goals on the site? Does the site provide interesting information, the foundation for an engaging user experience? Does the site support users and provide sufficient feedback to help them complete tasks? Does the site use a clear navigation structure that users understand? Does the site offer useful, convenient functionality to allow users to perform tasks such as subscribing to email notifications or personalizing their My USDA page?

7. U.S. Department of Agriculture eGovernment Program 7 USDA.gov: Test Structure Each test session enabled the participant to explore and provide feedback on the features and functionality of the usda.gov site.  Test participants used an “HTML mockup” as their interaction mechanism.  A sample test session was comprised of the following tasks: 1.Review and browse through the home page. 2.Find specific pieces of information relevant to your user group. For example: Producers were first instructed to locate information on crop insurance and then on export requirements for a product. Consumers/Families were asked to first locate information on food assistance programs for children and then on park permits for an upcoming trip. 3.Find a news article, event, and report. 4.Customize your My USDA page.  Following each session, test observers participated in a debrief session to highlight usability issues and other observations from the test.  Final test data was compiled into a results matrix. All items in the matrix were prioritized based on their usability impact. A design recommendation was proposed for each issue detailed in the matrix.

8. U.S. Department of Agriculture eGovernment Program 8 Key Finding – Navigation  Participants often did not notice the Information For dropdown box.  It was common for participants to glance over the box altogether, since they commonly associated it with the My USDA section of the site.  Some participants thought they had to be logged in to the site to use the Information For box. Recommendation:  Information For should be positioned in its own area, with the topic heading "Browse by Audience".  This will draw more attention to the box and separate it from My USDA. If possible, this issue should be re-tested after we create a design in which the user will more readily be able to find the Information For area and explore its benefits. USDA.gov: Key Findings and Recommendations 1

9. U.S. Department of Agriculture eGovernment Program 9 Key Finding – Navigation  Participants tended to glance over the right navigation menu, and sometimes missed that it was related to the page-specific content.  Additionally, some participants felt the right navigation menu was in conflict with the left navigation menu and it was cumbersome to navigate between the two of them or understand what they should be using. Recommendation:  Coordinate graphically the left navigation and top navigation menu to highlight correlation and express the higher precedence of those menus over the right menu.  Make left navigation graphical links to further increase its precedence graphically over the right menu.  Better integrate graphically the right menu in to the content area. USDA.gov: Key Findings and Recommendations 2

10. U.S. Department of Agriculture eGovernment Program 10 Additional findings are documented in the Appendix.

11. U.S. Department of Agriculture eGovernment Program 11  Welcome  Introduction: Usability Studies  Usability Studies at USDA:  USDA.gov  FSIS  ERS  Q&A and Next Steps  Appendix Agenda

12. U.S. Department of Agriculture eGovernment Program 12 FSIS: Goals of Testing The objective of this design evaluation was to assess usability issues that impact the user experience for www.fsis.usda.gov.www.fsis.usda.gov  FSIS Usability Testing enabled the re-design team to inform their design process with relevant customer feedback. Layout- Tests helped confirm that proposed layout was logical for users and reflected their needs. Navigation- Good navigation allowed users to find information quickly. It provided subtle clues on what the site offered, and where users were on the site. Slow and confusing navigation forced users off the site. Content- Usability tests helped the re-design team analyze user understanding and opinion of defined content areas. Tests also gave insight into the accuracy and clarity of the labels used to identify that content.

13. U.S. Department of Agriculture eGovernment Program 13 FSIS: Usability Approach FSIS followed a defined Usability Testing process to ensure consistent, conclusive results.  Conducted seven one-hour test sessions on Thursday, October 30 th, with participants from various pre-determined site user groups. Scientists / Researchers Constituent Groups Consumers / Educators Businesses  Tests used a paper-based prototype of the re-designed site. The facilitator used detailed test scripts to lead users through each session.  Facilitator encouraged participants to ‘think aloud’ during the tests. Their physical actions, and verbal feedback were video-taped and logged.  Final test data was compiled and a design recommendation was proposed for each usability issue.

14. U.S. Department of Agriculture eGovernment Program 14 FSIS: Key Findings and Recommendations 1 Background  Consumer focused articles, one-page fact sheets, and food safety newsletters were originally grouped under the global navigation item “Publications”. Key Finding – Publications  Users were not clear on what to expect of the “Publications” section when reading the label. Some thought the term implied professional or technical documents.  When asked to find an article on the safe preparation of turkey users did not think to look in this area. Recommendation  Remove “Publications” from the global navigation bar.  All consumer publications, backgrounders, and topics of interest should instead be labeled as “Fact Sheets”.  Science related publications, such as guidebooks, should be included in the “Science” area of the site.

15. U.S. Department of Agriculture eGovernment Program 15 FSIS: Key Findings and Recommendations 2 Background  FSIS content is separated into five main content areas- Education, Science, Regulations & Policies, Food Recalls, Food Security & Emergency Preparedness.  Originally all consumer education information AND employee education and training information was located in this “Education” area. Key Finding – Workforce Training  Users did not understand why “Workforce Training” was included in the “Education” area of the site. Recommendation  Remove FSIS employee specific information from the “Education” area aimed at consumers and educators.  “FSIS Workforce Training” content should instead be included in the “FSIS Employee” area of the site.

16. U.S. Department of Agriculture eGovernment Program 16 Usability sessions informed more than a dozen significant design changes. Additional findings are documented in the Appendix.

17. U.S. Department of Agriculture eGovernment Program 17  Welcome  Introduction: Usability Studies  Usability Studies at USDA:  USDA.gov  FSIS  ERS  Q&A and Next Steps  Appendix Agenda

18. U.S. Department of Agriculture eGovernment Program 18 ERS intends to regularly test its online services and products for:  Performance,  Quality,  Usability, and  Value to users Appointed a “Web Analytics Manager” to:  Track / analyze site usage trends  Perform usability testing and expert (heuristic) analysis of ERS web products  Train and coach staff on usability and other user assessment techniques  Manage ERS’ usability testing facilities and equipment  Brief, "sell,” and educate ERS managers, technical experts and program executives on performance measurement, user assessment, and customer relationship management initiatives  Implement an integrated Web performance measurement system ERS: Usability Approach

19. U.S. Department of Agriculture eGovernment Program 19 ERS: Methodology “Guerilla” Usability Testing:  Conducted on-site at ERS  Follows testing protocol  Good for agencies with limited resources  Produces valuable results Testing protocol includes:  Preliminary interview using scripts  Specific tasks  Post-test debriefing

20. U.S. Department of Agriculture eGovernment Program 20 Agricultural Resource Management Survey Test  Objective: Increase user access to actual ARMS data Create an easy-to-use and rich interface to enable users to understand the data Support ERS decision makers, enabling them to better understand how customers “use” the data Address specific challenges faced by users of ARMS data products  Lessons Learned: Need to improve access to ARMS data by Showing user what’s available on the page Enabling users to see shortcuts and providing more visual cues to take people to the next step Cutting text, chunking text, incorporate more headings, subheadings Need to improve the ARMS Data tool by Improving the structure at the programming and interface level Setting expectations: say upfront what you can and cannot get Graying out areas that are not available Crafting “self-evident” help Clarifying terms  Outcome: Received management buying on new prototype Ready for second round of testing

21. U.S. Department of Agriculture eGovernment Program 21 Latest Test: Redesigned Data Page  Objective: To qualify difficulties in usability so team can prioritize choices for redevelopment  Lessons Learned: Users didn’t pay much attention to headings Users didn’t understand or care about the key indicators section Users saw only one search box on the page, even though there are two Users want lists--they want to find info quickly and easily  Outcome: Will make “easy” cosmetic changes Are developing a data classification scheme Will test the new classification scheme on the data page before doing too much work

22. U.S. Department of Agriculture eGovernment Program 22  Welcome  Introduction: Usability Studies  Usability Studies at USDA:  USDA.gov  FSIS  ERS  Q&A and Next Steps  Appendix Agenda

23. U.S. Department of Agriculture eGovernment Program 23 Questions and Answers

24. U.S. Department of Agriculture eGovernment Program 24 Next Steps  Revised Agency eGovernment Tactical Plans likely due in 2 nd Quarter of FY 2004. Schedule Agency eGovernment Steering Committee Meetings for first week of February.  Upcoming EGWG Topics: Usability Panel II: January 21 st, 2004. USDA Smart Choice Enablers Update: February 4 th, 2004.

25. U.S. Department of Agriculture eGovernment Program 25  Welcome  Introduction: Usability Studies  Usability Studies at USDA:  USDA.gov  FSIS  ERS  Q&A and Next Steps  Appendix Agenda

26. U.S. Department of Agriculture eGovernment Program 26  Usability Studies Statistics ………………………………… 27  Presenter Contact Information ……………………………. 28  USDA.gov Testing Methodology ………………………….. 29  USDA.gov Additional Key Findings ………...……………. 30  FSIS Additional Key Findings …………………………...… 32  ERS Personas ………………………………………………... 35  ERS Additional Key Findings ……………………………... 37 Appendix

27. U.S. Department of Agriculture eGovernment Program 27 Why is Usability Important? An engaging user experience on a particular Web site results in increased usage of that site, improved user productivity, and reduced training, support, and development costs. There is always room for improvement…  The average usability test results in 70-100 recommendations for improvements. 1  Users cannot find the information they seek on Web sites 60% of the time leading to wasted time, reduced productivity, increased frustration and loss of repeat visits and money. 2  95% of users will use less than 5% of features of a site; 75% of the features will never be used. 3  In private industry, 62% of online shoppers gave up at least once while looking for products. 4  For every $10 spent defining and solving critical usability problems early in development using professional usability research, you will save about $100 in development costs. 5 1 Mary Harrison, Richard Henneman, and Louis Blatt, “Design of Human Factors Cost Justification Tool” 2 Nielsen Norman Group, “Usability Return on Investment” 3 MauroNewMedia, Charles Mauro 4 Internet Access in U.S. Public Schools and Classrooms, National Center for Education Statistics 5 MauroNewMedia, Charles Mauro

28. U.S. Department of Agriculture eGovernment Program 28 Usability Studies at USDA  USDA.gov Kim Taylor, Office of Communications, Director of Web Services Kim.J.Taylor@usda.gov, (202) 720-2168Kim.J.Taylor@usda.gov  Food Safety and Inspection Service (FSIS) Holly McPeak, Public Affairs Specialist holly.mcpeak@fsis.usda.gov, (301) 504-0168holly.mcpeak@fsis.usda.gov  Economic Research Service (ERS) Brenda Powell, Web Analytics Manager bapowell@ers.usda.gov, (202) 694-5134bapowell@ers.usda.gov

29. U.S. Department of Agriculture eGovernment Program 29 USDA.gov: Methodology The USDA.gov team followed a consistent methodology for usability testing that is based on industry best practices and previous testing experience. GATHER INPUTS PRODUCE OUTPUTS Conduct pre- session briefing Conduct participant test session Debrief PREPARE AND EXECUTE TEST 6-7x Edit video Create presentation Deliver Results Create recruiting guide & recruit Create test materials Conduct trial session Review site / application Define test objectives 1 – 2 days Determine test format and participant profile 7 – 10 days 3 days Gather Inputs Prepare And Execute Test Produce Outputs C Compile Test Data

30. U.S. Department of Agriculture eGovernment Program 30 USDA.gov: Key Findings and Recommendations 3 Key Finding – My USDA  A large number of participants interviewed would use the My USDA customization feature.  Thirteen out of sixteen participants said that they would use this feature to personalize their USDA home page. Recommendation:  Promote the benefits of My USDA customization from the home page and continue to feature it prominently on the navigation bar.

31. U.S. Department of Agriculture eGovernment Program 31 Key Finding – Overall  Inexperienced participants were sometimes confused as to how to enter and use the site. Recommendation:  Through use of color, graphical text and simple icons, make the primary navigation areas of the site appear "clickable".  Use the I Want To section to draw inexperienced users into the site. USDA.gov: Key Findings and Recommendations 4

32. U.S. Department of Agriculture eGovernment Program 32 FSIS: Key Findings and Recommendations 3 Background  Speeches and communications to Congress were originally only located with other news and events information on the top navigation bar. Key Finding – Speeches  Users (specifically those from Businesses and Constituent Groups) verbalized a desire to read through recent speeches and presentations from agency leaders.  They also desired quick access to communications distributed to Congress. Recommendation  Speeches should be made available to users from both the “News and Events” area, as well as the “Businesses”, and “Constituent Groups” audience landing pages.

33. U.S. Department of Agriculture eGovernment Program 33 FSIS: Key Findings and Recommendations 4 Background  The proposed homepage prominently features two recent agency news releases.  These releases will be updates several times each week. Key Finding – Recall Releases  Knowledgeable users recognize that FSIS releases recall related news frequently.  These users did not think value was added by displaying recall releases within the “In the News” section of the homepage. Recommendation  The “In the News” area should be reserved for non- recall related news releases.  Recall releases can still be accessed through both the FSIS Recalls section and the News and Events section of the re-designed site.

34. U.S. Department of Agriculture eGovernment Program 34 FSIS: Key Findings and Recommendations 5 Key Finding - Depth  Users were surprised at the depth of information on the site.  There is a significant amount of content included that they would not originally think to find at FSIS. Recommendation  Promote “value add” information more clearly on the homepage.  Content that is not necessarily top of mind for users could be rotated through the promotional area on the homepage as a content marketing technique.

35. U.S. Department of Agriculture eGovernment Program 35 ERS: Development of Personas Personas are not real people, but hypothetical archetypes used to classify and represent a specific group of users.  Provide focus for design decisions  Describe users’ needs and tasks  Help decide what groups to test

36. U.S. Department of Agriculture eGovernment Program 36 An example of a persona USDA Senior Management Gatekeeper: Matthew Johnson Program Staff Director, USDA Background 51-years-old Married, 3 children, 1 grandchild Ph.D. in Agricultural Economics Comfortable using a computer, intermediate Internet user, with a T1 connection at work and dial-up at home Uses email extensively; uses the Web about 1.5 hours a day Informational goals Narrow subject focus Seeks decision level information Prefers summaries, but also quick access to specific information Access to experts in subject matter Seeks recent developments and publications in areas of interest Values timeliness, credibility and accuracy of information

37. U.S. Department of Agriculture eGovernment Program 37 ERS Usability Test: The Food borne Illness Calculator  Objective: To see whether the users understand the layout of the site and the functions of the calculator.  Lessons Learned: Product builders learned that the users had problems in the following areas: Navigation Page layout Visual clues Text  Outcome: The application developers incorporated many of the suggestions into the calculator. The calculator is easier to use and understand now than when it was first built.