1. 0 Usability Testing Resources Demetrios Karis, Ph.D. demetrios.karis@gmail.com November, 2015 Please let me know if you find bad links or have suggestions for other good sources of information

2. 1 © 2015 Karis Usability Video Example: Formative Usability http://www.youtube.com/watch?v=QckIzHC99Xc Search: “Rocket Surgery Made Easy by Steve Krug: Usability Demo” Time 0:00 - 1:00: Introduction to the video by Steve Krug 1:00 - 3:15: Introduction/Instructions to the participant 3:15 - 6:00: Initial questions to participant; e.g., about web usage 6:00 - 8:10: Home page tour; “Tell me what you think about this home page” 8:10 - 13:45: Task 1. “You need a car for...how much would this cost…” 13:45 - 20:30:Task 2: Availability of cars 20:30 - 20:50:End of session, payment to participant 20:50 - 23:25:Steve Krug’s presentation of the top three usability problems 23:25 - 24:26:Simple design changes that could address problems & final words Evaluation of the Zipcar website

3. 2 © 2015 Karis Usability Notes on Video Example Common moderating mistake during initial questions when Steve asks about how much time the participant spends on email versus web browsing: “About half and half?” This is a leading question. Home page tour: this is a common initial task. An alternative is to also let the participant go anywhere they want on the site. Task 1: This is a very complex task. Most tasks are not this complex (Task 2 is more common). Nothing is wrong with complex tasks if they represent what a user might try to do on the site. However, for a complex task, better to give it to the participant in a written form as well as reading it (which Steve does). Top three usability problems: Not all usability experts would agree that these are the top three, but I think all would agree that the problems identified are indeed problems. Design changes to address problems: These, of course, should be tested, but I think that they are elegant solutions that would be easy to implement.

4. 3 © 2015 Karis Usability Good Books, Practical Advice Lots of books, articles, websites, tutorials, templates, etc. at end in Resource Material

5. 4 © 2015 Karis Usability “Simple, informal, small-sample, do-it- yourself usability testing”

6. 5 © 2015 Karis Usability Paper Prototyping Handout Search online for “paper prototyping” and then select “Images”. Page setup dialog from Microsoft Word

7. 6 © 2015 Karis Usability Sample Documents Supplementary material for Handbook of Usability Testing http://www.wiley.com/go/usabilitytesting –“Downloads” in middle of page, then Download All Content Sample Test Plan Testing Preparation (checklists) Session Script Recording Consent Form Background Questionnaire Post-test Rating Scale/Questionnaire Guidelines for Observers Excel Spreadsheets for Recording Data

8. 7 © 2015 Karis Usability Sample Documents (2) Companion site for Moderating Usability Tests: http://www.elsevierdirect.com/v2/companion.jsp?ISBN=9780123739339 http://www.elsevierdirect.com/v2/companion.jsp?ISBN=9780123739339 PDF Files: –Screening questions –Chapter 5: Understanding your participation in this study –Checklist for pretest briefing –Self-administered questionnaire example –Post-session interview example –Chapter 8: Understanding your participation in this study –Electronic consent agreement

9. 8 © 2015 Karis Usability Sample Documents (3) Usability.gov Templates and downloadable documents (& lots of information) –http://www.usability.gov/how-to-and-tools/resources/templates.html#Usabilityhttp://www.usability.gov/how-to-and-tools/resources/templates.html#Usability –Consent forms –Test plans –Moderating guides –Scenarios and task lists –Many more Steve Krug, http://www.sensible.com/downloads-rsme.htmlhttp://www.sensible.com/downloads-rsme.html Usability test script, checklists, instructions for observers, etc. Also included in his books Handout

10. 9 © 2015 Karis Usability Comparative Usability Evaluation (CUE) Four most important findings “The number of usability problems in a typical website is often so large that you can’t hope to find more than a fraction of the problems in an ordinary usability test.” “Six - or even 15 - test participants are nowhere near enough to find 80% of the usability problems. Six test participants will, however, provide sufficient information to drive a useful iterative development process.” “There’s no measurable difference in the quality of the results produced by usability tests and expert reviews.” “Even professional usability evaluators make many mistakes in usability test task construction, problem reporting, and recommendations.” http://www.dialogdesign.dk/CUE.html

11. 10 © 2015 Karis Usability UXPA* Code of Professional Conduct Ethical Principles Act in the Best Interest of Everyone Be Honest with Everyone Do No Harm and If Possible Provide Benefits –“Usability practitioners shall not expose participants to any unreasonable physical, mental or emotional stress.” Act with Integrity Avoid Conflicts of Interest Respect Privacy, Confidentiality, and Anonymity –“Usability practitioners shall not reveal information that identifies colleagues or participants without their permission and shall take reasonable precautions to avoid such information from being disclosed unintentionally.” Provide All Resultant Data * UXPA = User Experience Professionals Association

12. 11 © 2015 Karis Usability Videos on Moderating Videos associated with Moderating Usability Tests: http://www.elsevierdirect.com/v2/companion.jsp?ISBN=9780123739339 http://www.elsevierdirect.com/v2/companion.jsp?ISBN=9780123739339 –Staged videos, each a 5-7 minute segment of a test session Potentially dangerous! (they include poor practices on purpose) –See Moderating Usability for my commentary and adviceModerating Usability –https://docs.google.com/document/d/1Nu9iY7nF0PPc44xHXRijzXQNyEpuIPqFPJvHvT9kroo/edit?usp=sharinghttps://docs.google.com/document/d/1Nu9iY7nF0PPc44xHXRijzXQNyEpuIPqFPJvHvT9kroo/edit?usp=sharing –Watch a video, read corresponding discussion in book and my commentary, watch the panel discussing the video, then watch video again Role-playing videos 1.Pre-test briefing with a checklist 2.Pre-test briefing following a script 3.Interacting during the session (1) 4.Interacting during the session (2) 5.Interacting in a remote testing session 6.Post-test interview Discussion panel videos –Introduction to the authors and the expert panel –Expert panel’s reaction to the pre-test briefing videos –Expert panel’s reaction to the interacting during the session (1) –Expert panel’s reaction to the interacting during the session (2) –Expert panel’s reaction to the remote testing session video –Expert panel’s reaction to the post-test video

13. 12 © 2015 Karis Usability Mechanical Turk “Amazon Mechanical Turk is a marketplace for work that requires human intelligence.” From Tutorial: http://ir.ischool.utexas.edu/wsdm2011_tutorial.pdfhttp://ir.ischool.utexas.edu/wsdm2011_tutorial.pdf Requesters create “Human Intelligence Tasks” (HITs) via web services API or dashboard Workers (sometimes called “Turkers”) log in, choose HITs, perform them Requesters assess results, pay per HIT satisfactorily completed From Mturk site (March 2013): 500,000 workers from 190 countries; millions of HITs completed Many published articles, e.g.: Aniket Kittur, Ed H. Chi, Bongwon Suh. Crowdsourcing User Studies With Mechanical Turk. In Proceedings of the ACM Conference on Human-factors in Computing Systems (CHI2008), pp.453-456. ACM Press, 2008. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data? Perspectives on Psychological Science, 6(1). Conclusion: “Overall, Mturk can be used to obtain high-quality data inexpensively and rapidly.”

14. 13 © 2015 Karis Usability Paper Prototyping Mentioned previously –Snyder, C. (2003). Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces. Video example –http://www.nngroup.com/reports/paper-prototyping-training-video/http://www.nngroup.com/reports/paper-prototyping-training-video/ –40 minute training video, $68 Tools –iPhone prototyping with POP, https://popapp.in/https://popapp.in/ Design on paper Take pictures Simulate: create a storyboard by linking your sketches together

15. 14 © 2015 Karis Usability Prototyping Tools List of 25 tools –http://memeburn.com/2012/09/25-of-the-best-wireframing-and- prototyping-tools-ever/http://memeburn.com/2012/09/25-of-the-best-wireframing-and- prototyping-tools-ever/ Axure –Highly recommended, widely used –http://www.axure.com/http://www.axure.com/

16. 15 © 2015 Karis Usability References: Books, Articles, & Websites Information on Multiple Topics –BoK “The Usability Body of Knowledge (BoK) project is dedicated to creating a living reference that represents the collective knowledge of the usability profession.” Some information and references now, but still in early stages. Eventually it will be very useful. http://www.usabilitybok.org http://www.usabilitybok.org –Wikipedia Small star icon on top right corner indicates article is featured –“Featured articles are considered to be the best articles in Wikipedia, as determined by Wikipedia’s editors. Before being listed here, articles are reviewed at featured article candidates for accuracy, neutrality, completeness, and style according to our featured article criteria. There are 3,153 featured articles out of 3,538,010 articles on the English Wikipedia. Thus, about one in 1,120 articles is listed here.” + indicated a good article –Good articles meet a core set of editorial standards but are not featured article quality Wikipedia continues to improve –APS now has a Wikipedia initiative to rewrite /improve articles on psychological topics http://www.psychologicalscience.org/index.php/members/aps-wikipedia-initiative –Mahzarin Banaji, “Harnessing the Power of Wikipedia for Scientific Psychology: A Call to Action” (APS Observer, Vol. 24, No. 2,, February 2011, page 5). Update: Over 3,300 psychological scientists and their students have joined the Wikipedia Initiative (Jan. 2014)

17. 16 © 2015 Karis Usability References Highly Cited Books on Usability and User Experience Albert, B., Tullis, T., & Tedesco, D. (2010). Beyond the Usability Lab: Conducting Large-scale Online User Experience Studies. DeMarco Brown, D. (2012). Agile User Experience Design: A Practitioner’s Guide to Making it Work. Dumas, J., & Loring, B. (2008). Moderating Usability Tests: Principles & Practices for Interacting. Fogg, B. J. (2003). Persuasive Technology. Hearst, M. A. (2009). Search User Interfaces. –Can read entire book free on the web! Krug, S. –(2014). Don’t Make me Think, Revisited: A Common Sense Approach to Web Usability. –(2009). Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems

18. 17 © 2015 Karis Usability References (2) Highly Cited Books on Usability and User Experience (cont.) Norman. D –(2002). The Design of Everyday Things. –(2005). Emotional Design: Why We Love (or Hate) Everyday Things Rubin, J., & Chisnell, D. (2008). Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Second Edition. Snyder, C. (2003). Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces. Tullis, T., & Albert, B. (2008). Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics.

19. 18 © 2015 Karis Usability References (3) Usability: Managing UX and Organizational Issues Lund, A. (2011). User Experience Management: Essential Skills for Leading Effective UX Teams. Morgan Kaufmann. Bias, R. G., & Mayhew, D. J. (2005). Cost-Justifying Usability: An Update for the Internet Age. Elsevier (Second Edition).

20. 19 © 2015 Karis Usability References (4) Expert Reviews vs. Traditional Usability Testing Molich, R., & Dumas, J. (2008). “Comparative Usability Evaluation (CUE-4),” Behaviour & Information Technology, Vol. 27(3). –CUE-1 to CUE-9: See http://www.dialogdesign.dk/CUE.htmlhttp://www.dialogdesign.dk/CUE.html Low Fidelity Prototyping Snyder, C. (2003). Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces. Morgan Kaufmann Publishers. Virzi, R. A., Sokolov, J., & Karis, D. (1996). Usability Problem Identification Using Both Low- and High-Fidelity Prototypes. Proceedings of the Conference on Human Factors in Computing Systems (CHI 96, 236-243) ACM. Also –Buxton, B. (2007). Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann Publishers. –Warfel, T. Z. (2009). Prototyping: A Practitioner’s Guide. Rosenfeld Media.

21. 20 © 2015 Karis Usability References (5) UX and Agile Diana Brown (2012). Agile User Experience Design: A Practitioner's Guide to Making It Work. Janice Fraser (2014). The Lean Product Book: How Smart Teams Work Better. Jeff Gothelf & Josh Seiden (2013). Lean UX: Applying Lean Principles to Improve User Experience. Laura Klein (2013). UX for Lean Startups: Faster, Smarter User Experience Research and Design. Lindsay Ratcliffe & Marc McNeill (2011). Agile Experience Design: A Digital Designer's Guide to Agile, Lean, and Continuous.

22. 21 © 2015 Karis Usability References: Mechanical Turk “Amazon Mechanical Turk is a marketplace for work that requires human intelligence.” From Horton & Chilton (2010): median hourly wage = $1.38 Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data? Perspectives on Psychological Science, 6(1). Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research. PLoS ONE 8(3):e57410. doi:10.1371/journal.pone.0057410 Horton, J. J., & Chilton, L. B. (2010). The labor economics of paid crowdsourcing. In Proceedings from EC ‘10: The 11 th ACM Conference on Electronic Commerce (pp. 209-218). New York, NY: ACM Tutorial: http://ir.ischool.utexas.edu/wsdm2011_tutorial.pdfhttp://ir.ischool.utexas.edu/wsdm2011_tutorial.pdf

23. 22 © 2015 Karis Usability References: Mechanical Turk (2) Kittur, A., Chi, E. H., Suh, B. (2008). Crowdsourcing User Studies With Mechanical Turk. In Proceedings of the ACM Conference on Human-factors in Computing Systems (CHI2008), pp.453- 456. ACM Press. Kittur, A., Nickerson, J. V., Bernstein, M. S., Gerber, E. M., Shaw, A., Zimmerman, J., Lease, M., & Horton, J. J. (2013). The Future of Crowd Work. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW ’13), pp. 1301 – 1317. ACM Press. Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44, 1-23. Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013). Using Mechanical Turk to Study Clinical Populations. Clinical Psychological Science, 1 (2), 213 – 220.

24. 23 © 2015 Karis Usability Usability Resources: Websites UPA (now UXPA) –http://www.upassoc.org/http://www.upassoc.org/ SIGCHI –http://www.sigchi.org/resourceshttp://www.sigchi.org/resources Usability.gov –http://usability.gov/http://usability.gov/ Usability Net –http://www.usabilitynet.org/home.htmhttp://www.usabilitynet.org/home.htm Society for Technical Communication Usability & User Experience Community –http://www.stcsig.org/usability/index.htmlhttp://www.stcsig.org/usability/index.html Human Factors and Ergonomics Society –https://www.hfes.org//Web/Default.aspxhttps://www.hfes.org//Web/Default.aspx –Publications, standards, educational resources The Encyclopedia of Human-Computer Interaction –http://www.interaction-design.org/books/hci.htmlhttp://www.interaction-design.org/books/hci.html –Free online, and includes image library BostonCHI has videos of monthly lectures available here: http://www.bostonchi.org/videos/ http://www.bostonchi.org/videos/

25. 24 © 2015 Karis Usability Usability Resources: Websites (2) Measuring the User Experience: A companion website to the book by Tom Tullis and Bill Albert –http://measuringuserexperience.com/http://measuringuserexperience.com/ Jeff Sauro’s site, Measuring Usability –http://www.measuringusability.com/index.phphttp://www.measuringusability.com/index.php –Includes calculators for A/B comparisons, sample size, and more Templates, Forms, and Examples –For example: Test report templates, questions to ask at a kick-off meeting, a usability test plan –http://www.usability.gov/templates/index.html#Usabilityhttp://www.usability.gov/templates/index.html#Usability –www.wiley.com/go/usabilitytestingwww.wiley.com/go/usabilitytesting –http://www.stcsig.org/usability/resources/toolkit/toolkit.htmlhttp://www.stcsig.org/usability/resources/toolkit/toolkit.html

26. 25 © 2015 Karis Usability Usability/UX Blogs & Newsletters/Articles http://boxesandarrows.com/ http://uxmag.com/ http://www.uie.com/ http://usabilitytesting.wordpress.com/ Nielsen Norman Group –Jacob Nielsens Alertbox newsletter: http://www.nngroup.com/articles/http://www.nngroup.com/articles/ –Don Norman’s articles: http://jnd.org/http://jnd.org/ –Bruce “Tog” Tognazzini’s articles: http://www.asktog.com/http://www.asktog.com/

27. 26 © 2015 Karis Usability Mobile User Experience Android mobile design –http://developer.android.com/design /index.htmlhttp://developer.android.com/design /index.html iOS Human Interface Guidelines –iPhone OS –http://developer.apple.com/library/io s/#documentation/UserExperience/ Conceptual/MobileHIG/Introduction/ Introduction.htmlhttp://developer.apple.com/library/io s/#documentation/UserExperience/ Conceptual/MobileHIG/Introduction/ Introduction.html Windows Mobile Design Guidelines –http://msdn.microsoft.com/en- us/library/bb158602http://msdn.microsoft.com/en- us/library/bb158602 LukeW Ideation & Design http://www.lukew.com/ Some good information about mobile design; you can download presentations and there’s a touch gesture reference guide at http://www.lukew.com/ff/entry.asp? 1071 http://www.lukew.com/ff/entry.asp? 1071 MEX (Mobile User Experience) Conference and blog/newsletter http://www.pmn.co.uk/mex/ We’ve never been to the conference, but the blog & newsletter are often very interesting

28. 27 © 2015 Karis Usability Mobile User Experience (2) Design principles for BlackBerry devices –http://docs.blackberry.com/en/developers/deliverables/20196/Design_principles_f or_BB_devices_2_0_545792_11.jsphttp://docs.blackberry.com/en/developers/deliverables/20196/Design_principles_f or_BB_devices_2_0_545792_11.jsp Forum.Nokia UX & Mobile Design –http://www.forum.nokia.com/Design/User_experience_program/http://www.forum.nokia.com/Design/User_experience_program/ –http://wiki.forum.nokia.com/index.php/Category:Mobile_Designhttp://wiki.forum.nokia.com/index.php/Category:Mobile_Design HP Palm Human Interface Guidelines –http://developer.palm.com/index.php?option=com_content&view=article&id=1836 &Itemid=52http://developer.palm.com/index.php?option=com_content&view=article&id=1836 &Itemid=52 Mobile Web Resources –http://mobilewebbestpractices.com/resources/http://mobilewebbestpractices.com/resources/

29. 28 © 2015 Karis Usability Slideshare –Slideshare (e.g., search for ethnography or usability testing) http://www.slideshare.net/ Some excellent decks…some mediocre Most Boston UPA Conference decks are there

30. 29 © 2015 Karis Usability Usability Testing Tools & Technology Jen McGinn, Usability Testing Tools and Technology for Under $100 (Video and slides) http://igniteshow.com/videos/usability-testing-tools-and-technology-under-100 http://igniteshow.com/videos/usability-testing-tools-and-technology-under-100 Running Remote Sessions –WebEx –GoToMeeting –LiveMeeting –Techsmith’s Screencast –Fuze Meeting –Adobe Connect –JoinMe (https://join.me/)https://join.me/ –Mikogo (http://www.mikogo.com/)http://www.mikogo.com/ Screen Recording Software –BSR Screen Redorder http://bsrsoft.com/ –Camtasia Studio http://www.techsmith.com/download/camtasia/default.asp –Silverback (http://silverbackapp.com/)http://silverbackapp.com/

31. 30 © 2015 Karis Usability Usability Testing Tools & Technology (2) Remote unmoderated usability testing –Aka: asynchronous, automated –Vendors (some provide videos of participants interacting with your site) Loop11: http://www.loop11.com/http://www.loop11.com/ Userlytics: http://www.userlytics.com/http://www.userlytics.com/ Keynote Webeffective: http://www.keynote.com/products/customer_experience/web_ux_research_tools/webeffective.html http://www.keynote.com/products/customer_experience/web_ux_research_tools/webeffective.html UserZoom: http://www.userzoom.com/http://www.userzoom.com/ Webnographer: http://www.webnographer.com/http://www.webnographer.com/ OpenHallway: http://openhallway.com/http://openhallway.com/ UserTesting.com: http://www.usertesting.com/http://www.usertesting.com/ Whatusersdo: http://whatusersdo.com/http://whatusersdo.com/ UserFeel: http://www.userfeel.com/http://www.userfeel.com/ Kupima: http://kupima.com/http://kupima.com/

32. 31 © 2015 Karis Usability Usability Testing Tools & Technology (3) Mobile Testing –Using a camera to record screen More versatile, includes finger movements (whether clicking, swiping, or pinching is often important) Document cameras (examples): IPEVO or Elmo –See Mr. Tappy for nifty camera mount –Locking focus on camera often recommended –Turning off overhead lights may help –Mirror the screen (reflections apps) Participant can move around more naturally Captures everything on screen, better picture quality Can use a camera to record hand and body movements For iOS –http://www.reflectorapp.com/http://www.reflectorapp.com/ AirPlay mirrors your iPhone or iPad to any Mac or PC, wirelessly. View your iOS device on your Mac and continue using it while you're mirroring. Easily capture the screen of your iOS device and save it as a video file for later review. –Use the phone’s camera and mic! www.uxrecorder.com www.magitest.com http://lookback.io/ : “See the screen, gestures, face, & voice of your test users”http://lookback.io/

33. 32 © 2015 Karis Usability Usability Testing Tools & Technology (4) Presentation on mobile testing with lots of details and practical advice: http://www.slideshare.net/UPABoston/usability-testing-on-mobile- devices-no-more-excuseshttp://www.slideshare.net/UPABoston/usability-testing-on-mobile- devices-no-more-excuses

34. 33 © 2015 Karis Usability Usability Testing Tools & Technology (5) Eye Tracking SMI –http://www.smivision.com/en/gaze-and-eye-tracking-systems/home.htmlhttp://www.smivision.com/en/gaze-and-eye-tracking-systems/home.html Tobii –http://www.tobii.com/http://www.tobii.com/ SR Research –http://www.sr-research.com/http://www.sr-research.com/ How to Conduct Eye Tracking Studies by Kara Pernice and Jakob Nielsen (2009) –http://media.nngroup.com/media/reports/free/How_to_Conduct_Eyetrac king_Studies.pdfhttp://media.nngroup.com/media/reports/free/How_to_Conduct_Eyetrac king_Studies.pdf

35. 34 © 2015 Karis Usability Usability Testing Tools & Technology (6) Usability Hub, https://usabilityhub.com/https://usabilityhub.com/ –Five second test, http://fivesecondtest.comhttp://fivesecondtest.com Are your landing pages easy to understand? Five second tests help you understand people’s first impressions of your designs. By finding out what a person recalls about your design in just 5 seconds you can ensure that your message is being communicated as effectively as possible. –Click test Where do users click? –Nav flow Is your website easy to navigate?

36. 35 © 2015 Karis Usability Usability Labs Usability labs –Rentals and participant recruiting http://www.stcsig.org/usability/topics/recruiting-firms.html –Building or renting equipment http://www.stcsig.org/usability/topics/usabilty-labs.html http://www.quirks.com/ –Directories of lab facilities –“Here you'll find free directories of 7,000+ market research companies and focus group facilities, 3,000+ market research articles, market research jobs, events, online research and much more!”

37. 36 © 2015 Karis Usability Usability Metrics Tullis and Stetson review article on Usability questionnaires –www.upassoc.org/usability_resources/.../UPA-2004-TullisStetson.pdfwww.upassoc.org/usability_resources/.../UPA-2004-TullisStetson.pdf System Usability Scale (SUS) –http://en.wikipedia.org/wiki/System_Usability_Scalehttp://en.wikipedia.org/wiki/System_Usability_Scale Questionnaire for User Interaction Satisfaction (QUIS) –http://lap.umd.edu/quis/http://lap.umd.edu/quis/ Website Analysis and MeasureMent Inventory (WAMMI) –http://www.wammi.com/index.htmlhttp://www.wammi.com/index.html Software Usability Measurement Inventory (SUMI) –http://sumi.ucc.ie/index.htmlhttp://sumi.ucc.ie/index.html Usefulness, Satisfaction, and Ease of Use Questionnaire (USE) –http://www.stcsig.org/usability/newsletter/0110_measuring_with_use. htmlhttp://www.stcsig.org/usability/newsletter/0110_measuring_with_use. html

38. 37 © 2015 Karis Usability Distribution of SUS Scores Measuring the User Experience: A companion website to the book by Tom Tullis and Bill Albert ( http://measuringuserexperience.com/) http://measuringuserexperience.com/ See also Sauro, J. (2011). SUStisfied?: Little-Known System Usability Scale Facts. User Experience, 10(3). Average SUS = 68 5,000 users, 500 studies The positively and negatively worded items hurt more than help SUS is as reliable as usability questionnaires you need to pay for, such as SUMI, QUIS, and WAMMI

39. 38 © 2015 Karis Usability UserTesting.com Video Example Walmart Sample Video –Link on home page, “videos of real people” –https://www.usertesting.com/videos/sE%2fdl0GKLiU%3d?token=pqGG W9JZCjQjelISf1nhKHRnvRwLkCoixoZS2RGHlJU=https://www.usertesting.com/videos/sE%2fdl0GKLiU%3d?token=pqGG W9JZCjQjelISf1nhKHRnvRwLkCoixoZS2RGHlJU –Tabs Annotations Clips: they created 4 clips Tasks (4) Answers (to post-test questions)

40. 39 © 2015 Karis Usability Creating & Testing Information Architecture Optimal Workshop http://www.optimalworkshop.com/ Card Sorting: how do people group features, options, or information? –What options or information should go under each menu item? And how many menu items should there be? Tree Testing –Given an item, where will users look for it? First Click Testing –Given a task, where on a page will users click first? Try the demos! –You can go through a simple card sort, tree testing, and first click test and see how the results are presented. Free Trials –Free card sorting: Survey 10 people with 30 cards using OptimalSort –Free tree testing: Survey 10 people with 3 tasks using Treejack –Free first click tests: Survey 10 people with 3 tasks using Chalkmark See also http://fivesecondtest.com http://fivesecondtest.com