1. IEEM552 - Human Computer Systems Week 2 - A usability study February 23, 1999 Dr. Vincent Duffy - IEEM 552 http://iesu5.ieem.ust.hk/dfaculty/duffy/ 552 email: vduffy@ust.hk 1

2. Today and next few weeks v Week 4 - Tuesday, 23rd February u in week 4 - select papers for oral presentations u finalize groups of 2 or 3, u present during wk.6,7,9,10, or 11 –first a practice exercise - homework due by noon Tuesday 23rd by email (no attachment) –details in just a few minutes minute v Week 5 - Monday, March 1 –OSHC seminar 9am-6pm (no Tuesday class) 2

3. For today - week 4 v Postpone demonstration until week 6 v Administrative –Determine teams and select papers for oral presentations u handout (references & week due) –Discuss tripMonday, March 1 (week 5) u no class meeting Tuesday u handout (who is registered, who needs travel arrangments, reg. code) u what to look for at the conference? 3

4. Today continued v Guidelines on VDT (video display terminal) use in the workplace v Permission for the I.S. Lab – user access card v New content: v Economic benefit or Cost/benefits Analysis: Assessing Usability (Ch.1-p.11-17) –Mantei and Teori method & v For Discussion/lecture : A usability study –summary of Nielsen - thinking aloud method 4

5. Things to consider for presentation v See 10 ways to look at a research paper –also - 1. Objective of the research –2. Brief discussion of literature support –3. Development of hypotheses (if any) … or why the research was done in this way –4. Methods –5. Results –6. Discussion/conclusions u any practical application for the research? u Theoretical contribution? –7. Future work: any suggestions 5

6.  1. research ideas/question -what is the purpose?  2. what background support? - literature review  3. theoretical basis for analyzing question/hypothesis?  4. applicability-practical contribution?  5. theoretical contribution?  6. appropriate methodologies for carrying out study? (determining variables, data collection, method used to test hypotheses, validity of measures and reliability)  7. appropriate statistical analyses and assumptions?  8. presentation of results: what do they really mean?  9. conclusions drawn: are they reasonable?  10. future work/research directions: any possibilities? 10 ways to look at a research paper systematically for critical analysis 6

7. For oral presentation v Please determine groups of two v choose the paper of your interest for presentation v library catalog - images reserve 7

8. v v Due Week 4 – Today- Estimating the Number of Subjects Needed for a Thinking Aloud Test, J.Nielsen, International Journal of Human-Computer Studies, 41, 385-397, 1994. (no presentation) v v Week 5 - March 1 - A Study on Prevalence of Musculoskeletal Discomfort Among Visual Display Terminals (VDT) Users in Hong Kong, W.S. Tang, ERGON-AXIA, Proceedings of the First World Congress on Ergonomics for Global Quality and Productivity, Ed. Bishu, Karwowski and Goonetileke, p.6-30, 1998. (no presentation) v v Week 6 - March 9 - An Empirical Assessment of Task Complexity for Computerized Menu Systems, J.A. Jacko, International Journal of Cognitive Ergonomics, 1 (2), 137-147, 1997. v v Week 7 - March 16 - The Effect of Pictogram-Based Interface Design on Human- Computer Performance, L.A. Miller and K.M. Stanney, International Journal of Human-Computer Interaction, 9 (2), 119-131, 1997. v v Week 8 - Mid-term Exam v v Week 9 - March 30 - Using Natural Language Conventions in the User Interface Design of Automatic Speech Recognition Systems, D.J. Brems, M.D.Rabin, J.L.Waggett, Human Factors, 265-282, 1995. v v Week 10 - April 13 - Development and Evaluation of the Windows Computer Experience Questionnaire (WCEQ), L.A. Miller, K.M. Stanney, W.Wooten, International Journal of Human-Computer Interaction, 9 (3), 201-212, 1997. v v Week 11 - April 20 - Temporal Issues of Quality of Working Life and Stress in Human-Computer Interaction, P.Carayon, International Journal of Human-Computer Interaction, 9 (4), 325-342, 1997. v v Week 12 - April 27 - Persuasiveness of Expert Systems, J.J.Dijkstra, W.B.G.Liebrand, E.Timminga, Behaviour and Information Technology, 17 (3), 155-163, 1998.

9. Homework due before next class meeting v v 1. Please read the following before going to the Symposium. – –Week 5 - March 1 - A Study on Prevalence of Musculoskeletal Discomfort Among Visual Display Terminals (VDT) Users in Hong Kong, W.S. Tang, ERGON-AXIA, Proceedings of the First World Congress on Ergonomics for Global Quality and Productivity, Ed. Bishu, Karwowski and Goonetileke, p.6-30, 1998. (no presentation) v v 2. While at the Symposium. Please plan to discuss your knowledge with someone there and ask someone an intelligent question about it. v v 3. Please tell the name and affiliation of the person you spoke with and summarize your question and answer/discussion (up to 1 paragraph).. – –Please submit it by noon Tuesday of Week 6 (can be done in group of 2). v 4. Please read the following and be prepared to ask a question during/after the discussion/presentation) – –Week 6 - March 9 - An Empirical Assessment of Task Complexity for Computerized Menu Systems, J.A. Jacko, International Journal of Cognitive Ergonomics, 1 (2), 137-147, 1997. 9

10. Recall: H.W. For week 4 v 2. Economic Benefits of User Interface design v read chapter 1-Eberts-User Interface Design v consider the Mantei and Teori method. v Q. Could it have been used by Microsoft (for Word for DOS) for determining the economic benefits of increased usability? v Yes/No and 1 sentence is sufficient. 10

11. Economic benefits of Usability analysis v assessing user interfaces v benefits? –faster task completion times, less errors, reduced costs for training (help required) u Bullinger, Burmester, Vossen, 1995 –learning times, user error, cost of maintenance u Eberts, 1994; Mantei and Teory, 1988; Karat 1992 v costs? –focus groups, building mock-ups, prototype, design changes, running the studies, making test environment, conducting the user survey 11

12. v total savings –learning time + user errors + maintenance costs v learning time –turnover, training time saved, wages v cost of errors per year –# of employees, searches/hr, hr/yr, P(error), time to fix error, wages v maintenance costs (design change savings) –# of changes (25?), hrs to change (8?), wages –late change 4x early cost (late cost minus early cost) Economic benefits of Usability analysis 12

13. Recall: H.W. For week 4 v 2. Economic Benefits of User Interface design v read chapter 1-Eberts-User Interface Design v consider the Mantei and Teori method. v Q. Could it have been used by Microsoft (for Word for DOS) for determining the economic benefits of increased usability? v Yes/No and 1 sentence is sufficient. 13

14. v Assumption of this method? –Mantei and Teory - mostly concerned with in-house software use v if company is selling software, then how different? –improvement= increased revenue –= cost * percent of increase * sale of old units v Why not for Microsoft case? –It is about whether or not the software will sell better than that already on the market. Economic benefits of Usability analysis 14

15. Recall H.W. - week 4 v 1. research paper for next week –Nielsen, J. (1994) Estimating the number of subjects needed for a thinking aloud test Int. J. of Human Computer Studies 385-397. –using the 10 ways to look at a research paper –choose 3 subheadings in the ‘10 ways’ and write 3 sentences about each. –Choose 1 subheading and write 1 sentence about why you didn’t choose it. 15

16. Nielsen paper v Thinking aloud…how does it work –see video v Do you agree or disagree w/the following –Thinking aloud method was not widely used because of its complexity and high cost. –True/false? u Actually…usability studies where many subjects are needed to show quantitative results are of high cost. u Nielsen is pointing out that a large % of problems can be found with a small # of subjects. 16

17. Remember: homework due before next class meeting v 1. Please read the following before going to the Symposium. –Week 5 - March 1 - A Study on Prevalence of Musculoskeletal Discomfort Among Visual Display Terminals (VDT) Users in Hong Kong, W.S. Tang, ERGON-AXIA, Proceedings of the First World Congress on Ergonomics for Global Quality and Productivity, Ed. Bishu, Karwowski and Goonetileke, p.6-30, 1998. (no presentation) v 2. While at the Symposium. Please plan to discuss your knowledge with someone there and ask someone an intelligent question about it. v 3. Please tell the name and affiliation of the person you spoke with and summarize your question and answer/discussion (up to 1 paragraph).. –Please submit it by noon Tuesday of Week 6 (can be done in group of 2). v 4. Please read the following and be prepared to ask a question during/after the discussion/presentation) –Week 6 - March 9 - An Empirical Assessment of Task Complexity for Computerized Menu Systems, J.A. Jacko, International Journal of Cognitive Ergonomics, 1 (2), 137-147, 1997. 17

18. for next class meeting - week 6 v in class demonstration - March 9 –postponed from this week –all groups will participate in collecting data –compare two interfaces –analyze the data, interpret the results –estimating the benefits - –we will compare library cat for dos/www v meet in the I.S. Lab –first half of class –3rd floor - near 3203 18