IEEM552 - Human Computer Systems Week 2 - A usability study February 23, 1999 Dr. Vincent Duffy - IEEM

Today and next few weeks v Week 4 - Tuesday, 23rd February u in week 4 - select papers for oral presentations u finalize groups of 2 or 3, u present during wk.6,7,9,10, or 11 –first a practice exercise - homework due by noon Tuesday 23rd by (no attachment) –details in just a few minutes minute v Week 5 - Monday, March 1 –OSHC seminar 9am-6pm (no Tuesday class) 2

For today - week 4 v Postpone demonstration until week 6 v Administrative –Determine teams and select papers for oral presentations u handout (references & week due) –Discuss tripMonday, March 1 (week 5) u no class meeting Tuesday u handout (who is registered, who needs travel arrangments, reg. code) u what to look for at the conference? 3

Today continued v Guidelines on VDT (video display terminal) use in the workplace v Permission for the I.S. Lab – user access card v New content: v Economic benefit or Cost/benefits Analysis: Assessing Usability (Ch.1-p.11-17) –Mantei and Teori method & v For Discussion/lecture : A usability study –summary of Nielsen - thinking aloud method 4

Things to consider for presentation v See 10 ways to look at a research paper –also - 1. Objective of the research –2. Brief discussion of literature support –3. Development of hypotheses (if any) … or why the research was done in this way –4. Methods –5. Results –6. Discussion/conclusions u any practical application for the research? u Theoretical contribution? –7. Future work: any suggestions 5

 1. research ideas/question -what is the purpose?  2. what background support? - literature review  3. theoretical basis for analyzing question/hypothesis?  4. applicability-practical contribution?  5. theoretical contribution?  6. appropriate methodologies for carrying out study? (determining variables, data collection, method used to test hypotheses, validity of measures and reliability)  7. appropriate statistical analyses and assumptions?  8. presentation of results: what do they really mean?  9. conclusions drawn: are they reasonable?  10. future work/research directions: any possibilities? 10 ways to look at a research paper systematically for critical analysis 6

For oral presentation v Please determine groups of two v choose the paper of your interest for presentation v library catalog - images reserve 7

v v Due Week 4 – Today- Estimating the Number of Subjects Needed for a Thinking Aloud Test, J.Nielsen, International Journal of Human-Computer Studies, 41, , (no presentation) v v Week 5 - March 1 - A Study on Prevalence of Musculoskeletal Discomfort Among Visual Display Terminals (VDT) Users in Hong Kong, W.S. Tang, ERGON-AXIA, Proceedings of the First World Congress on Ergonomics for Global Quality and Productivity, Ed. Bishu, Karwowski and Goonetileke, p.6-30, (no presentation) v v Week 6 - March 9 - An Empirical Assessment of Task Complexity for Computerized Menu Systems, J.A. Jacko, International Journal of Cognitive Ergonomics, 1 (2), , v v Week 7 - March 16 - The Effect of Pictogram-Based Interface Design on Human- Computer Performance, L.A. Miller and K.M. Stanney, International Journal of Human-Computer Interaction, 9 (2), , v v Week 8 - Mid-term Exam v v Week 9 - March 30 - Using Natural Language Conventions in the User Interface Design of Automatic Speech Recognition Systems, D.J. Brems, M.D.Rabin, J.L.Waggett, Human Factors, , v v Week 10 - April 13 - Development and Evaluation of the Windows Computer Experience Questionnaire (WCEQ), L.A. Miller, K.M. Stanney, W.Wooten, International Journal of Human-Computer Interaction, 9 (3), , v v Week 11 - April 20 - Temporal Issues of Quality of Working Life and Stress in Human-Computer Interaction, P.Carayon, International Journal of Human-Computer Interaction, 9 (4), , v v Week 12 - April 27 - Persuasiveness of Expert Systems, J.J.Dijkstra, W.B.G.Liebrand, E.Timminga, Behaviour and Information Technology, 17 (3), , 1998.

Homework due before next class meeting v v 1. Please read the following before going to the Symposium. – –Week 5 - March 1 - A Study on Prevalence of Musculoskeletal Discomfort Among Visual Display Terminals (VDT) Users in Hong Kong, W.S. Tang, ERGON-AXIA, Proceedings of the First World Congress on Ergonomics for Global Quality and Productivity, Ed. Bishu, Karwowski and Goonetileke, p.6-30, (no presentation) v v 2. While at the Symposium. Please plan to discuss your knowledge with someone there and ask someone an intelligent question about it. v v 3. Please tell the name and affiliation of the person you spoke with and summarize your question and answer/discussion (up to 1 paragraph).. – –Please submit it by noon Tuesday of Week 6 (can be done in group of 2). v 4. Please read the following and be prepared to ask a question during/after the discussion/presentation) – –Week 6 - March 9 - An Empirical Assessment of Task Complexity for Computerized Menu Systems, J.A. Jacko, International Journal of Cognitive Ergonomics, 1 (2), ,

Recall: H.W. For week 4 v 2. Economic Benefits of User Interface design v read chapter 1-Eberts-User Interface Design v consider the Mantei and Teori method. v Q. Could it have been used by Microsoft (for Word for DOS) for determining the economic benefits of increased usability? v Yes/No and 1 sentence is sufficient. 10

Economic benefits of Usability analysis v assessing user interfaces v benefits? –faster task completion times, less errors, reduced costs for training (help required) u Bullinger, Burmester, Vossen, 1995 –learning times, user error, cost of maintenance u Eberts, 1994; Mantei and Teory, 1988; Karat 1992 v costs? –focus groups, building mock-ups, prototype, design changes, running the studies, making test environment, conducting the user survey 11

v total savings –learning time + user errors + maintenance costs v learning time –turnover, training time saved, wages v cost of errors per year –# of employees, searches/hr, hr/yr, P(error), time to fix error, wages v maintenance costs (design change savings) –# of changes (25?), hrs to change (8?), wages –late change 4x early cost (late cost minus early cost) Economic benefits of Usability analysis 12

Recall: H.W. For week 4 v 2. Economic Benefits of User Interface design v read chapter 1-Eberts-User Interface Design v consider the Mantei and Teori method. v Q. Could it have been used by Microsoft (for Word for DOS) for determining the economic benefits of increased usability? v Yes/No and 1 sentence is sufficient. 13

v Assumption of this method? –Mantei and Teory - mostly concerned with in-house software use v if company is selling software, then how different? –improvement= increased revenue –= cost * percent of increase * sale of old units v Why not for Microsoft case? –It is about whether or not the software will sell better than that already on the market. Economic benefits of Usability analysis 14

Recall H.W. - week 4 v 1. research paper for next week –Nielsen, J. (1994) Estimating the number of subjects needed for a thinking aloud test Int. J. of Human Computer Studies –using the 10 ways to look at a research paper –choose 3 subheadings in the ‘10 ways’ and write 3 sentences about each. –Choose 1 subheading and write 1 sentence about why you didn’t choose it. 15

Nielsen paper v Thinking aloud…how does it work –see video v Do you agree or disagree w/the following –Thinking aloud method was not widely used because of its complexity and high cost. –True/false? u Actually…usability studies where many subjects are needed to show quantitative results are of high cost. u Nielsen is pointing out that a large % of problems can be found with a small # of subjects. 16

Remember: homework due before next class meeting v 1. Please read the following before going to the Symposium. –Week 5 - March 1 - A Study on Prevalence of Musculoskeletal Discomfort Among Visual Display Terminals (VDT) Users in Hong Kong, W.S. Tang, ERGON-AXIA, Proceedings of the First World Congress on Ergonomics for Global Quality and Productivity, Ed. Bishu, Karwowski and Goonetileke, p.6-30, (no presentation) v 2. While at the Symposium. Please plan to discuss your knowledge with someone there and ask someone an intelligent question about it. v 3. Please tell the name and affiliation of the person you spoke with and summarize your question and answer/discussion (up to 1 paragraph).. –Please submit it by noon Tuesday of Week 6 (can be done in group of 2). v 4. Please read the following and be prepared to ask a question during/after the discussion/presentation) –Week 6 - March 9 - An Empirical Assessment of Task Complexity for Computerized Menu Systems, J.A. Jacko, International Journal of Cognitive Ergonomics, 1 (2), ,

for next class meeting - week 6 v in class demonstration - March 9 –postponed from this week –all groups will participate in collecting data –compare two interfaces –analyze the data, interpret the results –estimating the benefits - –we will compare library cat for dos/www v meet in the I.S. Lab –first half of class –3rd floor - near