1. Usability and Human Factors Unit 4c Human Factors and Healthcare

2. Effects of Heavy Nursing Workload MechanismsDescriptionExamples TimeInsufficient time to perform tasks safely, apply safe practices, or monitor patients, and may reduce their communication with physicians. Little time to double- check medications MotivationDissatisfied with job, thus affecting their motivation for high-quality performance. Frustration and negative attitude toward job Stress and burnoutStress and burnout, which can have a negative impact on their performance. Reduced physical and cognitive resources to perform adequately Errors in decision making (attention) Contribute to errors, such as slips and lapses or mistakes. Forgetting to administer medications Systemic/organizati onal impact Could affect the safety of care provided by another nurse. A charge nurse may not be available to help other nurses with their patients when needed. Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 2 Carayon, Ayse P. Gurses Mechanism: Time –Description: Insufficient time to perform tasks safely, apply safe practices, or monitor patients, and may reduce their communication with physicians. –Examples: Little time to double-check medications Mechanism: Motivation –Description: Dissatisfied with job, thus affecting their motivation for high-quality performance. –Examples: Frustration and negative attitude toward job Mechanism: Stress and burnout –Description: Stress and burnout, which can have a negative impact on their performance. –Examples: Reduced physical and cognitive resources to perform adequately Mechanism: Errors in decision making (attention) –Description: Contribute to errors, such as slips and lapses or mistakes. –Examples: Forgetting to administer medications Mechanism: Systemic/organizational impact –Description: Could affect the safety of care provided by another nurse. –Examples: A charge nurse may not be available to help other nurses with their patients when needed.

3. Mental Workload Multidimensional construct –Task demands –Expectations regarding the quality of performance –Person effort Can vary across persons and situations Difficult to measure Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 3

4. NASA Task Load Index A multi-dimensional rating procedure Provides overall workload score based on a weighted average of ratings on 6 subscales: –Mental demands –Physical demands –Temporal demands –Own performance –Effort –Frustration Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 4

5. NASA TLX Index: Rating Scale Definitions MENTAL DEMAND: Low/High How much mental & perceptual activity was required (e.g., thinking, deciding, calculating, remembering, looking, searching, etc.)? Was the task easy or demanding, simple or complex, exacting or forgiving? PHYSICAL DEMAND: Low/High How much physical activity was required (e.g., pushing, pulling, turning, controlling, activating, etc.)? Was the task easy or demanding, slow or brisk, slack or strenuous, restful or laborious? TEMPORAL DEMAND: Low/High How much time pressure did you feel due to the rate or pace at which the tasks or task elements occurred? Was the pace slow and leisurely or rapid and frantic? Component 15/Unit 4c 5 Health IT Workforce Curriculum Version 1.0/Fall 2010 MENTAL DEMAND: –Low/High –How much mental & perceptual activity was required (e.g., thinking, deciding, calculating, remembering, looking, searching, etc.)? –Was the task easy or demanding, simple or complex, exacting or forgiving? PHYSICAL DEMAND: –Low/High –How much physical activity was required (e.g., pushing, pulling, turning, controlling, activating, etc.)? –Was the task easy or demanding, slow or brisk, slack or strenuous, restful or laborious? TEMPORAL DEMAND: –Low/High –How much time pressure did you feel due to the rate or pace at which the tasks or task elements occurred? –Was the pace slow and leisurely or rapid and frantic?

6. NASA TLX Index: Rating Scale Definitions (cont.) EFFORT: Low/High How hard did you have to work (mentally and physically) to accomplish your level of performance? PERFORMANCE: Good/Poor How successful do you think you were in accomplishing the goals of the task set by the experimenter (or yourself)? How satisfied were you with your performance in accomplishing these goals? FRUSTRATION LEVEL: Low/High How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, relaxed and complacent did you feel during the task? Component 15/Unit 4c 6 Health IT Workforce Curriculum Version 1.0/Fall 2010 EFFORT: –Low/High –How hard did you have to work (mentally and physically) to accomplish your level of performance? PERFORMANCE: –Good/Poor –How successful do you think you were in accomplishing the goals of the task set by the experimenter (or yourself)? –How satisfied were you with your performance in accomplishing these goals? FRUSTRATION LEVEL: –Low/High –How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, relaxed and complacent did you feel during the task?

7. Medical Devices Healthcare products, excluding drugs, which are used for human beings for the purpose of prevention, diagnosis, monitoring, treatment or alleviation of an illness Center for Devices and Radiological Health, a division of the Food and Drug Administration Common source of medical error –Operator error (incorrectly entering a drug cartridge concentration) –Patient error –Mechanical problems 60% deaths & serious injuries attributed to operator error Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 7

8. Patient Controlled Analgesics Method of pain relief which uses disposable or electronic infusion devices and allows patients to self-administer analgesic drugs as required Patients determine when and how much medication they receive Abbott Lifecare PCA Plus II is the Markey leader –Used in more than 22 million patients Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 8

9. Lin, Isla, Doniz et al (1998) Purpose: redesign a PCA user interface according to HF techniques and principles –Improve efficiency and safety Cognitive task analysis was used to evaluate and redesign the system Experiment involving 12 student nurses programming the device Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 9

10. CTA Findings Structure of many subtasks in programming sequence is unnecessarily complex No global view of decision process Lack of external representation of dialog structure –No overview of how many parameters –Order sequence –Steps that have been completed and those that remain –Misleading visual grouping of controls Limited amount of feedback on small screen Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 10

11. Redesign of Interface System structure simplified by minimizing the number of message display screens Choices presented in parallel rather than serially Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 11

12. Experimental Evaluation Programming the new interface was 15% faster The average workload rating for the old interface was twice as high New interface led to 10 errors as compared to 20 for the old one Medical equipment can be made safer and more efficient by adopting human factors design principles Component 15/Unit 4c 12 Health IT Workforce Curriculum Version 1.0/Fall 2010 Programming the new interface was 15% faster The average workload rating for the old interface was twice as high New interface led to 10 errors as compared to 20 for the old one Medical equipment can be made safer and more efficient by adopting human factors design principles

13. Mobile Health Devices Handheld devices used in healthcare context by clinician, patient or health consumer –Palm PDAs, Pocket PC, Smart phones (Treo 650) and Blackberries –ePocrates, LexiDrugs, Up-to-Date, PalmCIS –Mobile Patient Monitoring, Glucowatch, Electronic Health Diaries, & advanced function glucose meters Improve information access Enhance workflow Communication Promote evidence- based decision making & empower patients Lack of a stable interaction paradigm, especially on patient side Component 15/Unit 4c 13 Health IT Workforce Curriculum Version 1.0/Fall 2010

14. UltraSmart Meter Hierarchical menu Nonlinear/tangled Max depth of 5 Breadth 2-8 (items on a single menu) Input: button press Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 14

15. Hierarchical Menu Systems Flow of control between human and device Interactive style provides visible options to user Structure of system can be fairly transparent –Constrained path of options May require minimal training Learning by exploring menu structure Breadth-depth tradeoffs Component 15/Unit 4c 15 Health IT Workforce Curriculum Version 1.0/Fall 2010

16. Objectives of Study Usability and learnability of LifeScan UltraSmart –Identify aspects of device facilitate or impede productive use –Determine whether older adults could develop basic competency (learnability) Autonomously perform or learn to perform the range of tasks through experience and/or reading the instructional materials –Characterize prerequisite competencies, knowledge and skills Robust Mental Model of System Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 16

17. Mental Models Anticipate events Reason about outcomes Project backwards from event to explain Try to reconstruct episodes from memory Mental representation of how things work Generate possible future states of system For example, “if I click OK on this button, what will happen next”? Running of a model corresponds to a process of mental simulation Temporal and spatial reasoning Component 15/Unit 4c 17 Health IT Workforce Curriculum Version 1.0/Fall 2010 Mental representation of how things work –Anticipate events –Reason about outcomes –Project backwards from event to explain –Try to reconstruct episodes from memory Running of a model corresponds to a process of mental simulation –Generate possible future states of system –For example, “if I click OK on this button, what will happen next”? Temporal and spatial reasoning

18. Mental Model of Cell Phone Shallower notion of menu’s depth Lack of understanding of hierarchical representation associated with poor performance Older adults had inferior mental models of phone menu than younger adults (Ziefle & Bay, 2004) Repeatedly open the same node Follow suboptimal paths Increase time to locate information Lack of a mental model can lead to disorientation in menu selection tasks Component 15/Unit 4c 18 Health IT Workforce Curriculum Version 1.0/Fall 2010 Older adults had inferior mental models of phone menu than younger adults (Ziefle & Bay, 2004) –Shallower notion of menu’s depth –Lack of understanding of hierarchical representation associated with poor performance Lack of a mental model can lead to disorientation in menu selection tasks –Repeatedly open the same node –Follow suboptimal paths –Increase time to locate information

19. Glucose Meter Usability Evaluation Cognitive task analysis Performed by 3 coders Usability Testing with 5 older adults Two sessions video and audio taped Adults were given a week to learn to use the glucose meter Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 19 Cognitive task analysis –Performed by 3 coders Usability Testing with 5 older adults –Two sessions video and audio taped –Adults were given a week to learn to use the glucose meter

20. Summary of Usability Testing All subjects experienced significant problems in the first session Data entry tasks Demonstrated degree of improvement in second session Highly variable Subjects exhibited trial and error Perseverated through fruitless searches Wrong place Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 20 All subjects experienced significant problems in the first session –Data entry tasks Demonstrated degree of improvement in second session –Highly variable Subjects exhibited trial and error –Perseverated through fruitless searches –Wrong place

21. Observed Problems Limited visibility of system stateInadequate prompts Not informative guiding subjects to next step Errors result from following the wrong path Error correction—returning to prior state is nontrivial Confusing task-action mappings Component 15/Unit 4c 21 Health IT Workforce Curriculum Version 1.0/Fall 2010 Limited visibility of system state Inadequate prompts –Not informative guiding subjects to next step Errors result from following the wrong path –Error correction—returning to prior state is nontrivial Confusing task-action mappings

22. Conclusions UltraSmart™ offers wide range of resources to support self-management in patient with diabetes Steep learning curve, especially for older adults Most would experience difficulty in developing a sufficiently robust mental model to negotiate menu structure –Taxing on Memory Difficult design problem Empirical testing can yield more effective design solutions Component 15/Unit 4c Health IT Workforce Curriculum Version 1.0/Fall 2010 22