Presentation on theme: "Laura Lin Gosbee 10/30/03 Incorporating Human Factors Engineering into Risk Reduction Strategies Laura Lin Gosbee Human Factors Engineer."— Presentation transcript:
Laura Lin Gosbee 10/30/03 Incorporating Human Factors Engineering into Risk Reduction Strategies Laura Lin Gosbee Human Factors Engineer
Laura Lin Gosbee 10/30/03 My Background Human Factors University of Toronto – Institute of Biomedical Engineering/ Dept. of Mechanical & Industrial Engineering Northrop Grumman Corporation – Information Technology Sector Applied R&D Systems Design & Evaluation Areas of Application Health Care - Toronto Hospital, Toronto, Canada Nuclear Power - Westinghouse, Pittsburgh, PA Military - Wright-Patterson Air Force Base, Dayton, OH
Laura Lin Gosbee 10/30/03 Overview Human Factors Approach What is it? What can it be applied to? How is it useful to your hospital? Applying Human Factors Engineering Proactive and Reactive Strategies Methods and Tools Human Factors case studies
Laura Lin Gosbee 10/30/03 Human Factors - What is it? Human Factors (HF) is the study of human capabilities and limitations and the application of that knowledge to the design of systems
Laura Lin Gosbee 10/30/03 What can it be applied to? Home Medical Device Control Room CPOE Electronic Patient Records Defibrillator/Pacemaker MRI safety Crash Carts ASO Policies Automated Dispensing Device Spectrum of Complexity
Laura Lin Gosbee 10/30/03 What can it be applied to? Equipment Software Paper Forms Policies, Protocols, Procedures Workspace Layout Training & Education Analyze Design Design Test Test What’s the commonality? All involve human-system interaction
Laura Lin Gosbee 10/30/03 How is it useful to your hospital? Add usability data to procurement decisions Augment RCA and FMEA processes by: Relying less on guess work or brain-storming to identify root causes or failure modes Expanding actions from RCAs beyond: be more careful, more training, more policies, more rules to follow policies
Laura Lin Gosbee 10/30/03 QualityHealthcare.org “Fundamental insights about human error and the nature of accidents …shifts attention away from the fault of individuals to the larger contributing factors, often hidden and almost always out of their control.”
Laura Lin Gosbee 10/30/03 Example of Finding Root Causes Fighter planes before WWII Problem: planes crashing on night sorties, explanation elusive “Simulate” by putting blanket over cockpit No lighting to see gauges, lack of other visual cues Pilot error? No. HF design issue!
Laura Lin Gosbee 10/30/03 Key Messages 1.Not all HF problems are as seemingly obvious 2.Errors can be latent: “dormant” until night-time sorties 3.Example of human limitation: can’t see in the dark 4.Can’t always overcome with more training 5.Policies, e.g., permitting only daytime sorties, don’t tackle the problem 6.Most effective solution is in design: add cockpit lighting
Laura Lin Gosbee 10/30/03 Radar Scope to Detect Enemy Ships
Laura Lin Gosbee 10/30/03 100% 90% 80% 70% Time (hours) 1234 Performance Performance Graph (curve)
Laura Lin Gosbee 10/30/03 How can we move the curve upwards? 100% 90% 80% 70% Time (hours) 1234 Performance colour shape blinking auditory feedback tactile feedback
Laura Lin Gosbee 10/30/03 Another Example: New house, new faucets Which way do you turn them? A BC D
Laura Lin Gosbee 10/30/03 Expectations and Biases Operation does not match expectations No visual cues on how to operate Negative transfer of training Inappropriate mental model: Lever faucets B
Laura Lin Gosbee 10/30/03 In This Example Outcome not serious Finite number of ways to operate Danger of trial and error is not high Not always the case In health care: systems not necessarily as forgiving not limited to a few ways of using it trial and error not safe or allowed
Laura Lin Gosbee 10/30/03 Example: PCA pump Nurses program pump: drug concentration, dose, 4 hour limit, etc. Programming screen: 4 hour limit set? Yes or no? Does it mean: a) “have you already set it?” b) “do you want to set it?” At one hospital, for the first year, nurses interpreted as a) and answered “no”
Laura Lin Gosbee 10/30/03 Spend the next few minutes thinking of an example that you experienced (or saw happen to someone else) share it with the person next to you…
Laura Lin Gosbee 10/30/03 When is HF Applied? HF Methods can be applied to PROACTIVE and REACTIVE risk reduction strategies
Laura Lin Gosbee 10/30/03 Proactive Strategies When you want to keep “latent errors” from creeping into your hospital, incorporate HF when: Purchasing new products Designing and developing: in-house products or processes training & education programs workspace layout
Laura Lin Gosbee 10/30/03 Proactive Strategies Guide development of in-house products: policies/protocols, paper forms, software, devices, etc. Test/Validate before implementation Obtain usability data to compare products before purchasing Conduct a prospective risk assessment (FMEA) of systems
Laura Lin Gosbee 10/30/03 Targets for Proactive Strategies Equipment (syringes, automated dispensing devices) Connections for IV tubing, Oxygen Drug labels and storage Policies, procedures and protocols (new and old) Software (CPOE, patient records) Paper forms (order forms, checklists, warning signs, policy manuals)
Laura Lin Gosbee 10/30/03 Reactive Strategies When you have to solve existing problems, incorporate HF when: Investigating adverse events
Laura Lin Gosbee 10/30/03 Reactive Strategies Use HF to help identify vulnerabilities or contributing factors in an RCA Use HF methods when developing and validating RCA actions
Laura Lin Gosbee 10/30/03 Targets for Reactive Strategies Equipment (syringes, automated dispensing devices) Connections for IV tubing, Oxygen Drug labels and storage Policies, procedures and protocols (new and old) Software (CPOE, patient records) Paper forms (order forms, checklists, warning signs, policy manuals)
Laura Lin Gosbee 10/30/03 Sampling of HF Methods and Tools HF Analysis Task Analysis Field Observations Interviews Bench Tests Cognitive Walkthrough Heuristic Evaluation HF Design HF Design Principles Results from Analysis phase Standards & Guidelines HF Testing Human-in-the-loop User-centered testing Usability Testing Cognitive Walkthrough Heuristic Evaluation Simulator Testing (part-task, full-scale)
Laura Lin Gosbee 10/30/03 What can it be applied to? Equipment Software Paper Forms Policies, Protocols, Procedures Workspace Layout Training & Education Analyze Design Design Test Test
Laura Lin Gosbee 10/30/03 HF Design Old Task Structure Redesigned Task Structure
Laura Lin Gosbee 10/30/03 What can be tested? Devices Software Forms Processes Policies Procedures, etc. Testing works just as well for written material as it does for devices! HF Testing Products from policy/protocols: Instruction material Cognitive aids (cheat sheets) User manuals Memos Alerts or Warning Signs
Laura Lin Gosbee 10/30/03 Example - Protocol to be Tested: HF Testing Protocol for Preparing Medication Step 1 - Step 2 - Step 3 - Step 4 – Etc. Useful? Usable? Learnable? Accepted? Readability Content Layout Usability of associated material: drug labels, vials, tubing, connections, IV pumps, cartridges storage & retrieval
Laura Lin Gosbee 10/30/03 HF Methods & Tools A few of many… 1.Cognitive Walkthrough 2.Heuristic Evaluation 3.Usability Testing
Laura Lin Gosbee 10/30/03 1. Cognitive Walkthrough Preparation: Recruit end “users” Develop “scenarios” & “tasks” Methodology Participants think aloud as they “walk” you through using a system/protocol Use a variety of scenarios Ask them to verbalize any decision or mental step Record difficulties
Laura Lin Gosbee 10/30/03 2. Heuristic Evaluation Preparation HF design principles Design guidelines/standards (AAMI/ANSI) Guidelines for medical applications, e.g., Do it by Design, Write it Right (FDA) Methodology Essentially a usability audit Evaluate against guidelines & design principles Evaluate both form (e.g., visual design) and content
Laura Lin Gosbee 10/30/03 1)Overall Human Factors 2)Feedback and Visibility of System Status 3)Consistent Model or Metaphor 4)Functionality of Controls 5)Displayed Messages 6)Recognition and Recovery from Errors 7)Minimizing User Memory Load 8)Readable and understandable labels and warnings Human Factors Engineering Guidelines
Laura Lin Gosbee 10/30/03 Triggering Questions * Feedback and Visibility of System Status When you try to do things, how do you know you are successful? · Do you know what the device is doing at any given moment? If you are distracted, can you tell immediately where you've left off? If you handed the device to someone, does it take them long to figure out where you've left off and what the device is doing? * Human Factors Tool developed by: John Gosbee & Laura Lin Gosbee
Laura Lin Gosbee 10/30/03 Triggering Questions * Functionality of Controls Is it obvious what each button or switch will do? Do they work the same in different circumstances? Do some buttons/switches look too similar to others? Are they grouped/located in a logical manner? Are some controls more critical than others? If so, how are they differentiated from other controls? * Human Factors Tool developed by: John Gosbee & Laura Lin Gosbee
Laura Lin Gosbee 10/30/03 Displayed Messages Is the message display big enough? Can you understand the messages? Is the language simple and natural? Is the information useful? Do you need more information? Is it displayed long enough for it to be useful? Triggering Questions * * Human Factors Tool developed by: John Gosbee & Laura Lin Gosbee
Laura Lin Gosbee 10/30/03 3. Usability Testing Preparation Recruit end users Develop “scenarios” and “tasks” Select test site Performance measures (e.g., errors, time to recover from errors, mental workload) Test plan (experimental design & protocol) Methodology Generally, participants are asked to carry out a ‘task’ while performance data is collected unobtrusively
Laura Lin Gosbee 10/30/03 Performance Benchmarks Examples: 90% people using it “cold” can do it without errors 80% people interpreting the policy can understand the rationale or implications Performance Goals
Laura Lin Gosbee 10/30/03 Example: Epinephrine Auto-Injector Epinephrine used for Anaphylaxis Auto-injector administers epinephrine Device is used in emergency situations
Laura Lin Gosbee 10/30/03 A Quick and Dirty Usability Test Design Goal (performance goal): lay person can use it in an emergency with no prior training Volunteer with no previous experience with auto-injectors Using a “trainer” – NO NEEDLES!
Laura Lin Gosbee 10/30/03 Scenario I just ate some cookies with traces of peanuts I am going into anaphylactic shock… I’ve stopped breathing You are alone with me and found my auto- injector Need to administer it quickly before the unthinkable…
Laura Lin Gosbee 10/30/03 Lessons Design helps/hinders human performance Environment of use specifies performance requirements (benchmarks) lay person, little or no training, timeliness * Gosbee LL. Nuts! I can’t figure out how to use my life-saving auto-injector: Human factors issues associated with an epinephrine auto-injector. Joint Commission Journal on Quality and Safety (in press).
Laura Lin Gosbee 10/30/03 Case Study: HF analysis of “ confusion ” Hospital in Salt Lake City had many issues with code teams One issue they observed many times was confusion and delay with medication drawer retrieval They performed usability (HFE) testing with 9-11 end-users Retrieving 10 medications Five versions of the drawer * McLaughlin RC. Redesigning the crash cart: usability testing improves one facility's medication drawers. Am J Nurs. 2003;103(4):64A,64D,64G- 64H.
Laura Lin Gosbee 10/30/03 Baseline Drawer ( “ Laundry hamper ” ) Range = 2:43-3:58 min, Avg=3:07 min
Laura Lin Gosbee 10/30/03 Code Cart Drawer Fifth Version Range = :55-1:25 min, Avg=1:08 Note the lack of labels for each spot
Laura Lin Gosbee 10/30/03 Case Study: PCA pump University of Toronto and Toronto General Hospital 1993: nurse complaints about “ new device ” : PCA pumps HF study: HF analysis redesign user testing
Laura Lin Gosbee 10/30/03 PCA: Interface Redesign Existing DesignNew Design
Laura Lin Gosbee 10/30/03 PCA: Programming Sequence Redesign Existing DesignNew Design Decision Message-guided Action Action Legend
Laura Lin Gosbee 10/30/03 User population Tested with 2 user populations: Novice users Nursing students n=12 Expert users Recovery Room Nurses n=12
Laura Lin Gosbee 10/30/03 Results Dependent Variable Nursing Students Recovery Room Nurses Errors50% fewer *55% fewer * Task Completion Time 15% faster *18% faster * Mental Workload 53% lower14% lower User Preference 100% *90% * Advantages seen with new design: * Denotes statistically significant
Laura Lin Gosbee 10/30/03 Implications Redesigning this PCA pump reduced time and errors $$ Redesigning the device is in the hands of the device maker, but… Provide your input to device makers (proactive) Ask for HF/usability test data from them (proactive) Consider HF in procurement decisions (proactive) Provide HF training for front-line personnel (proactive) Consider HF in RCAs (reactive)
Laura Lin Gosbee 10/30/03 What else can you do? Could other interventions reduce errors, time etc.? Maybe. BUT limited effectiveness - just compensating for a poorly designed device What kind of interventions?
Laura Lin Gosbee 10/30/03 Other Interventions Other less effective interventions within your control Provide cognitive aids e.g., Redesign pca order form: use HFE analysis to design and validate Workplace redesign e.g., program pumps in an area with fewer distractions; may impact other processes (e.g., cause disruptions in workflow); use HF to design & validate Training use HF analysis to develop training materials; understand that no amount of training can completely overcome a poorly designed device; don’t schedule training 6 months before devices arrive without follow- up training Policies e.g., double check by 2 nd person; less reliable, adds time, effort
Laura Lin Gosbee 10/30/03 Example of a Cognitive Aid PCA Order Form Concentration Units Concentration Bolus Dose Mode PCA Dose Continuous Rate Lockout 4 Hr Limit Test to validate Design depends on configuration of the specific device
Laura Lin Gosbee 10/30/03 Leveraging Factors Device Design User Training Work Environment Policies PCA Order Form
Laura Lin Gosbee 10/30/03 Policies to Fix Design Issues: Unintended Consequences Does it solve problems without introducing other unintended problems? Example: County Jail Policies
Laura Lin Gosbee 10/30/03 Policy #1: All officers shall remain with their prisoner while booking (paperwork) is completed
Laura Lin Gosbee 10/30/03 Forcing Function to reinforce Policy # 1 : Desk was removed, forcing officers to use small counter space near prisoner Policy #1: All officers shall remain with their prisoner while booking (paperwork) is completed
Laura Lin Gosbee 10/30/03 Unintended Result: Prisoner belongings get obscured by clutter. Policy # 2 gets violated.
Laura Lin Gosbee 10/30/03 Policy # 2: All prisoner belongings shall be admitted into evidence or the property room before prisoner is booked. Unintended Result: Prisoner belongings get obscured by clutter. Policy # 2 gets violated.
Laura Lin Gosbee 10/30/03 “Failure to follow these rules will result in more rules” Policy # 3: “Failure to follow these rules will result in more rules”
Laura Lin Gosbee 10/30/03 Redesign countertop space to accommodate process Redesign the software so that clerk can register belongings at any time Redesign forms: make it obvious that they must enter prisoner belongings before anything else Think about HF design interventions
Laura Lin Gosbee 10/30/03 Last Example: Auto-Injector revisited Epinephrine auto-injector for anaphylaxis Patients are advised: Carry it at all times May want to carry 2 in case: one fails or need another dose
Laura Lin Gosbee 10/30/03 Case Study 3 yr old with history of severe nut allergies History of contact-induced reactions Compliance is a dismal 40% !!
Laura Lin Gosbee 10/30/03 Solutions? More education? Create policy? Reminders? Reprimand? HF design intervention?
Laura Lin Gosbee 10/30/03 HF Design Problem and Intervention Design Problem: Bulky, won’t fit in pockets Can’t withstand temperature extremes Can’t be exposed to sunlight HF Intervention: Protective carrying case that clips to belt Hang it by the door Attach it to keys
Laura Lin Gosbee 10/30/03 Effectiveness of Solution Compliance went from 40% to 98% 2 percent non-compliance attributed to factors known as: “Terrible-Twos” and “Fearsome Fours” Sleep deprivation Time pressure Interruptions Occupational Stress
Laura Lin Gosbee 10/30/03 References Gosbee JW, Lin L. The role of human factors engineering in medical device and medical system errors. In C. Vincent (ed.) Clinical Risk Management: Enhancing Patient Safety. London: BMJ Press. 2001. Lin L, Isla R, Doniz K, Harkness H, Vicente K, Doyle DJ. Applying human factors engineering to the design of medical equipment: Patient-controlled analgesia. J. Clin. Monit. 1998; 14: 253-263 Lin L, Vicente K, Doyle DJ. Patient Safety, potential adverse drug events, and medical device design: A human factors engineering approach. J. Biomed. Informatics. 2002. Schneider PJ, Gosbee JW, Lathan C, Shapiro RG, Marx D. The application of human factors engineering to improve medication use safety. American Society of Health-System Pharmacy Journal. 2002.
Laura Lin Gosbee 10/30/03 References Rubin J. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. New York: John Wiley & Sons. 1994 Roth, E. M. & O'Hara, J. Exploring the impact of advanced alarms, displays, and computerized procedures on teams. In Proc. of the Human Factors and Ergonomics Society Annual Meeting Santa Monica, CA: Human Factors and Ergonomics Society. P.158-162. 1999. Guerlain S, LeBeau K, et.al. Effect of a standardized data collection form on the examination and diagnosis of patients with abdominal pain. In Proc. of Human Factors and Ergonomics Society Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society. 2001. Wieringa DR, Farkas DK. Procedure writing across domains: Nuclear power plant procedures and computer documentation. Proceedings of the Association for Computing Machinery. New York: ACM Press. P. 49-58. 1991.
Laura Lin Gosbee 10/30/03 Questions? Comments? Laura Lin Gosbee Human Factors Engineer LLGosbee@covad.net