October 10, 2014 Health IT Implementation, Usability and Safety Workgroup David Bates, chair Larry Wolf, co-chair

Membership 1 NameOrganization David W. Bates (Chair)Brigham and Women's Hospital Larry Wolf (Co-Chair)Kindred Healthcare Joan Ash Oregon Health & Science University TBDVendor representative Janey Barnes User-View Inc. John Berneike St. Mark's Family Medicine Bernadette CapiliNew York University Michelle Dougherty American Health Information Management Association Paul Egerman Software Entrepreneur Terry Fairbanks Emergency Physician Tejal Gandhi National Patient Safety Foundation George Hernandez ICLOPS Robert Jarrin Qualcomm Incorporated Mike LardieriNorth Shore-LIJ Health System Bennett LauberThe Usability People LLC Alisa RayCCHIT Steven Stack American Medical Association 1

Ex Officio Members 2 NameOrganization Svetlana LowryEx Officio, NIST Megan SawchukEx Officio, Centers for Disease Control and Prevention Jeanie ScottEx Officio, Department of Veterans Affairs Jon WhiteEx Officio, AHRQ/HHS Ellen Makar ONC staff lead

Meeting Schedule MeetingsTask Monday, September 22, :00 PM-4:00 PM Eastern Time Review charge Work to date=- background / history Preliminary goals discussion of deliverable Friday, October 10, :00 PM-3:00 PM Eastern Time Presentation of usability research MedStar and NIST Friday, October 24, :00 PM-3:00 PM Eastern Time ECRI and TJC results of adverse event database analysis Usability Testing Implementation Science (field reports) Certification – Alicia Morton Friday, November 7, :00 PM-3:00 PM Eastern Time Tuesday, November 25, :00 PM-4:00 PM Eastern Time Possibly cancel Friday, December 5, :00 AM-12:00 PM Eastern Time Possibly cancel Friday, December 12, :00 PM-3:00 PM Eastern Time Post-implementation Usability & Safety, Risk Mgt & Shared Responsibility Safety Center Report Out Realignment of timeline/ goals for

Agenda CEHRT Regulation Briefing Usability Presentations – MedStar Health Raj Ratwani and Terry Fairbanks – NIST Lana Lowry Public Comment 4

ONC Certification Authority Stage 2 – 2014 Edition EHR Certification Criteria on “user-centered design” and “quality management systems.” – Increased transparency based on information available through certification. See ONC’s CHPL site. ONC Authorized Certifying Body (ACB) can conduct surveillance in live environments. – ACB’s are “health oversight agencies” under HIPAA – See ONC FAQ #45 5

We proposed a ‘‘safety-enhanced design’’ (SED) certification criterion for the Proposed Voluntary Edition that was unchanged as compared to the 2014 Edition certification criterion. We did, however, solicit public comment regarding whether we should modify the certification criterion. Specifically, we requested comment regarding whether: The scope of SED should be expanded to include additional certification criteria Formative usability tests should be explicitly required, or used as substitutes for summative testing There are explicit usability tests that should be required in addition to summative testing There should be a minimum number of test subjects explicitly required for usability testing Safety- enhanced Design 6 New: Safety-enhanced design. User centered design processes must be applied to each capability an EHR technology includes that is specified in the following certification criteria: § (a)(1), (2), (6) through (8), (16) and (18) through (20) and (b)(3), (4), and (9). Response: We will, however, consider all the thoughtful comments we received regarding expanding the scope and testing of the SED certification criterion in relation to future rulemaking activity concerning a SED certification criterion. Current: Safety-enhanced design. User-centered design processes must be applied to each capability an EHR technology includes that is specified in the following certification criteria: § (a)(1), (2), (6) through (8), and (16) and (b)(3) and (4).

UCD in CEHRT Regulation Safety-enhanced design. User centered design processes must be applied to each capability an EHR technology includes that is specified in the following certification criteria: § (a)(1), (2), (6) through (8), and (16) and (b)(3) and (4). § (a)(1) (CPOE); § (a)(2) (Drug/drug, drug-allergy interaction checks) § (a)(6) (Medication list); § (a)(7) (Medication allergy list) § (a)(8) (Clinical decision support) § (a)(16) (Electronic medication administration record) § (b)(3) (Electronic prescribing) § (b)(4) (Clinical information reconciliation). Fact Sheet: Table of 2014 certification criteria: Quick Guide: Guidance/Legislation/EHRIncentivePrograms/Downloads/CEHRT2014_FinalRule_QuickGuide.pdf 7

Human Factors Perspective on Advancing EHR Usability & Safety Rollin (Terry) Fairbanks, MD, MS Center Director National Center for Human Factors in Healthcare, MedStar Health Emergency Physician, MedStar Washington Hospital Center Associate Professor of Emergency Medicine, Georgetown University Raj Ratwani, PhD Scientific Director National Center for Human Factors in Healthcare, MedStar Health Assistant Professor of Emergency Medicine, Georgetown University

1: User Interface Design Displays and Controls Screen Design Clicks & Drags Colors & Navigation The Two Bins of Usability 2: Cognitive Task Support “Workflow Design” Smart Data Visualization Support Cognitive Work Functionality Photo credit to Bob Wears, MD, PhD

Anatomically oriented Is this the best way?

Bin 2 - Basic

Bin 2 - Advanced

nextgen

Focus Areas User centered design (UCD) and implementation (ONC/SHARPC project) – 11 in depth vendor visits Analysis of SED reports Our perspective on certification Analysis of health IT related patient safety event data

Vendor User Centered Design (UCD) Objective: – Understand vendor UCD processes and challenges – UCD: any formalized process for incorporating user needs throughout design, development and implementation Method: – Onsite meetings primarily with: Usability experts Business Analysts Product Managers

Vendor Demographics Vendor Demographic Summary VendorEst. RevenueEst. EmployeesEst. Usability Team Size Vendor 1$1 billion ppl Vendor 2$1 billion Vendor 2$1 billion+6000+NA Vendor 4$100 million Vendor 5$100 million+650NA Vendor 6$100 million Vendor 7$40 million Vendor 8$20 million Vendor 9$20 million150NA Vendor 10$10 million60NA Vendor 11$300,00010NA Range$300,00 - $1 billion

EHR Vendor UCD Processes Focused on customer requests Responding to user feedback is UCD No formalized method for incorporating and testing user needs throughout design and development No True UCD Understand UCD and its importance Striving to implement UCD processes UCD is not fully integrated yet Basic UCD Rigorous UCD processes in place Efficient testing methods Extensive infrastructure Well Developed UCD Challenges: General process Leadership Challenges: Resources Participant access Use case development Challenges: Detailed work flow analysis Safety data

Analysis of Safety Enhanced Design (SED) Reports Tremendous variability – As few as 3 participants (some with 20) Violates usability standards & creates double standard – Diverse range of participant expertise Some with no clinical expertise (eliminates bin 2) – Diverse experience levels – Variability in amount of training on the system Revisit guidelines to the authorized certification bodies (ACBs) Not all the SED reports are public

Perspectives on Certification Implementation processes: – Variability in implementation processes across vendors/providers – Few guidelines (SAFER guides are a start) – Customization: what is actually being certified? Most vendors expressed concern over the investments required to meet summative testing requirements

Our Perspective on UCD Certification Give vendors the option to either: – Attest to a UCD process and provide summative testing results OR – Attest to a UCD process and provide evidence of the UCD process being employed Several advantages: – Byproducts of the UCD process would serve to meet the cert requirement – Vendors can expend “usability resources” as desired based on need

Safety Monitoring and Analysis to Inform UCD Use machine learning (NLP) to analyze HIT related safety events – Example: Inpt dialysis nurse entered order in XXXX for Aranesp 100 mcg IV push q7d on incorrect pt. A pharmacist verified order but this order was never reviewed by floor nurse. Inpt dialysis nurse realized she entered order on incorrect patient.. moments after signing the electronic order and immediately removed the task on eMAR but did not discontinue order in MedConnect. The inpt dialysis nurse removed Aranesp dose from the Dialysis Pyxis (non a profiled device) for the correct patient and administered the correct dose. – Input as a “Medication” event in a database of 30,000+ events. Flagged as HIT related with NLP. Analyze these events in the context of UCD practices to provide insights on how to improve UCD – Which events would have been mitigated by formative testing? – By personas? etc

Discussion Raj Ratwani, PhD Rollin J (Terry) Fairbanks, MD MS

Next Meeting: Friday, October 24, :00 PM-3:00 PM Eastern Time 27