Presentation on theme: "Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets Yiye Zhang Rema Padman, PhD James E. Levin *, MD, PhD The H. John Heinz III College."— Presentation transcript:
Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets Yiye Zhang Rema Padman, PhD James E. Levin *, MD, PhD The H. John Heinz III College Carnegie Mellon University, Pittsburgh, PA, USA firstname.lastname@example.org@andrew.cmu.edu; email@example.com@cmu.edu MedInfo2013, Copenhagen, Denmark * Dr. James E. Levin passed away on February 11, 2013. We are greatly indebted to his vision, contributions and support that made this study possible.
Introduction Significant healthcare delivery challenges in the U.S. and worldwide – Cost, quality, safety, efficiency, satisfaction – 1999 landmark Institute of Medicine report indicated that 44,000 to 98,000 Americans die each year from medical errors 1 – Medication errors are a major component of these errors 2 Potential of healthcare information technology (HIT) – Traditional paper prescription prone to errors due to poor legibility and miscommunication during patient transfers – Computerized provider order entry (CPOE), a core feature of the electronic health record (EHR) system, has been recommended to mitigate errors in inpatient orders 1.Institute of Medicine. (1999). To Err is Human: Building a Safer Health System. Retrieved March 28, 2004, from http://www.iom.edu/http://www.iom.edu/ 2.Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients. JAMA 2001;285(16):2114–20.
Computerized Provider Order Entry (CPOE) CPOE systems are software applications designed to enhance patient safety by allowing clinicians to enter inpatient orders electronically; Used by one third of US hospitals 1 CPOE systems have been shown to improve patient care through better order legibility, reduced rule violations, improved clinician compliance with best practices, and advanced clinical decision support features 2 Within CPOE, order sets allows clinicians to place multiple, relevant orders for each patient with fewer mouse clicks, thus the creation of order sets is an important prerequisite to successful CPOE implementation and use 1.HIMSS Analytics: Healthcare IT Data, Research, and Analysis. http://www.himssanalytics.org/hc_providers/emr_adoption.asp.http://www.himssanalytics.org/hc_providers/emr_adoption.asp 2.Potts AL, Barr FE, Gregory DF, Wright L, Patel NR. Computerized physician order entry and medication errors in a pediatric critical care unit. Pediatrics. 2004 Jan;113(1 Pt 1):59-63.
Collection of individual orders commonly entered as an aggregate for a specific clinical purpose or procedure Typically developed by clinical experts in a generic format Support clinicians in high risk situations by serving as expert- recommended guidelines, reducing prescribing time by eliminating unnecessary duplication of work, and increasing clinician compliance with the current best practices 1 Order Sets 1. Payne TH, Hoey PJ, Nichol P, Lovis C. Preparation and use of pre-constructed orders, order sets, and order menus in a computerized provider order entry system. J Am Med Inform Assoc. 2003 Jul-Aug;10(4):322-9.
Challenges with Order Set Usage Large Variability in Order Set Usage 1 Difficult to maintain order set content and combinations up-to-date with current best practices Lack of involvement in order set development by physicians who are familiar with both the guidelines as well as the actual practice Providers switch to a la carte orders instead of ordering from order set, potentially resulting in unsafe and inefficient ordering process Poorly designed order sets contribute negatively to treatment quality by exposing users to excessive mouse clicks (physical cost) and cognitive workload (cognitive cost) 1. Zhang Y, Levin JE, Padman R. Data-driven Order Set Generation and Evaluation in the Pediatric Environment. AMIA Symposium Proc. 2012.
Physical and Cognitive Costs poor usability--such as poorly designed screens, hard-to-navigate files, conflicting warning messages, and need for excessive keystrokes or mouse clicks--adversely affects clinical efficiency and data quality - a recent report from Agency for Healthcare Research and Quality (AHRQ) 1 There is a need to design features of CPOE according to human factor best practices. 2,3 1: Schumacher RM, Lowry SZ. NIST Guide to the Process Approach for Improving the Usability of Electronic Health Records. 2010. 2. Wright P, Lickorish A, Milroy R. Remembering While Mousing: The Cognitive Costs of Mouse Clicks. SIGCHI Bulletin. 1994. 3. Horsky J, Kaufman DR, Oppenheim ML, Patel VL. A framework for analyzing the cognitive complexity of computer-assisted clinical ordering. J Biomed Inform 2003;36(1–2):4–22. Number of order sets Physical/ cognitive cost One order set All a la carte Optimal number of order sets
Research Question Can the development of order sets be automated using historical ordering data to learn new order sets that are evidence-based, up-to-date with current best practices, and incur least physical/cognitive costs?
Study Setting Childrens Hospital of Pittsburgh (CHP) of UPMC, a HIMSS level 7 pediatric facility Since October 2002, all inpatient orders at CHP have been entered directly into the CHP eRecord (Cerner Millenium) Over 12,000 pediatric patients admitted each year Over 10 million order actions in total On average, a patient at CHP is hospitalized for 5.5 days, and during that time, 36 unique individuals create 871 order actions ~ 2000 departmental, in-house order sets
Sample Appendectomy Orders PatientIDOrder Name Order time since admission (hours) Order Set NameDefaults 4092NPO-1.08 Admission Orders Appendicitis, Complicated ON 4092Up Ad Lib-1.08 Admission Orders Appendicitis, Complicated ON 4092Vital Signs4.85 Post Anesthesia Care Orders - Pediatric ON 4092fentanyl4.85 Post Anesthesia Care Orders - Pediatric OFF 4092BUpivacaine4.87N/A A la Carte being utilized Order set being utilized
Distribution of Orders: Appendectomy Minor Solid blue: a la carte, dotted yellow: order set
Optimization and Clustering Models Minimize Cognitive Click Cost (CCC) Subject to 1) Default option choice constraints 2) Cluster formation constraints 3) Time interval constraints Approach: Order set development from order items
Eliciting Cognitive Costs CCC with expert estimate (CCCE): Expert input CCC based on survey result (CCCS): Survey of 15 subjects including physicians and nurses Each survey contains 6 questions with sub-questions, asking subjects to estimate the time it takes them to perform tasks while placing orders with large, mid-size, and small order sets
Approach: Order Set Development Determine optimal time interval and number of order sets within the time interval that minimize MCC/CCC Cluster orders using bisecting K-means clustering within each time interval Map new order set assignment back to historical treatment data to evaluate goodness of clustering using MCC/CCC and coverage rate
Patient Time of order placement OrderOrder setDefault settingCCC Patient 11.4AO1ON? Patient 11.4BO2OFF? ….…………… Patient 110.0NO3ON? Ex. Fixed patient, time, and order ON if more than 80% patients use; OFF otherwise SelectDe-select Order Set/ A la Carte 1.2-- Default ON0.21.5 Default OFF0.50.1 Order Set 1 Order Set 2 Such that CCC can be lowered ! Order A Order B Order C Order E Order D Time interval 1, 2,…., n
Results: Significant Reduction in CCC and Increase in Coverage Rate CCCE per patient (actual mouse clicks) CCCS per patient (actual mouse clicks) CurrentNew% changeCurrentNew% change Appendectomy Minor 145.7 111.1 (77) 23.4% ** 229.6 116.1 (83) 49.4% *** Appendectomy Moderate 208.4 143.3 (110) 31.2% *** 289.7 162.7 (103) 43.8% *** ***: p-value less than 0.01, **: p-value less than 0.05, *: p-value less than 0.1
Closer Look: Appendectomy Minor - CCCE Time IntervalTraining SetTest Set T5: 0 to 2174 patients, 153 items23 patients, 72 items Number of orders Number of order sets Average Coverage Rate per OS CCC per Patient % Reduction in CCC per Patient Average Coverage Rate per OS CCC per Patient % Reduction in CCC per Patient Current68120.3410.2 25.0% 0.2919.1 31.9 % New66200.757.60.4713.0 12 order sets used per patient on average in training set 6 order sets used per patient on average in test set
Sample Case: Under Current Order Set ItemOrder Set (size)Default Admit toAdmission Orders General Pediatric Medical Order Set (63)ON HeightAdmission Orders General Pediatric Medical Order SetON WeightAdmission Orders General Pediatric Medical Order SetON Notify MD For Oxygen SaturationsAdmission Orders General Pediatric Medical Order SetOFF Notify MD For TPRAdmission Orders General Pediatric Medical Order SetOFF Regular (4 yrs & >) DietAdmission Orders General Pediatric Medical Order SetOFF Up Ad LibAdmission Orders General Pediatric Medical Order SetOFF Vital SignsAdmission Orders General Pediatric Medical Order SetOFF Subsequent Oxygen TherapyOxygen Therapy (2)ON Initial Oxygen TherapyOxygen TherapyOFF Subsequent Pulse Oximetry Continuous Pulse Oximetry Continuous (2)ON Initial Pulse Oximetry ContinuousPulse Oximetry ContinuousOFF CCCE = 20.3, CCCS = 76, number of actual mouse clicks (MC) = 15
Sample Case: Under New Order Sets ItemMCC (size) CCC E (size) CCCS (size)Default Admit toC1 (3)C1 (2)C1 (5)ON HeightC2 (3) C2 (4)OFF WeightC2 OFF Notify MD For Oxygen Saturations a la carteC3 (3)C3 (6)ON Notify MD For TPRa la carteC2 OFF Regular (4 yrs & >) Dieta la carteC3 OFF Up Ad Liba la carteC4 (2)C3OFF Vital SignsC1 ON Subsequent Oxygen TherapyC3 (3)C5 (2)C1OFF Initial Oxygen TherapyC3C5C1OFF Subsequent Pulse Oximetry Continuous C4 (3)C6 (2)C4 (3)ON Initial Pulse Oximetry Continuous C4C6C4ON CCCE = 12.9 (36.4% drop ), CCCS = 23.1 (69.6% drop), MC = 13 (15.4% drop)
Order Set development based on data-driven approaches is promising Can be generalized for not only CHP order sets but also for order sets in other settings with different workflows Conclusions
Limitations and Challenges Large variations in ordering patterns Influence on usage by the current order sets Rare combinations of orders need to be addressed separately in a data-driven approach Constant CCC weights assumption Incorporation of new scientific evidence
Future Work Develop new approaches and extend/test current methods on other diagnoses and in other settings 1 Implemented an order set development platform and tested on pneumonia patients Incorporate alternate methods using heuristic optimization Evaluation by physicians on the usability and clinical validity of newly created order sets Currently looking for interested institutions to partner on the clinical evaluation studies 1: Zhang Y, Padman R, Levin JE. Data-driven Order Set Development Using Tabu Search. Heinz College Working Paper, Carnegie Mellon University, Pittsburgh, PA, May 2013.
Relevant Publications Zhang Y, Padman R, Levin JE. Clustering Methods for Data-driven Order Set Development in the Pediatric Environment. INFORMS 2012 DM-HI Workshop Proc., October 2012 Zhang Y, Levin JE, Padman R. Data-driven Order Set Generation and Evaluation in the Pediatric Environment. AMIA Symposium Proc., November 2012. Zhang Y, Levin JE, Padman R. Toward Order Set Optimization Using Click Cost Criteria in the Pediatric Environment. HICSS-46 Proc., January 2013. Zhang Y, Padman R, Levin JE. Data-driven Order Set Development in the Pediatric Environment: Toward Safer and More Efficient Patient Care. Heinz College Working Paper, Carnegie Mellon University, Pittsburgh, PA, December 2012.