1. The Alert That Cried Wolf: Optimization of Clinical Decision Support Alerts
Juanqin (Stephanie) Wei, PharmD
PGY-1 Pharmacy Resident

2. Disclosures
Neither the presenter nor the planning committee has received any commercial support associated with this educational activity.

3. Objectives
List the roles of clinical decision support alerts in practice
Identify actions that can be taken to reduce the risk of alert fatigue

4. Outline Background RWJUH Somerset: CDS in test mode Conclusion
Clinical decision support (CDS)
Alert fatigue
RWJUH Somerset: CDS in test mode
Cumulative acetaminophen CDS alert
Order set revision
Pre-alert and post-alert evaluation
Conclusion

5. Background: Clinical Decision Support (CDS)

6. What is Clinical Decision Support?
Health information technology (IT) function that builds on the foundation of an electronic health record (EHR)
Presents information at appropriate times
Filters and organizes information
Enhances patient health and healthcare
Assists providers to make higher quality decisions
Does NOT replace clinical judgement
75 Fed. Reg. 144 (July 28, 2010).

7. Meaningful Use
Centers for Medicare & Medicaid Services (CMS) EHR Incentive Program
Stage 3
Use of multiple CDS interventions to apply quality measures
Enable drug-drug and drug-allergy interaction checks
75 Fed. Reg. 144 (July 28, 2010).

8. Goals of CDS In order to provide benefit, CDS must provide:
The right information
To the right people
Through the right channels
In the right intervention formats
At the right points in workflow
Osheroff J, Teich J, Levick D, et al

9. Types of CDS Computerized alerts and reminders
Clinical guideline based order sets
Condition-specific order sets
Documentation templates
Relevant reference information
Osheroff J, Teich J, Levick D, et al

10. CDS Alerts: Advantages and Disadvantages
Variety of automated alert types and checks
Additional layer of safety
Customization tailored to site of use
Disadvantages
Excessive number of alerts
Slows workflow
Maintenance of custom alerts

11. Background: Alert fatigue

12. What is “Alert Fatigue”?
Desensitization to significant alerts due to an overload of insignificant or irrelevant alerts
Leads to:
False sense of safety
Increased risk of adverse events

13. CDS Alerts: Outcomes Bates DW et al., 1994 Weingart SN et al., 2003
Evaluated potential for preventability by computerized information systems
81% potential medication error reduction
Weingart SN et al., 2003
91.2% override rate of drug-allergy interactions
89.4% override rate of high severity drug-drug interactions
36.5% of alerts evaluated by physicians to be inappropriate
Bates: Analysis of 133 adverse events

14. CDS Alerts: Outcomes Beccaro MA et al., 2010
Customization of CDS alerts for 250 medications
Dose range checking alert firing rate decreased from 23.4% to 7.4%
Drug-drug interaction alert firing rate decreased from 13.5% to 4.8%
No significant increase in reported medication errors
Retrospective analysis of CDS alerts (dose range checks, drug-drug interactions, and drug-allergy)
Customization based on clinical judgement
No increase in errors despite an increase in reporting and total orders

15. RWJUH Somerset: cds alerts in test mode

16. RWJUHS: Monitoring Total Daily Dose (TDD) of Acetaminophen
January 2014: FDA alert
Discontinue the use of combination products containing greater than 325 mg of acetaminophen (APAP)
Zhou L et al., 2012
6% of hospitalized patients exceeded the APAP maximum TDD of 4000 mg
22% of 65 years or older exceeded the APAP maximum TDD of 3000 mg
17% of patients with chronic liver disease

17. RWJUHS: Monitoring TDD of APAP
Medication safety committee
Multiple orders of APAP-containing medications prescribed for multiple indications
Discussed need for APAP dosing restriction
Cerner APAP cumulative dosing alert
Calculated daily cumulative dose based on eMAR
Maximum TDD defined as 4000 mg for ≥ 50 kg and 75 mg/kg/day for < 50 kg
Suppressed until defined threshold of 3000 mg reached

18. RWJUHS: Monitoring TDD of APAP
Alert test mode run
0.2% incidence of APAP TDD administrations over 4000 mg
Multiple cases of exceeding the TDD occurred in Labor and Delivery
Review of order sets to reduce risk
Oxycodone/Acetaminophen 5/325 mg
Original: 2 tablets by mouth every 3 hours as needed
Revision: 2 tablets by mouth every 4 hours as needed
predisposition to exceed maximu

19. RWJUHS: Monitoring TDD of APAP
Post alert and order set revision study evaluation
Cumulative dosing alert run in test mode
Revisions were implemented July 2014
Pre-alert period: January 1st, 2014 – June 4th, 2014
Post-alert period: January 1st, 2015 – June 4th, 2015

20. RWJUHS: Monitoring TDD of APAP
Outcomes
Pre-Alert
(n = 8,122)
Post-Alert
(n = 8,456)
Odds Ratio (95% CI)
p-value
Primary Outcome
TDD ≥ 3 grams (%)
194 (2.4)
99 (1.2)
0.50
( )
< 0.001
Age ≥ 65 years
old (%)
81 (41.7)
37 (37.7)
0.83
( )
0.4700
Chronic liver
disease (%)
6 (3.1)
1 (1.0)
0.32
( )
0.2943
Secondary Outcome
TDD ≥ 4 grams (%)
16 (0.2)
4 (0.05)
0.24
( )
0.0107

21. Conclusions
Running the CDS alert in test mode allowed identification and correction of root causes
Significant reductions were seen in supratherapeutic APAP dosing
A live alert did not need to be implemented, reducing alert fatigue risk

22. Assessment Question 1
Which of the following is NOT an appropriate role for CDS alerts?
Alerting providers to drug-drug interactions
Acting as a replacement for clinical judgement
Listing alternative recommendations
Reminding providers of incomplete tasks or orders

23. Assessment Question 1
Which of the following is NOT an appropriate role for CDS alerts?
Alerting providers to drug-drug interactions
Acting as a replacement for clinical judgement
Listing alternative recommendations
Reminding providers of incomplete tasks or orders

24. Assessment Question 2
Of the following, which is the most appropriate method to reduce the risk of alert fatigue from CDS alerts?
Remove CDS alerts entirely from the EHR
Exempt all providers from receiving CDS alerts
Resize CDS alert pop-ups to be smaller
Customize the specificity of CDS alert triggers

25. Assessment Question 2
Of the following, which is the most appropriate method to reduce the risk of alert fatigue from CDS alerts?
Remove CDS alerts entirely from the EHR
Exempt all providers from receiving CDS alerts
Resize CDS alert pop-ups to be smaller
Customize the specificity of CDS alert triggers

26. References
Medicare and Medicaid Programs: Electronic Health Record Incentive Program, 75 Fed. Reg (July 28, 2010). Federal Register: The Daily Journal of the United States. Web. 2 February 2017.
Osheroff J, Teich J, Levick D, et al. Improving outcomes with CDS: an implementer's guide. 2nd ed. Chicago: HIMSS Publishing; 2012.
Bates DW, O'Neil AC, Boyle D, et al. Potential identifiability and preventability of adverse events using information systems. J Am Med Inform Assoc Sep-Oct;1(5):404–411.
Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med Nov 24;163(21):2625– 2631.
Beccaro MA, Villanueva R, Knudson KM, et al. Decision Support Alerts for Medication Ordering in a Computerized Provider Order Entry (CPOE) System. Appl Clin Inform. 2010;1(3):346-62
U.S. Food and Drug Administration [Internet]. Silver Spring (MD): U.S. Department of Health and Human Services;c2016. FDA recommends health care professionals discontinue prescribing and dispensing prescription combination drug products with more than 325 mg of acetaminophen to protect consumers [cited 2017 Feb 02]; [1 screen]. Available from:
Zhou L, Maviglia SM, Mahoney LM, et al. Supratherapeutic dosing of acetaminophen among hospitalized patients. Arch Intern Med Dec;172(22):

27. The Alert That Cried Wolf: Optimization of Clinical Decision Support Alerts
Juanqin (Stephanie) Wei, PharmD
PGY-1 Pharmacy Resident