1. Alerts in Clinical Information Systems: Building Frameworks and Prototypes Project presented by Rolf Wipfli Project team: Rolf Wipfli Prof. Christian Lovis

2. 2 ©2010 University Hospitals of Geneva Content of presentation Framework with different views on alert systemsFramework with different views on alert systems Possible implementation of alerts in a Clinical Information SystemPossible implementation of alerts in a Clinical Information System Ongoing researchOngoing research PhD thesis (Psychology)PhD thesis (Psychology)

3. 3 ©2010 University Hospitals of Geneva Why medical alerts are important? Provides decision support for handling the ever growing quantity of medical data and knowledgeProvides decision support for handling the ever growing quantity of medical data and knowledge Studies show positive effect on prescribing behavior (e.g., more efficient) or reduced error ratesStudies show positive effect on prescribing behavior (e.g., more efficient) or reduced error rates Schedlbauer et al. (2009), Weingart et al. (2009), Koppel et al. (2005), Raschke et al. (1998)

4. 4 ©2010 University Hospitals of Geneva Problems identified High number of alert overrides due to: Unspecific or non applicable alertsUnspecific or non applicable alerts Alert fatigue caused by irrelevant and repeating alertsAlert fatigue caused by irrelevant and repeating alerts Usability issues: misunderstood or unnoticed alerts, interruption of work processesUsability issues: misunderstood or unnoticed alerts, interruption of work processes Problem of system acceptanceProblem of system acceptance Van der Sijs et al. (2006), Hsieh et al. (2004), Shah et al. (2006)

5. 5 ©2010 University Hospitals of Geneva Open issues How to make alerts more context depended?How to make alerts more context depended? How to integrate alerts in CIS? When and how should they be displayed in order to be best adapted to physician's work processes?How to integrate alerts in CIS? When and how should they be displayed in order to be best adapted to physician's work processes? How can physicians best give feedback to improve the alert system?How can physicians best give feedback to improve the alert system?

6. 6 ©2010 University Hospitals of Geneva System-centric view

7. 7 ©2010 University Hospitals of Geneva User-centeric view Norman (2002)

8. 8 ©2010 University Hospitals of Geneva Evaluation: Levels of decision making Rasmussen (1986)

9. 9 ©2010 University Hospitals of Geneva Organization-centric view Kuuti (1996), Carayon et al. (2007)

10. 10 ©2010 University Hospitals of Geneva Requirements System needs to hold alert information seamlessly in information cycle, providing feedback functionality [system-centric view]System needs to hold alert information seamlessly in information cycle, providing feedback functionality [system-centric view] The interface should use unified processes for alert interaction [user-centric view]The interface should use unified processes for alert interaction [user-centric view] Alerts support medical decision making, hence it should correspond to theories of human decision making [levels of decision making]Alerts support medical decision making, hence it should correspond to theories of human decision making [levels of decision making] Alert system should implement the whole context (collaboration with other care providers, etc.) [organizational view]Alert system should implement the whole context (collaboration with other care providers, etc.) [organizational view]

11. 11 ©2010 University Hospitals of Geneva Mockup skill-based alert screen

12. 12 ©2010 University Hospitals of Geneva Mockup rule-based alert screen

13. 13 ©2010 University Hospitals of Geneva Mockup knowledge-based alert screen

14. 14 ©2010 University Hospitals of Geneva Next steps Further development of frameworkFurther development of framework Interviews with physicians, work observationsInterviews with physicians, work observations Build scenario-based prototypeBuild scenario-based prototype Experimentally test prototypeExperimentally test prototype Population: physicians of different departments at University Hospitals of GenevaPopulation: physicians of different departments at University Hospitals of Geneva Method: Scenario-based usability testMethod: Scenario-based usability test

15. 15 ©2010 University Hospitals of Geneva

16. 16 ©2010 University Hospitals of Geneva Bibliography Hsieh, T. C., Kuperman, G. J., Jaggi, T., Hojnowski-Diaz, P., Fiskio, J., Williams, D. H., et al. (2004). Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. Journal of the American Medical Informatics Association : JAMIA, 11(6), 482-91. doi: 10.1197/jamia.M1556.Hsieh, T. C., Kuperman, G. J., Jaggi, T., Hojnowski-Diaz, P., Fiskio, J., Williams, D. H., et al. (2004). Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. Journal of the American Medical Informatics Association : JAMIA, 11(6), 482-91. doi: 10.1197/jamia.M1556. Koppel, R., Metlay, J. P., Cohen, A., Abaluck, B., Localio, A. R., Kimmel, S. E., et al. (2005). Role of computerized physician order entry systems in facilitating medication errors. JAMA : the journal of the American Medical Association, 293(10), 1197-203. doi: 10.1001/jama.293.10.1197.Koppel, R., Metlay, J. P., Cohen, A., Abaluck, B., Localio, A. R., Kimmel, S. E., et al. (2005). Role of computerized physician order entry systems in facilitating medication errors. JAMA : the journal of the American Medical Association, 293(10), 1197-203. doi: 10.1001/jama.293.10.1197. Kuuti, K. (1996). Activity Theory as a Potential Framwork for Human-Computer Interaction Research. In B. A. Nardi, Context and Consciousness: Activity Theory and Human-Computer Interaction (pp. 17-44). MIT Press.Kuuti, K. (1996). Activity Theory as a Potential Framwork for Human-Computer Interaction Research. In B. A. Nardi, Context and Consciousness: Activity Theory and Human-Computer Interaction (pp. 17-44). MIT Press.

17. 17 ©2010 University Hospitals of Geneva Bibliography Norman, D. A. (2002). The design of everyday things. New York NY: Basic Books.Norman, D. A. (2002). The design of everyday things. New York NY: Basic Books. Raschke, R. A., Collihare, B., Wunderlich, T. A., Guidry, J. R., Leibowitz, A. I., Peirce, J. C., et al. (1998). A Computer Alert System to Prevent Injury From Adverse Drug Events: Development and Evaluation in a Community Teaching Hospital. JAMA: The Journal of the American Medical Association, 280(15), 1317-1320. doi: 10.1001/jama.280.15.1317.Raschke, R. A., Collihare, B., Wunderlich, T. A., Guidry, J. R., Leibowitz, A. I., Peirce, J. C., et al. (1998). A Computer Alert System to Prevent Injury From Adverse Drug Events: Development and Evaluation in a Community Teaching Hospital. JAMA: The Journal of the American Medical Association, 280(15), 1317-1320. doi: 10.1001/jama.280.15.1317. Rasmussen, J. (1986). Information processing and human-machine interaction. (A. P. Sage). Amsterdam, The Netherlands: Elsevier Science Publishers.Rasmussen, J. (1986). Information processing and human-machine interaction. (A. P. Sage). Amsterdam, The Netherlands: Elsevier Science Publishers.

18. 18 ©2010 University Hospitals of Geneva Bibliography Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., et al. (2006). Improving acceptance of computerized prescribing alerts in ambulatory care. Journal of the American Medical Informatics Association : JAMIA, 13(1), 5-11. doi: 10.1197/jamia.M1868.Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., et al. (2006). Improving acceptance of computerized prescribing alerts in ambulatory care. Journal of the American Medical Informatics Association : JAMIA, 13(1), 5-11. doi: 10.1197/jamia.M1868. Schedlbauer, A., Prasad, V., Mulvaney, C., Phansalkar, S., Stanton, W., Bates, D. W., et al. (2009). What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior? Journal of the American Medical Informatics Association : JAMIA, 16(4), 531-8. doi: 10.1197/jamia.M2910.Schedlbauer, A., Prasad, V., Mulvaney, C., Phansalkar, S., Stanton, W., Bates, D. W., et al. (2009). What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior? Journal of the American Medical Informatics Association : JAMIA, 16(4), 531-8. doi: 10.1197/jamia.M2910.

19. 19 ©2010 University Hospitals of Geneva Bibliography van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006). Overriding of drug safety alerts in computerized physician order entry. Journal of the American Medical Informatics Association : JAMIA, 13(2), 138-47. doi: 10.1197/jamia.M1809.van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006). Overriding of drug safety alerts in computerized physician order entry. Journal of the American Medical Informatics Association : JAMIA, 13(2), 138-47. doi: 10.1197/jamia.M1809. Weingart, S. N., Simchowitz, B., Padolsky, H., Isaac, T., Seger, A. C., Massagli, M., et al. (2009). An empirical model to estimate the potential impact of medication safety alerts on patient safety, health care utilization, and cost in ambulatory care. Archives of internal medicine, 169(16), 1465-73. doi: 10.1001/archinternmed.2009.252.Weingart, S. N., Simchowitz, B., Padolsky, H., Isaac, T., Seger, A. C., Massagli, M., et al. (2009). An empirical model to estimate the potential impact of medication safety alerts on patient safety, health care utilization, and cost in ambulatory care. Archives of internal medicine, 169(16), 1465-73. doi: 10.1001/archinternmed.2009.252.