1. Process Improvement, System Design, and Usability Evaluation:
Health Informatics in Low- and Middle-Income Countries
Short Course for Health Information System Professionals
Process Improvement, System Design, and Usability Evaluation:
People and Technology
Lecture A

2. People and Technology, Studies of Technology
Learning Objectives—Lecture A
Explain the importance of technology in health
Describe the contributions of human-computer interaction (HCI) to the health field
Define the concept of system usability
Distinguish between human factors and HCI as they apply to usability
Explain how cognitive, physical, and organizational ergonomics can be applied to human factors engineering
By the end of this lecture, students should be able to:
Explain the importance of technology in health.
Describe the contributions of human-computer interaction to the health field
Define the concept of system usability

3. Good Design (Stacie Branch, 2015, CC BY-SA 4.0)
In the next few slides, we will discuss some principles of good and poor design. One example of a good design is the point-of-sales system. This system is widely used, and intuitive. What make it intuitive and simple is that the layout is effective. One can disagree with the aesthetics of the color scheme but the contrasts are clear. The options are clearly laid out on the display, which allows you to monitor the calculations in real time on the same screen. This serves to minimize memory load and reduces the likelihood of an error.
(Stacie Branch, 2015, CC BY-SA 4.0)

4. Computerized Provider Order Entry Systems
This is an example of a computerized provider order entry system. It is a vitally important tool used for managing patients, ordering medications, ordering laboratory tests, and so forth. This system is central to the practice of medicine. Unfortunately, they are extraordinarily complex and have a steep learning curve. Just looking at this interface, you can see that it's dense with buttons, has a wide assortment of information, a long pick list, and doesn't readily support task functions.
(Horsky, Kaufman, Oppenheim, & Patel, 2003)

5. MedlinePlus Senior’s Health
In general, MedlinePlus is an excellent site for health consumers. This page presents content related to an elderly person’s health. It appears to be well-organized and effectively presented. Of course, this image is just a static page. You will get an opportunity to interact with it and answer some questions about its usability. Think back to the usability principles that we introduced in the first lecture (on Slide 9). Does it serve to minimize memory load? Is the system status visible? In this context, system visibility could be determined by clicking on a link and asking whether it is easy to find out where you are, relative to this front page. If users make an error, such as clicking on the wrong link, can they easily correct their error? Not all of the principles will be applicable, but it is a good way to begin conceptualizing judgments of usability. An important point to keep in mind is the target population. This is designed for seniors, and some of the answers to the above questions may be different for this population than they would be for younger adults.
(Kaufman, 2008)

6. Problematic Interfaces
Many of you are familiar with this controversial and confusing ballot used in the state of Florida for the 2000 United States presidential election. This is an interesting example of a deeply problematic interface design that was done without attention to human factors principles, resulting in significant consequences. Even something as mundane as a poorly designed ballot can result in significant usability problems and have serious consequences.
Image retrieved from Wikipedia

7. Modified Patient-Controlled Analgesic Device Interface
Study by Lin, Vicente, & Doyle, 2002
Lin, Vicente, & Doyle (2002) investigated the use of patient-controlled analgesic devices (PCAs). PCAs are devices controlled by patients to administer pain medication in particular doses. For example, a cancer patient or even someone with a broken leg may be given access to the PCA device to administer pain medication on their own. Before the devices are given to patients, the systems are programmed by nurses or technicians so that they control the overall rate of the medication administration and maximum dose for a give period of time. Excessive doses of pain medication can have severe consequences for a patient.
PCA interfaces vary in complexity, and the one shown on the left-hand side is known to be problematic. For example, the buttons support dual functions. The device has a very limited and poor interface, and poor feedback. In fact, this device is known to have contributed to a number of patient fatalities over time. The picture on the right represents a redesign, or rather, the computer simulation of a redesign of the interface. In their study, Lin and colleagues found that the original display introduced substantial cognitive complexity into the task and that a redesigned interface that adhered to human factors principles could lead to significantly faster, easier, and more reliable performance. Although it needs to be tested in a real-world setting, a redesigned interface is likely to significantly reduce errors.
(Lin, Isla, Doniz, Harkness, et. al, 2002)

8. Glucose Meters (Kaufman, 2008)
A wide range of glucose meters and many different kinds of products are available for people with diabetes to monitor their blood glucose. Shown here is an advanced-function glucose meter—roughly the equivalent of a Smartphone in design. It supports an enormous range of activities, in addition to obviously basic glucose measurement. You could store all sorts of information about your food intake, exercise, doctor's visits, and more. It's potentially a very powerful tool. Unfortunately, it's exceedingly complex to use. This tool was tested among older adults and it was found to be too complex for them to use productively. If you are a regular user of a Smartphone or a similar device, you may be able to more effectively negotiate this system, but many patients, especially older ones, are likely to experience considerable difficulty.
(Kaufman, 2008)

9. Interfaces of Note (CC BY-NC 2.0)
This is obviously not a medical technology. Rather, it's a car stereo system. It appears to emphasize form over function. There are many buttons. Though it's not visible in this image, it has very small fonts that are barely readable. It has a fancy animation sequence, which may be cool for a short period, but it's not a very productive use of space. Just an aside, car stereo systems contribute to more car accidents than cell phones. So human factors can play an important role, even in this context.
(CC BY-NC 2.0)

10. Do We Need Such Complexity?
Increasingly, you'll see photocopiers of immense complexity that support a wide range of tasks, from basic photocopying to complex collating. Most users require only a small subset of those functions, yet this manifest diversity of functionally resolves in layers of unnecessary complexity, as evidenced in this design.
(Kaufman, 1998)

11. News Sites Image retrieved from TheGuardian.com
The website of The Guardian, a British daily newspaper, won the 2016 Webby Award for best user experience.
Image retrieved from TheGuardian.com

12. Rhapsody (Interactive, 2016)
There are many other examples of systems that are reasonably well-designed. This is a common music player. It's a slightly older version, but it has a nice layout. And it's very easy to use and very intuitive to learn.
(Interactive, 2016)

13. Human Factors and Healthcare
Health devices
Electronic health record systems
Technology is central to the healthcare system. In fact, it is involved in every single interaction we have in the context of healthcare. Technology in healthcare can include devices or health information systems. Devices in healthcare include, for example, glucose meters used by patients with diabetes, infusion pumps, and telemedicine systems that may be used to talk to patients in their homes. Health information systems, such as electronic health records, are increasingly prevalent and show a remarkable diversity and level of complexity. Human factors is a discipline devoted to the study of technology systems and how people work with them.
(Kaufman, )
(Kaufman, )
Health IT Workforce Curriculum Version 4.0

14. People and Technology
Most of us genuinely love and value our technology, but we all experience moments of great frustration with it. By the way, this picture is meant to be illustrative. It's not recommended that you try this at home or at the office.

15. Transformative Nature of Technology
Technologies have enormous social and cognitive impact
Precipitates large-scale social changes
Technological changes have positive and negative consequences
Technologies do not just enable us to perform certain kinds of functions. They often also have a transformative effect. Technologies throughout history have had enormous social and cognitive impact and they certainly continue to do so. They have precipitated large-scale social changes. Technological changes invariably have both positive and negative consequences. We can find ways to promote the positive elements of technology and reduce the negative ones, to improve user satisfaction with the technology and ultimately to improve usability and productivity. Usability is the study of the ease with which people can employ a tool or technology to achieve a particular goal, which we will discuss in greater depth later in this unit.
Health IT Workforce Curriculum Version 4.0

16. Transformative Technologies
Here are a few interesting examples of transformative technologies. They've had long trajectories over many decades, sometimes over a century in which they have greatly changed shape, form, and function. Telephones are an interesting case in point. The initial design for a phone was that there would be one and only one phone for a single town. Penetration of the consumer market was remarkably slow, and it wasn't until the introduction of a long cord that allowed housewives to talk on the phone while doing household chores that the phone became a common household technology. The vacuum cleaner is a technology with something of a surprisingly controversial history. Initially, it was expected to free women from tedious and labor-intensive housework, and it did that to a certain degree. At the same time, it greatly raised the expectations of cleanliness of the house and to some extent this actually increased the amount of daily housework. Similarly, electronic health records were designed to make the practice of medicine much easier, by providing ready access by physicians to information. However, the reality is that the technology greatly increases the amount of time physicians spend documenting patient information. These are all unintended consequences of new technology. All technologies have unintended consequences.
(Dulaunoy, 2007, CC BY-NC-SA 2.0)
(Casonato, 2008, CC BY-NC-SA 2.0)

17. Modern Technologies (CC BY-NC-SA 2.0)
These are example of modern transformative technologies. They look rather different from the old telephones that you've seen and other technologies from the previous slide. They are slicker and in many respects easier to use, though sometimes that is somewhat deceptive. And they're such a central part of our lives that we really don't question their benefits. Of course, this won't be the case for all health information technologies. Many of them are still in a transition period. One technological advancement, the electronic health record system, is in the process of transitioning to the point where it's becoming increasingly central in all facets of healthcare, but doesn't yet have the widespread acceptance of, for example, something like the cell phone or an iPad.
(CC BY-NC-SA 2.0)

18. Early Medical Technologies
Many of the earliest medical technologies were not wholeheartedly welcomed. The stethoscope is an interesting case in point. Everybody can recognize a stethoscope as symbolic of the practice of medicine. However, there was a time when it was viewed with great skepticism; many practitioners thought this technology had the potential to cloud their judgments about patients. They also thought it had the potential to come between them and their patients. One could chart a dramatic cultural change over time, as the stethoscope became increasingly accepted and more widely adopted.
Early technologies are often quite difficult to use, but some demonstrate instrumental value. The iron lung, which was a critical technology, was developed in the 1920s. It’s the equivalent of today's respirator and it kept many patients alive during polio epidemics. It was a difficult system to use and it has given rise to many successive iterations in development, resulting in today's modern respiratory and pulmonary management systems.
The key point is that technologies evolve over time, as does their acceptance among practitioners.
(Curious Expeditions, 2008, CC BY-NC-SA 2.0;
Emmalouise99, 2007, CC BY-NC-SA 2.0)

19. What is usability?
Usability means how useful, usable, and satisfying a system is for the intended users to accomplish goals in the work domain by performing certain sequences of tasks 
(Zhang & Walji, 2011)
“Usability” has three subcomponents:
Learnability 
Efficiency 
Error tolerance 
(National Center for Cognitive Informatics and Decision Making, n.d.)
What do we mean by usability? It reflects the quality of a user's experience when interacting with the product or a system. We can enumerate several factors that affect the user's experience.
Usability means how useful, usable, and satisfying a system is for the intended users to accomplish goals in the work domain, by performing certain sequences of tasks
The scientists at the National Center for Cognitive Informatics and Decision Making (NCCD) have given a lot of thought to this definition. Let’s look at those three key concepts—  useful, usable, and satisfying—that make up usability for an electronic health record (EHR).
Useful – How well does the system support the work you’re trying to do? A system is useful if it can perform the functions necessary to get the job done. It’s that simple. 
Usable – Is it easy to learn, easy to use, and error-tolerant? You’d expect “usable” to be a part of Usability, right? But in this case, usable is a measurable concept and each of the subcomponents can be measured precisely:
Learnability – Ease of learning is important. Can a new user learn the system quickly and easily? Some technologies are merely walk up and learn technologies, the classic case being an ATM machine. However, most medical systems, medical devices, and complicated software have a learning curve.
Efficiency – how fast can a new user complete a task with the system?
Error tolerance – how well does the system help users avoid and recover from errors?
Satisfying – No, not the satisfaction you get from eating a good meal, but the satisfaction of a job well done. This is a subjective feeling, admittedly, but important if an EHR is to succeed in the workplace. Does the user like using the system? Does he or she feel that it’s a useful tool for their daily work? Or do they just dread having to enter data in the EHR? If so, it won’t be a satisfying experience, and no one would say that that EHR was usable.

20. Usability Principles Visibility of system status
Match between the system and the real world
User control and freedom
Consistency and standards
Error prevention
Recognition rather than recall
Flexibility and efficiency of use
Aesthetic and minimalist design
Help users recognize, diagnose, and recover from errors
Help and documentation
This list illustrates a set of widely accepted usability principles for evaluating the usability of a system. These principles were proposed by Jakob Nielsen, a highly regarded authority on usability, and they are part of a particular method of usability assessment know as the heuristic evaluation. We learned more about this method in the lecture on usability evaluation. I will briefly talk about a few of the principles.
“Visibility of system status” refers to how easily one can determine what state the system is in at a given moment. For example, if you just clicked on a link on a web page and it’s taking a long time to load, you should be able to tell whether the server is slow or overloaded or whether the page is no longer accessible.
“Recognition rather than recall” suggests that a user should not have to memorize complex command sequences to use an application. A system that provides multiple cues in text or icons to guide the user will diminish memory load. When memory load is reduced, a user can devote more of their energy to the task at hand.
Errors are inevitable, but a good system should allow a user to “recover from an error” without the risk of disastrous consequences, such as loss of data.
“Consistency and standards” refer to the fact that a system should adhere to widely acceptable standards and that there should be a measure of consistency across all displays.
Finally, an application that is engaging and pleasurable to use will likely be more widely adopted than one that is not.
(Nielsen, n.d.)
Health IT Workforce Curriculum Version 4.0

21. HCI and Human Factors Basic and applied discipline
Study of people using computers to work
People, systems, tasks
Ergonomics and human factors
User performance in context of any system
Computer, mechanical, or manual
Understanding
Humans in relation to their use of technologies
Human-computer interaction, or HCI, and human factors are sister disciplines, though human factors has been around a lot longer. It dates from the turn of the 20th century. Some of the earliest work was applied to productivity of workers in factory settings. For example, they would experiment with different kinds of environmental variables, such as temperature and lighting, to see how they affected the productivity of workers. Human computer interaction is a more recent discipline, dating from the early 1980s. Human computer interaction is focused on the study of people using computers in the context of work. The focus is on people, systems, and tasks. This emphasizes a particular kind of systems approach and so problems are never purely attributed to a system. It's really an interaction between an individual's skills and the resources provided by a system.
Tasks provide a convenient way to conceptualize the things that people do. So, for example, in using an electronic health record system, you could characterize any number of tasks related to retrieving particular kinds of information to make judgments about a patient or documenting certain findings. For example, if a physician recently performed a physical examination, a task would be to use the electronic health record system to document the results of the exam.
Human factors involves a broader study of user performance, not only in the context of computing systems but really with any system—computer, mechanical, or manual. It also has for our purposes a strong focus on the workplace and the study of workflow. Both disciplines involve understanding people in relation to the use of technologies.

22. HCI and Health Information Technologies
Enormous advances in health information technologies in the last decade
Potential for improved healthcare
Clinicians and patients employ health information technologies
Current designs are suboptimal
Increasingly, there are more options
There have been enormous advances in health information technologies in the past 15 years. There is tremendous potential for improved healthcare if we are able to harness these technologies and use them productively. Increasingly, clinicians and patients will be expected to employ health information technology. In fact, clinicians are routinely required to use many of these health information technologies. Although patients are a little bit behind the curve, that's beginning to change and there will increasingly be demands on patients to be involved in their own healthcare: for example, using personal health records and understanding their health status on the basis of information available in these health information technologies. We also know that current designs are frequently suboptimal. Some systems are more or less optimal, depending on a particular population of users and depending on a particular context. If you're somebody involved in the purchase of these technologies, or if you're a stakeholder of any kind, it's important to know that more and more options are available to you.
Health IT Workforce Curriculum Version 4.0

23. Why are human factors increasingly important?
Diversity of user groups
Complexity of systems
Increased use of technology
Increased “costs” of human error
Social emphasis on well-being and quality of life
The study of human factors has grown in importance and in public awareness in recent decades. Some of this is due to an increase in the use of technology by a diverse population of users (most of whom are not experts), a growing awareness of safety, and the cost of error. In industry, there is an increasing recognition of the need for better quality control and the fact that it conveys a competitive edge. The car industry is a case in point. There is a general perception that Japanese and European cars are better-quality, and this has hurt the sales of North American cars.

24. Human Factors Focus and Goals
Human beings and their interactions with products and equipment, tasks, environments
Micro, macro, ambient
Goal
Design systems and system components to match the capabilities and limitations of humans who use them
Optimize working and living conditions
What is the central focus of HF work? Broadly, it is people and their interaction with the gamut of technologies and systems. This interaction can occur at different levels. The micro level, is, according to Bubb (2012), “more engineering oriented” and ”gives rules for the technical design of workplaces and working means.” In contrast, Bubb defines “macro ergonomics”, as providing “rules for the creation of organization, company groups and study groups.”
The goal of this work is to optimize these technologies, such that they match the capabilities and limitations of people who use them. As we will discuss, cognitive psychology plays an important role in human factors analysis. 24
Health IT Workforce Curriculum Version 4.0

25. Examples of Application Areas
Military
Highway systems
For example, readability and comprehensibility of signs
Automobile design and driver behavior
Human-computer systems
Healthcare and patient safety
Aging and accessibility
Nuclear power
Workplace layout and furnishings
Airline industry
For example, pilot workload, aircraft design, automation, aircraft maintenance
Quality control
These are just some examples of application areas in human factors. We will focus on computer systems and patient safety, but we will also touch on other areas of interest that exemplify important concepts and principles.

26. Nuclear Power Plant Control Room
Here is a picture of a control room in a nuclear power plant. Safety is a crucial issue in these settings. The television cartoon character Homer Simpson is probably not the embodiment of the ideal nuclear power safety inspector, although he is the best known. The picture illustrates the immense complexity of such an environment and one can imagine the demands on human operators. There have been several significant accidents in such plants, and they have served as excellent case studies in human factors engineering.
(Lambrev, 2009)

27. Airplane Cockpit (Headly, 2005, CC BY NC-SA 4.0)
The aviation industry is one of the first to embrace the discipline of human factors as well as human factors engineering principles. Although airplane crashes make the front pages of our newspapers when they happen, the airline industry has had an excellent safety record in recent decades. Much work has been done on airplane cockpit displays (see work done by Christopher Wickens) and airlines have evolved a rather elaborate set of procedures for ensuring safe practices, which includes rigorous training of pilots and ensuring maintenance of pilots’ competencies. We hear about the failures in aviation, but on balance they set a high standard for safety.
In the past decade or so, the healthcare sector has to some extent patterned itself after the aviation industry. Checklists, structured communication techniques, error reporting, and simulator training are some of ways in which the healthcare sector has endeavored to adopt the safety practices and methods of the aviation industry.
(Headly, 2005, CC BY NC-SA 4.0) 27

28. Human Factors in Medicine
Infusion pumps
Anesthesia equipment
Medication errors
Effects of fatigue on resident’s performance
Judgmental limitations in medical decision making
Inadequate infection control
Unintended consequences of automation
These are some examples of applications of human factors work in medicine. Infusion pumps are used to infuse fluid and medications in patients’ circulatory system. In recent years, there have been growing concerns about the safety of these devices. More than 1 million patients are injured annually by medication errors. Although most are relatively minor, some result in serious harm and even fatalities. The effects of fatigue, night shift work, and sleep deprivation on human performance and the safety of medical care are subjects of enormous interest within the medical community. Although this remains a controversial issue, several states have mandated that medical residents limit the number of consecutive hours in a given shift. Infection control continues to be a serious problem in hospitals. Simple intervention like the close scrutiny of hand-washing behavior of clinicians has had a significant impact on controlling infection rates in hospitals. All technology has unintended consequences, ranging from positive to highly detrimental to patient safety.
We will return to the issue of patient safety and discuss some of these examples after we review some of the core concepts and central issues in human factors.

29. Human Factors and Ergonomics
Scientific discipline concerned with understanding interactions:
Among humans
Other elements of a system
Profession that applies theory, principles, data, methods to design to optimize:
Human well-being
Overall system performance
System can be a technology, device, person, team, organization, policy, etc.
As we discussed previously, human factors is a profession that applies theory, principles, data, and methods to design in order to optimize human well-being and overall system performance. Unlike HCI, a system does not merely refer to a computing system. It may be a device, person, team, organization, or policy, to name a few.
(Carayon, 2007)

30. Human Factors Ergonomics: Three Major Domains
Physical ergonomics
Cognitive ergonomics
Organizational/macro-ergonomics
Ergonomics is broadly conceived as the study of work and factors that affect it. The term is sometimes used interchangeably with human factors. In any case, we can characterize 3 major domains: physical ergonomics, cognitive ergonomics, and organizational or macro-ergonomics. Let’s look at each of these now.
Health IT Workforce Curriculum Version 4.0

31. Physical Ergonomics Concerned with physical activity
Topics: Repetitive movements, workplace layout, safety, and health
Application to health: Reducing and preventing injury, designing workstations and work rooms for optimal human performance
Examples:
Designing a patient room to facilitate and support patient care
Designing medical labels so that they are readable and understandable
Physical ergonomics is concerned with physical activity and covers a wide range of related issues, including understanding and reducing injuries in the workplace (such as repetitive stress injury). In the healthcare domain, this could include reducing and preventing injury and designing workstations and workrooms for optimal human performance. An example would be designing medication labels so that they are readable and understandable.
Health IT Workforce Curriculum Version 4.0

32. Organizational Ergonomics
Concerned with socio-technical systems
Topics: Communication, teamwork, participatory design, quality management
Application to health: Reducing stress and burnout, improving satisfaction and retention, implementing improvement activities
Examples:
Management training in surgery teams
Designing work schedules for reduced fatigue and enhanced performance
Organizational ergonomics is concerned with the study of sociotechnical systems. The topics in this area include communication, teamwork, participatory design, and quality management. An example of an application to health is taking steps to reduce stress and employee burnout. Redesigning work schedules is one way to diminish the risk of burnout.
Health IT Workforce Curriculum Version 4.0

33. Cognitive Ergonomics Concerned with mental processes
Topics: Mental workload, decision making, skilled performance, HCI, work stress,
Application to health: Usability, designing training systems, usable interfaces
Examples:
Event report systems
Implementing incident analysis system
Recall that cognition, as defined by the Merriam-Webster dictionary, is “conscious mental activities: the activities of thinking, understanding, learning, and remembering.” So cognitive ergonomics addresses the gamut of cognitive issues: decision making, skilled performance, and mental workload, which is an issue that we will come back to later in this lecture. It also addresses usability of systems, which is a central issue in this course. This lecture focuses predominantly on cognitive issues.
Health IT Workforce Curriculum Version 4.0

34. Human Factors Design Implications
Reduce the number of competing channels of information
Make sources of information as distinct as possible
Provide feedback
Use redundant cues
Work/rest scheduling
Training
On the basis of human factors research, we can discern the following design implications.
If possible, reduce the number of channels that one needs to attend to.
Make sources of information as distinct as possible. For example, an intensive care nurse may hear 10 or 12 different kinds of alarms that signal various kinds of patient needs or concerns. However, the sounds are sufficiently distinct that he or she will know which ones warrant immediate attention and which ones can wait a little longer.
Electronic health records can be designed (or templates developed) that correspond better to the desired clinician workflow and displays can be structured to provide easier access to needed information.
Work scheduling can be used to reduce fatigue and improve performance.
Although training is not a substitute for poor design, it can provide workers with strategies and resources to work more productively within the limitations of a given system.

35. Why It May Matter to You
Technology is a potential source of error and can compromise patient safety
Position to make a difference in product selection
The best technologies have some deficiencies
Better and worse choices
So, why does this matter to you? Well, technology is a potential source of error and we know it can compromise patient safety. Potentially, either now or at some point in your career, you'll be in a position to make a difference in product selection. It may vary depending on the kind of position that you have. You may be in a leadership position or you might be a user of the technology, and to the extent that you can understand and communicate and persuade people, you can make a difference. It's important to know that even the best technologies have some deficiencies, and every single setting and context is somewhat different. There are always better and worse choices, whether you are purchasing a system from a vendor or whether you are developing one in-house, and usability and human factors should play an important role in that process.

36. People and Technology Summary—Lecture A
Social and cognitive consequences of new technologies
Usability and usability principles
HCI and human factors as a discipline
Why these disciplines matter in the context of health information technologies
NEXT: Patient safety, introduction to design, and user‑centered design
This concludes Lecture A of People and Technology, Studies of Technology. This introductory lecture introduced issues pertaining to the social and cognitive consequences of new technologies, especially in the context of healthcare. The lecture also introduced the disciplines of human computer interaction and human factors. This led us to the very important concept of usability and why it matters in the context of health information technologies.

37. People and Technology References—Lecture A
Bubb, H. (2012). Information ergonomics. In Herczeg, M., & Stein, M. (Eds.), Information ergonomics: A theoretical approach and practical experience in transportation (p. 23). Berlin, Heidelberg: Springer.
Horsky, J., Kaufman, D. R., Oppenheim, M. I., & Patel, V. L. (2003). A framework for analyzing the cognitive complexity of computer-assisted clinical ordering. Journal of Biomedical Informatics, 36,
Kaufman, D. R., Patel, V. L., Hilliman, C., Morin, P. C., Pevzner, J., Weinstock, G. R., Shea, S., & Starren, J. (2003). Usability in the real world: Assessing medical information technologies in patients’ homes. Journal of Biomedical Informatics, 36,
Kaufman, D. R., Pevzner, J., Rodriguez, M., Cimino, J. J., Ebner, S., Fields, L., et al. (2009). Understanding workflow in telehealth video visits: Observations from the IDEATel project. Journal of Biomedical Informatics, 42(4),
Kaufman, D. R. & Starren, J. B. (2006). A methodological framework for evaluating mobile health devices. In The Proceedings of the American Medical Informatics Annual Fall Symposium (p. 978). Philadelphia, PA: Hanley & Belfus.
Lin, L., Isla, R., Doniz, K., Harkness, H., Vicente, K. J., & Doyle, D. J. (1998). Applying human factors to the design of medical equipment: patient-controlled analgesia. Journal of Clinical Monitoring & Computing, 14(4),
No Audio.
Health IT Workforce Curriculum Version 4.0

38. People and Technology References—Lecture A
National Center for Cognitive Informatics and Decision Making in Healthcare. What is usability. Retrieved from
Nielsen, J. (1993). Usability engineering. Boston, MA: Academic Press.
Norman, D. A. (1986). Cognitive engineering. In D. A. Norman & S. W. Draper (Eds.), User centered system design: New perspectives on human-computer interaction (pp ). Hillsdale, NJ: Lawrence Erlbaum Associates.
Zhang, J. & Walji, M. (2011). TURF: Toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6),
No Audio.
Health IT Workforce Curriculum Version 4.0

39. People and Technology References—Lecture A
Images
Horsky, J., Kaufman, D.R., Oppenheim, M.I., & Patel, V.L. (2003). A framework for analyzing the cognitive complexity of computer-assisted clinical ordering. Journal of Biomedical Informatics, 36, 4-22.
Clockwise from the bottom left-hand corner
Kaufman, D. R., & Starren, J. B. (2006). A methodological framework for evaluating mobile health devices. In The Proceedings of the American Medical Informatics Annual Fall Symposium (p. 978). Philadelphia, PA: Hanley & Belfus.
Kaufman, D. R., Pevzner, J., Hilliman, C., Weinstock, R. S., Teresi, J., Shea, S., & Starren, J. (2006). Re- designing a telehealth diabetes management program for a digital divide seniors population. Home, Healthcare, Management & Practice, 18,
Infusion Pump—unpublished work (2002) with Patel, Kubose and Kaufman.
Kaufman, D. R., Pevzner, J., Rodriguez, M., Cimino, J. J., Ebner, S., Fields, L., et al. (2009). Understanding workflow in telehealth video visits: Observations from the IDEATel project. Journal of Biomedical Informatics, 42(4),
Kaufman, D. R., Patel, V. L., Hilliman, C., Morin, P. C., Pevzner, J., Weinstock, G. R., Shea, S., & Starren, J. (2003). Usability in the real world: Assessing medical information technologies in patients’ homes. Journal of Biomedical Informatics, 36,
Kaufman, D., & Hripcsak, G. (2008). eClinicalWorks screen. Unpublished usability analysis of eClinicalWorks.
No Audio.
Health IT Workforce Curriculum Version 4.0

40. People and Technology References—Lecture A
Images
Kaufman, D. (1998). Personal image for a class on usability in the Graduate School of Education at UC Berkeley. Department of Biomedical Informatics, Columbia University Medical Center.
Retrieved June 20, 2016, from
Interactive, A. (2016). Rhapsody 6 PC Software Guide. Rhapsody Help. Retrieved June 24, 2016, from
Retrieved June 20, 2016, from amlb_texmlb_1,game_state=Live,game_tab=field
Clipart. Image Courtesy of Microsoft.
Very old telephone [online image]. (n.d.). Retrieved August 3, 2010, from
Poster advertisement for an Electro Lux vacuum with a woman using it [online image]. (n.d.). Retrieved August 3, 2010 from
No Audio.
Health IT Workforce Curriculum Version 4.0

41. People and Technology References—Lecture A Images
New technologies such as Blackberry, GPS, iPhone, iPad, and iPods [online images]. (n.d.). Retrieved August 3, 2010, from:
Ancient doctor's tools [online image]. (n.d.). Retrieved August 3, 2010, from
Ancient Egyptian stethoscope carved in stone [online image]. (n.d.). Retrieved August 3, 2010, from
Ancient enemas [online image]. (n.d.). Retrieved August 3, 2010, from
Mechanical therapy (hand-drawn picture of a man sitting on a complex machine) [online image]. (n.d.). Retrieved August 3, 2010, from
No Audio.

42. People and Technology References—Lecture A Images
Lambrev, Y. (2009). Kozloduy nuclear plant control room [online image]. Retrieved August 20, 2010, from _Control_Room_of_Units_3_and_4.jpg. [CC-BY-3.0 ( via Wikimedia Commons
Headly, A. (2005). Martin 404 cockpit [online image]. Retrieved September 10, 2010, from
Retrieved on August 3rd, 2010 from
Horsky, J., Kaufman, D. R., Oppenheim, M. I., & Patel, V. L. (2003). A framework for analyzing the cognitive complexity of computer-assisted clinical ordering. Journal of Biomedical Informatics, 36,
Horsky, J., Kaufman, D. R., Oppenheim, M. I., & Patel, V.L. (2003). A framework for analyzing the cognitive complexity of computer-assisted clinical ordering. Journal of Biomedical Informatics, 36, Retrieved from
No Audio. 42

43. People and Technology References—Lecture A Images
Retrieved August 3, 2010 from
Lin, L., Isla, R., Doniz, K., Harkness, H., Vicente, K. J., & Doyle, D. J. (2002). Applying human factors to the design of medical equipment: patient-controlled analgesia. Journal of Clinical Monitoring & Computing, 14(4),
Kaufman, D. R. & Starren, J. B. (2006). A methodological framework for evaluating mobile health devices. In The Proceedings of the American Medical Informatics Annual Fall Symposium (p. 978). Philadelphia, PA: Hanley & Belfus.
Candid. (n.d.). All sizes [online image]. Retrieved August 3, 2010, from
No Audio. 43

44. This material was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC This material was updated by The University of Texas Health Center at Houston under Award Number 90WT0006.
This presentation was produced with the support of the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AID-OAA-L MEASURE Evaluation is implemented by the Carolina Population Center, University of North Carolina at Chapel Hill in partnership with ICF International; John Snow, Inc.; Management Sciences for Health; Palladium; and Tulane University. Views expressed are not necessarily those of USAID or the United States government.