1. Usability and Human Factors
In this lecture we will first discuss a theoretical introduction to the study of cognition and human Computer Interaction. Most emphasis will be placed on information processing models in the context of a cognitive engineering approach. We will be talking about classical model of human information processing including perception, attention, memory and cognition. The implications for design will be illustrated in the context of Gestalt organizing principles of perception. This lecture will introduce the concept of mental models and explain their role in understanding interactive behavior. The last topic will focus on distributed cognition, which represents a new way to understand human computer interaction and challenges some concepts that are central to the conventional information processing models.
Cognition and Human Performance
Lecture a
This material (Comp15_Unit3a) was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC

2. Cognition and Human Performance Learning Objectives
Describe the impact of different kinds of representation on cognition as it applies to human computer interaction and web design (Lecture a)
Describe how humans process information and obtain skills (Lecture a)
Describe the Gestalt principles of perception and their relevance to human computer interaction and cognitive theory (Lecture a)
The unit objectives realized in this lesson include:
1) Describe the impact of different kinds of representation on cognition as it applies to human computer interaction and web design;
2) Describe how humans process information and obtain skills;
3) Describe the Gestalt principles of perception and their relevance to human computer interaction and cognitive theory
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

3. Cognition and Human Computer Interaction (HCI)
HCI is intersection between psychology/ social sciences and computer sciences/ technology
HCI seeks to understand and support human beings interacting with technology
Drawing on basic science research
Applied cognitive HCI work
Study of cognition and information processing is increasingly viewed as central to advances in HCI
This is a course on usability and human factors. Why should we be concerned with human cognition or theories that largely arise out of the psychological literature? There are 2 answers to that question. The first is that the computing experience is a deeply human endeavor that engages the cognitive and perceptual systems. The second is that the disciplines of HCI and Human Factors draw heavily on both basic and applied research. In this particular set of lectures, we will draw on cognitive theories in view to understand the way it affects the computing experience in healthcare contexts.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

4. HCI: Cognitive Engineering Approach
Principles, methods and tools to assess and guide design of computerized systems to support human performance
Focus on attention, perception, memory, comprehension, problem solving, and decision making
Analysis of cognitive tasks and processing constraints imposed by the human information processing system
The cognitive engineering approach is one of the most influential in HCI and Human Factors. Don Norman, who we discussed previously, is one of the pioneers of this approach. Cognitive engineering places a strong emphasis on cognitive, perceptual and motoric responses in the context of interacting with any system. The approach also embraces a cognitive task analytic perspective and endeavors to understand the ways in which human attributes (e.g., perceptual abilities) and limitations (e.g., limits on working memory) impact performance.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

5. Cognitive Theory and HCI
Informative
Useful research findings
Predictive
Providing tools to model user behavior
Prescriptive
Providing advice for design or evaluation
Explanatory
Accounting for user behavior
Generative
Provide design dimensions and constructs to inform design and selection of external representations
What can cognitive theory do for you? Or better stated, what can it do for HCI? Well, it can inform our understanding, be used to predict or explain end-user behavior in both general and more precise ways. For example, a general way would be a prediction that a given screen would exert a heavy memory load on any user because of its clutter and complexity. A more precise prediction might involve a model of a novice user performing a particular task such as searching Google for treatment for migraines. Cognitive theory can also seed novel design concepts that may be a better match with users’ models of the world.
Let’s consider a design that pays absolutely no attention to human capabilities.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

6. Driving with a Computer Keyboard
The example was drawn from a text by Jenny Preece and colleagues. Imagine driving with a keyboard instead of a steering wheel! That seems like an absurd idea, but it serves to highlight how some tools can be so ill suited for a particular purpose given human information processing capabilities and their limitations. Keep in mind that this is just an example. Please don’t try to drive using your keyboard on the highway.
Imagine trying to drive a car using a computer keyboard. The four arrow keys are used for steering, the space bar for braking and the return key for accelerating. To indicate left you need to press the F1 key and to indicate right, the F2 key. To sound your horn you need to press the F3 key. To switch the headlights on you need to use the F4 key and the F5 key for the windshield wipers. Now imagine as you were driving along a road a ball is suddenly kicked in front of you. WHAT would you do? Bash the arrow keys and the space bar madly while pressing the F3 key?
This example serves to illustrate that perceptual, motoric and cognitive responses can’t possibly be effectively supported by such tools. This is a case where safety would be clearly compromised. Although the example is a bit extreme, there are many instances in which tools completely fail to exploit human capabilities and make tasks unnecessarily difficult.
“Imagine trying to drive a car using a computer keyboard. The four arrow keys are used for steering, the space bar for braking and the return key for accelerating. To indicate left you need to press the F1 key and to indicate right, the F2 key. To sound your horn you need to press the F3 key. To switch the headlights on you need to use the F4 key and the F5 key for the windshield wipers. Now imagine as you were driving along a road a ball is suddenly kicked in front of you. What would you do? Bash the arrow keys and the space bar madly while pressing the F3 key?”
Keyboard:
Steering wheel:
Quote: Preece, J. Rogers, Y. & Sharp, H. (2007).
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

7. Human Information Processing
Human information processing theory deals with how people receive, store, integrate, retrieve, and use information.
A basic HIP model recognizes 3 subsystems:
Perceptual system processing sensory information
Motor system controls action
Cognitive system provides the processing that connects the two systems
HIP model provides a basis from which to make predictions about human performance
Human information processing theory deals with how people receive, store, integrate, retrieve, and use information. The basic model recognizes 3 subsystems:
Perceptual system that processes incoming sensory information
Motor system controls action and physical behavior
Cognitive system provides the processing that connects the two systems
We are going to briefly deal with different dimensions of these subsystems.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

8. A Model of Human Information Processing
Perception
Working Memory
Long -
Term Memory
Sensory
Register
Hearing
Vision
Olfaction
Haptic
Response
Selection
Execution
Feedback
Attention
Resources
Perceptual Encoding
Central Processing
Responding
Thought
Decision Making
This is a simplified model of human information processing. We can characterize it as a multistage process that begins when our senses such as vision and hearing come into contact with stimuli in the world. A very small percentage of these stimuli are passed along for further processing. Our attention mechanisms determine what information requires a particular kind of response. For example, if you are driving and you suddenly see a ball in the middle of the road, you will immediately swerve without giving it any thought—it’s a simple reflex reaction. However, if you come across an that offers you a career opportunity, it will necessitate some deliberation and decision-making. Information from the senses is transferred to the different memory systems for either immediate processing or for long-term action.
It’s important to keep in mind that this diagram is an idealization and a simplification of a very complex system. But this will suffice for the purposes of discussion. To reiterate, it is useful to distinguish 3 subsystems including perceptual mechanisms, central processing that gives meaning to perceptual input and a subsystem that supports action.
(Stillings, et al., 1995.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

9. Usability and Human Factors Cognition and Human Performance Lecture a
Perception
Complex dynamic process
Perception: Attach meaning to sensory inputs
Maps incoming sensory information into a mental representation stored in long term memory
Mapping proceeds by processes that often occur concurrently
Bottom-up feature analysis
Top-down processing
Ability to respond consistently to certain combination of features because of learned associations
Perception is a set of dynamic processes that attach meaning to sensory inputs. It maps incoming sensory information into a mental representation stored in long term memory. A simple example is that when you are in a public space, you will react to a face you know whereas you may not even perceive other faces that don’t attract your attention.
Bottom up processing refers to the analysis of features that one would try to integrate into a whole. Top-down processing involves using knowledge stored in long-term memory to recognize objects, people and events.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

10. Gestalt Organization Principles
Perceptual organizing principle:
How we perceive structure in our environment
Proximity
Containment
Similarity
Closure
Continuation
Basic principles with clear implications for web design
A nice illustration of the implications for design and web design, in particular, are seen in gestalt perceptual organizing principles. These predict the way the human perceptual system will react to objects we see on a display.
I am going to go through a series of slides to illustrate some of these principles.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

11. Usability and Human Factors Cognition and Human Performance Lecture a
No Clear Grouping
This illustration shows 16 blocks with no guidance on how to perceptually organize them. It makes it rather hard for viewers to follow.
(Goldstein, 2009.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

12. Usability and Human Factors Cognition and Human Performance Lecture a
Proximity
The proximity principle suggests that we perceptually group objects by their closeness to one another. We can clearly see 4 groups in this picture.
(Goldstein, 2009.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

13. Usability and Human Factors Cognition and Human Performance Lecture a
Containment
If you put things inside of boxes or frames or inside borders, that suggests a way to group them.
(Goldstein, 2009.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

14. Usability and Human Factors Cognition and Human Performance Lecture a
Similarity
Color can be used to effectively partition a space and let us know that they designate a type or category.
(Goldstein, 2009.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

15. Usability and Human Factors Cognition and Human Performance Lecture a
Closure
Humans have a natural tendency to see closure if a grouping suggests it. For example, the letters B and R are often handwritten in a way that the semi-circles in each letter do not appear to be fully closed. But people will typically not have any problem recognizing the letters for what they are. Of course, bad handwriting perceptually degrades them and makes that task all that much harder.
(Goldstein, 2009.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

16. Usability and Human Factors Cognition and Human Performance Lecture a
Anyone’s Guess
Arbitrary arrangements of objects provide no clues to categorical grouping. Certain MySpace pages have no clear boundaries between sections with media players, photos and textual information sections bleeding into one another. A lack of categorical grouping makes it more difficult for us to make sense of a web page and/or learn to use a system productively.
(Goldstein, 2009.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

17. Airport Status Example
Was the delay Boston
or Westchester?
Public displays are an important source of information. Let’s imagine, you are waiting to go to Boston and you want to know if your flight is delayed. You would check out a flight status board.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

18. Usability and Human Factors Cognition and Human Performance Lecture a
Flight Status Board 1
Newark
Philadelphia
On time
On time
JFK
Boston
On time
Delay
Schedule 1 offers you a neatly laid out screen, but it all looks the same. You need to serially scan the board to find the information you need.
La Guardia
Westchester
On time
On time
Newberg
Long Island
On time
On time
(Kaufman and Starren, 2010.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

19. Usability and Human Factors Cognition and Human Performance Lecture a
Simple Improvement 1
Newark
Philadelphia
On time
On time
JFK
Boston
On time
Delay
The color red offers a simple improvement because one’s eye is immediately drawn to the delay.
La Guardia
Westchester
On time
On time
Newberg
Long Island
On time
On time
(Kaufman and Starren, 2010.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

20. Usability and Human Factors Cognition and Human Performance Lecture a
Simple Improvement 2
Newark On Time
Philadelphia - On Time
JFK On Time
Boston Delay
This slide simplifies the display and one may perceptually pickup the information even more effortlessly because you are presented with an integrated whole. The perceptual and cognitive systems have less work to do.
La Guardia - On Time
Westchester - On Time
Newberg On Time
Long Island - On Time
(Kaufman and Starren, 2010.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

21. Usability and Human Factors Cognition and Human Performance Lecture a
Human Attention
Selective Mechanism
Resource needed for information processing
Limited
Sharable
Flexible
Attention mechanisms are used to allocate resources to the kind of work the cognitive system has to do. Attention resources are limited; you can share them across tasks if necessary. They are flexible. For example, multitasking requires divided attention.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

22. Selective Attention
Ability to ignore extraneous information and focus on relevant inputs
Performance typically declines as the number of sources of information increases
Humans can only process information at a finite rate
Information overload results in errors
If you paid attention or equal attention to everything that your senses perceive, you would simply be overwhelmed. In fact, that can happen in stressful work environments. Humans can only process so much information over a period of time and overload can result in errors. The four points on the slides describe both the potential of humans to employ attentional mechanisms productively and the downside of being confronted with a situation in which demands exceed capacity.
“Attention” is a topic of importance in human factors work. Information overload is not an uncommon problem in a busy workplace and can be clearly evidenced in overcrowded hospital settings.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

23. The Case of Bird Watching
If you’ve ever had a bird watching experience and you go out with a friend who is an experienced birdwatcher, he or she will spot things that you will not be able to see right away. You may have to serially scan through thick leaves before you see anything that resembles a bird. On the other hand, your bird savvy friend, can immediately attend to cues that are important (e.g., very slight movements, changes in light) and will perceive signal to your noise.
(Kaufman, 2011.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

24. Computer Provider Order Entry Systems
This is the Computer-order Entry screen that we have seen in previous lectures. Notice that there is a very lengthy pick list of tests. If you have considerable familiarity with the Glasgow coma scale, then you may be able to more readily perceive it on this alphabetized list. If you’re not, you will have much more work to do to find what you are looking for. This picture exemplifies a very complex screen that supports an enormous range of functions. The display is cluttered and it can be challenging to make selections given the density of information.
(Horsky, J., Kaufman, D.R., Oppenheim, M.I. & Patel, V.L., 2003.)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

25. Representational Effect
External representations such as images, graphs, tables, icons, audible sounds, written symbols are instruments of thinking
Representational Effects: Different representations of a common abstract structure (same meaning) can significantly affect performance
Compare 37 x 93 and XXXVII x XCIII
There is a widely documented and important phenomenon known as the Representation Effect. You can use different symbols to express the same idea. For example, if you want to communicate population growth in the state of Florida over the past 50 years, you can do so in words, in a tabular format or graphically in many different ways. How it will be perceived depends in part on your purpose and on your audience. But these different representations of the same ideas or same information will have significant perceptual and cognitive effects.
Compare multiplying 37 x 93 and then consider using Roman numerals XXXVII x XCIII. Even if you are very well versed in Roman numerals, chances are you will still find it difficult to multiply these numbers that way.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

26. Usability and Human Factors Cognition and Human Performance Lecture a
Digital vs. Analog
We are all acquainted with both digital and analog clocks. If you want to know the time in a hurry, then you can quickly perceive the numbers on a digital clock. On the other hand, if you want to find how much time you’ve spent surfing the web, the analog clock provides more resources for quickly perceiving that 12 minutes have elapsed since you started surfing. On a digital clock, such an inference involves a calculation and even if it is a simple one, will necessitate the use of more resources.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

27. Glucose and Blood Pressure Pictures
Tables can support quick and easy lookup and are a compact and efficient representational device. However, a particular external representation is likely to be effective for some population of users and not others. For example, reading a table requires a certain level of numeracy that is beyond the capabilities of certain patients with a very basic education. Numeracy refers to quantitative literacy. In our research, we found that some patients who had no trouble reading out their blood pressure on their monitors could not understand the same values when expressed in a tabular form on a computer screen.
The basic point is that the same information represented differently can have very significant effects for different populations. The representational effect has significant design implications.
BP value from a patient health record table. An unfamiliar representation
BP value as expressed on a BP monitor in a format that is familiar to patients
(Kaufman, D.R., Patel, V.L., Hilliman, C., et. Al )
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

28. Usability and Human Factors Cognition and Human Performance Lecture a
Risk Representations (Ancker et al, 2009) Random Stick Figure Sequential Stick Figure
Jessica Ancker has conducted research pertaining to communicating risks to patients. For example if you are 50 years and smoke, what is your risk of having a heart attack over the next 10 years? How does that differ from someone of the same age who does not smoke? There are 2 graphics. In the first one, dark blue stick figures who represent individuals with disease are randomly dispersed throughout a population. In the second graph on the right, the dark blue stick figures are sequentially presented at the bottom of the screen. Although they represent the same probability, people are likely to interpret them differently. Dr. Ancker has experimented with different kinds of dynamic and static representations to determine which ones work best for different populations.
Dark blue figure represents individual with disease
(Ancker JS, Chan C, Kukafka R )
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

29. Cognition and Human Performance Summary – Lecture a
Introduced a cognitive engineering approach
Characterized a model of information processing
Perception and attention in relation to human performance
Representational effects
Next lecture: human memory and different ways of characterizing knowledge
In this lecture, we explored different facets of human cognition in the context of a cognitive engineering approach. We introduced a basic model of human information processing and introduced issues pertaining to perception and attention. We also characterized how different kinds of representations can significantly affect the kinds of inferences that people make. The same information can be expressed in very different ways. In the next section, we will discuss issues relating to human memory and characterize different ways to characterize knowledge.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

30. Cognition and Human Performance References – Lecture a
Goldstein, E. Bruce (2009). "Perceiving Objects and Scenes § The Gestalt Approach to Object Perception". Sensation and perception (8th ed.). Cengage Learning. ISBN 
Preece, J. Rogers, Y. & Sharp, H. (2007) Interaction Design: Beyond Human-Computer Interaction. 2nd Edition. New York, NY: John Wiley & Sons. P. 92
Images
Slide 6: Keyboard:
Steering wheel:
Slide 8: Stillings, N. A., Weisler, S. E., Chase, C. H., Feinstein, M. H., Garfield, J. L., & Rissland, E. L. (1995). Cognitive science: An introduction (2nd ed.). Cambridge, MA: MIT Press.
Slide 11-16: Goldstein, E. Bruce (2009). "Perceiving Objects and Scenes § The Gestalt Approach to Object Perception". Sensation and perception (8th ed.). Cengage Learning. 
Slide 18-20: Kaufman, D., & Starren, J. (2010). Flight status. Department of Biomedical Informatics, Columbia University Medical Center.
Slide 23: Kaufman, D. (2010). Personal image of bird watching. Department of Biomedical Informatics, Columbia University Medical Center.
Slide 24: Horsky, J., Kaufman, D.R., Oppenheim, M.I. & Patel, V.L. (2003). A framework for analyzing the cognitive complexity of computer-assisted clinical ordering. Journal of Biomedical Informatics, 36, 4-22.
No Audio
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a

31. Cognition and Human Performance References – Lecture a
Images (cont.)
Slide 26: Retrieved on August 10th, 2010 from
Slide 27: Kaufman, D.R., Patel, V.L., Hilliman, C., Morin, P.C., Pevzner, J, Weinstock, Goland, R. Shea, S. & Starren, J. (2003). Usability in the real world: Assessing medical information technologies in patients’ homes. Journal of Biomedical Informatics, 36,
Slide 28: Ancker JS, Chan C, Kukafka R. (2009). Interactive graphics to demonstrate health risks: formative development and qualitative evaluation. Journal of Health Communication; 14:
No Audio
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors Cognition and Human Performance Lecture a