1. Usability and Human Factors
Cognition and Human Performance
Welcome to Usability and Human Factors, Cognition and Human Performance. This is lecture a, in which we will first discuss a theoretical introduction to the study of human computer interaction and cognition. Cognition is defined by the Merriam-Webster dictionary as “conscious mental activities- the activities of thinking, understanding, learning, and remembering.” Most emphasis will be placed on information processing models in the context of a cognitive engineering approach.
We will be talking about classical models of human information processing, including perception, attention, memory and cognition. The implications for design will be illustrated in the context of Gestalt organizing principles of perception.
We will also introduce the concept of mental models, and explain their role in understanding interactive behavior.
The last topic will focus on distributed cognition, which represents a new way to understand human computer interaction and challenges some concepts that are central to the conventional information processing models.
[Reference:
Lecture a
This material (Comp 15 Unit 3) was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC This material was updated by The University of Texas Health Science Center at Houston under Award Number 90WT0006.
This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit
Health IT Workforce Curriculum Version 4.0

2. Cognition and Human Performance Lecture a – Learning Objectives
Describe the impact of different kinds of representation on cognition as it applies to human computer interaction and web design (Lecture a)
Describe how humans process information and obtain skills (Lecture a)
Describe the Gestalt principles of perception and their relevance to human computer interaction and cognitive theory (Lecture a)
Describe the processes of memory and their relationship to web-design (Lecture b)
Describe the cognitive constructs for mental representation (Lecture b)
Describe the cognitive constructs for mental representation (Lecture c)
Explain how cognition and human performance models should inform iterative design processes (Lecture c)
The unit objectives for this lesson are:
1) Describe the impact of different kinds of representation on cognition as it applies to human computer interaction and web design;
2) Describe how humans process information and obtain skills;
3) Describe the Gestalt principles of perception and their relevance to human computer interaction and cognitive theory

3. Cognition and Human Computer Interaction (HCI)
HCI is intersection between psychology/ social sciences and computer sciences/ technology
HCI seeks to understand and support human beings interacting with technology
Study of cognition and information processing is increasingly viewed as central to advances in HCI
Drawing on basic science research
Applied cognitive HCI work
Given that this is a course on usability and human factors, why should we be concerned with human cognition or theories that largely arise out of the psychological literature? There are two answers to that question.
The first is that the computing experience is a deeply human endeavor that engages the cognitive and perceptual systems.
The second is that the disciplines of HCI and Human Factors draw heavily on both basic and applied research.
In this particular set of lectures, we will draw on cognitive theories in view to understand the way it affects the computing experience in healthcare contexts.

4. HCI: Cognitive Engineering Approach
Principles, methods and tools to assess and guide design of computerized systems to support human performance
Focus on attention, perception, memory, comprehension, problem solving, and decision making
Analysis of cognitive tasks and processing constraints imposed by the human information processing system
The cognitive engineering approach is one of the most influential in HCI and Human Factors. Don Norman, who we discussed previously, is one of the pioneers of this approach. Cognitive engineering places a strong emphasis on cognitive, perceptual and motoric responses in the context of interacting with any system. The approach also embraces a cognitive task analytic perspective and endeavors to understand the ways in which human attributes (e.g., perceptual abilities) and limitations (e.g., limits on working memory) impact performance.

5. Cognitive Theory and HCI
Informative
Useful research findings
Predictive
Providing tools to model user behavior
Prescriptive
Providing advice for design or evaluation
Explanatory
Accounting for user behavior
Generative
Provide design dimensions and constructs to inform design and selection of external representations
What can cognitive theory do for you? Or better stated, what can it do for HCI? Well, it can inform our understanding, and it can be used to predict or explain end-user behavior in both general and more precise ways. For example, a general way would be a prediction that a given screen would exert a heavy memory load on any user because of its clutter and complexity. A more precise prediction might involve a model of a novice user performing a particular task such as searching Google for treatment for migraines. Cognitive theory can also seed novel design concepts that may be a better match with users’ models of the world.
Let’s consider a design that pays absolutely no attention to human capabilities.

6. Driving with a Computer Keyboard
“Imagine trying to drive a car using a computer keyboard. The four arrow keys are used for steering, the space bar for braking and the return key for accelerating. To indicate left you need to press the F1 key and to indicate right, the F2 key. To sound your horn you need to press the F3 key. To switch the headlights on you need to use the F4 key and the F5 key for the windshield wipers. Now imagine as you were driving along a road a ball is suddenly kicked in front of you. What would you do? Bash the arrow keys and the space bar madly while pressing the F3 key?”
The example was drawn from a text by Jenny Preece and colleagues. Imagine driving with a keyboard instead of a steering wheel! That seems like an absurd idea, but it serves to highlight how some tools can be so poorly suited for a particular purpose given human information processing capabilities and their limitations. Keep in mind that this is just an example. Please don’t try to drive using your keyboard on the highway.
Imagine trying to drive a car using a computer keyboard. The four arrow keys are used for steering, the space bar for braking and the return key for accelerating. To indicate left you need to press the F1 key and to indicate right, the F2 key. To sound your horn you need to press the F3 key. To switch the headlights on you need to use the F4 key and the F5 key for the windshield wipers. Now imagine as you were driving along a road a ball is suddenly kicked in front of you. WHAT would you do? Bash the arrow keys and the space bar madly while pressing the F3 key?
This example serves to illustrate that perceptual, motorist and cognitive responses can’t possibly be effectively supported by such tools. This is a case where safety would be clearly compromised. Although the example is a bit extreme, there are many instances in which tools completely fail to exploit human capabilities and make tasks unnecessarily difficult.
Jessicafm, CC BY 2.0
Yvettemn, CC BY-NC-SA 2.0
Preece, J. Rogers, Y. & Sharp, H. (2007)

7. Human Information Processing
Human information processing theory deals with how people receive, store, integrate, retrieve, and use information
A basic HIP model recognizes 3 subsystems
HIP model provides a basis from which to make predictions about human performance
Human information processing theory deals with how people receive, store, integrate, retrieve, and use information.
The basic model recognizes 3 subsystems:
Perceptual system that processes incoming sensory information
Motor system controls action and physical behavior
Cognitive system provides the processing that connects the two systems
We are going to briefly deal with different dimensions of these subsystems.

8. A Model of Human Information Processing
This is a simplified model of human information processing. We can characterize it as a multistage process that begins when our senses such as vision and hearing come into contact with stimuli in the world. A very small percentage of these stimuli are passed along for further processing. Our attention mechanisms determine what information requires a particular kind of response. For example, if you are driving and you suddenly see a ball in the middle of the road, you will immediately swerve without giving it any thought—it’s a simple reflex reaction. However, if you come across an that offers you a career opportunity, it will require some deliberation and decision-making. Information from the senses is transferred to the different memory systems for either immediate processing or for long-term action.
It’s important to keep in mind that this diagram is an idealization and a simplification of a very complex system. But this will suffice for the purposes of discussion. To reiterate, it is useful to distinguish 3 subsystems including perceptual mechanisms, central processing that gives meaning to perceptual input and a subsystem that supports action.
(Stillings, et al., 1995.)

9. Perception Complex dynamic process
Perception: Attach meaning to sensory inputs
Maps incoming sensory information into a mental representation stored in long term memory
Mapping proceeds by processes that often occur concurrently
Bottom-up feature analysis
Top-down processing
Ability to respond consistently to certain combination of features because of learned associations
Perception is a set of dynamic processes that attach meaning to sensory inputs. It maps incoming sensory information into a mental representation stored in long term memory. A simple example is that when you are in a public space, you will react to a face you know whereas you may not even perceive other faces that don’t attract your attention.
“Bottom up processing” refers to the analysis of features that one would try to integrate into a whole. “Top-down processing” involves using knowledge stored in long-term memory to recognize objects, people and events.

10. Gestalt Organization Principles
Perceptual organizing principle:
How we perceive structure in our environment
Proximity
Containment
Similarity
Closure
Continuation
Basic principles with clear implications for web design
("Definition of GESTALT", n.d.)
A nice illustration of the implications for design and web design, in particular, are seen in gestalt perceptual organizing principles. Gestalt, according to the Merriam Webster dictionary, means “something that is made of many parts and yet is somehow more than or different from the combination of its parts; broadly : the general quality or character of something.” Gestalt perceptual organizing principles predict the way the human perceptual system will react to objects we see on a display.
Let’s go through a series of slides to illustrate some of these principles.

11. No Clear Grouping (Goldstein, 2009.)
This illustration shows 16 blocks with no guidance on how to perceptually organize them. It makes it rather hard for viewers to follow.
(Goldstein, 2009.)

12. Proximity (Goldstein, 2009.)
The proximity principle suggests that we perceptually group objects by their closeness to one another. We can clearly see 4 groups in this picture.
(Goldstein, 2009.)

13. Containment (Goldstein, 2009.)
If you put things inside of boxes or frames or inside borders, that suggests a way to group them.
(Goldstein, 2009.)

14. Similarity (Goldstein, 2009.)
Color can be used to effectively partition a space and lets us know that they designate a type or category.
(Goldstein, 2009.)

15. Closure
Humans have a natural tendency to see closure if a grouping suggests it. For example, the letters B and R are often handwritten in a way that the semi-circles in each letter do not appear to be fully closed. But people will typically not have any problem recognizing the letters for what they are. Of course, bad handwriting perceptually degrades them and makes that task all that much harder.
(Goldstein, 2009.)

16. Anyone’s Guess (Goldstein, 2009.)
Arbitrary arrangements of objects provide no clues to categorical grouping. A lack of categorical grouping makes it more difficult for us to make sense of a web page and/or learn to use a system productively.
(Goldstein, 2009.)

17. Airport Status Example
Was the delay Boston
or Westchester?
Public displays are an important source of information. Let’s imagine that you are waiting to go to Boston and you want to know if your flight is delayed. You would check out a flight status board.

18. Flight Status Board 1 (Kaufman and Starren, 2010.)
Schedule 1 offers you a neatly laid out screen, but it all looks the same. You need to serially scan the board – that is, examine each entry one at a time - to find the information you need.
(Kaufman and Starren, 2010.)

19. Simple Improvement 1 (Kaufman and Starren, 2010.)
The color red offers a simple improvement because one’s eye is immediately drawn to the delay.
(Kaufman and Starren, 2010.)

20. Simple Improvement 2 (Kaufman and Starren, 2010.)
This slide simplifies the display and you can perceptually pick up the information even more effortlessly, because you are presented with an integrated whole. The perceptual and cognitive systems have less work to do.
(Kaufman and Starren, 2010.)

21. Human Attention Selective Mechanism
Resource needed for information processing
Limited
Sharable
Flexible
Attention mechanisms are used to allocate resources to the kind of work the cognitive system has to do. Attention resources are limited; you can share them across tasks if necessary. They are flexible. For example, multitasking requires divided attention.

22. Selective Attention
Ability to ignore extraneous information and focus on relevant inputs
Performance typically declines as the number of sources of information increases
Humans can only process information at a finite rate
Information overload results in errors
If you paid attention or equal attention to everything that your senses perceive, you would simply be overwhelmed. In fact, that can happen in stressful work environments. Humans can only process so much information over a period of time and overload can result in errors. The four points on the slides describe both the potential of humans to employ attentional mechanisms productively and the downside of being confronted with a situation in which demands exceed capacity.
“Attention” is a topic of importance in human factors work. Information overload is not an uncommon problem in a busy workplace and can be clearly evidenced in overcrowded hospital settings.

23. The Case of Bird Watching
If you’ve ever had a bird watching experience and you go out with a friend who is an experienced birdwatcher, he or she will spot things that you will not be able to see right away. You may have to serially scan through thick leaves before you see anything that resembles a bird. On the other hand, your bird savvy friend, can immediately attend to cues that are important (e.g., very slight movements, changes in light) and will perceive signal to your noise.
(Kaufman, 2011.)

24. Computerized Provider Order Entry Systems
This is the computerized provider order entry screen that we have seen in previous lectures. Notice that there is a very lengthy pick list of tests. If you have considerable familiarity with the Glasgow coma scale, then you may be able to more readily perceive it on this alphabetized list. If you’re not, you will have much more work to do to find what you are looking for. This picture exemplifies a very complex screen that supports an enormous range of functions. The display is cluttered and it can be challenging to make selections given the density of information.
(Horsky, J., Kaufman, D.R., Oppenheim, M.I. & Patel, V.L., 2003.)

25. Representational Effect
Representational Effects: Different representations of a common abstract structure (same meaning) can significantly affect performance
Compare 37 x 93 and XXXVII x XCIII
External representations such as images, graphs, tables, icons, audible sounds, written symbols are instruments of thinking
There is a widely documented and important phenomenon known as the Representation Effect. You can use different symbols to express the same idea. For example, if you want to communicate population growth in the state of Florida over the past 50 years, you can do so in words, in a tabular format or graphically in many different ways. How it will be perceived depends in part on your purpose and on your audience. But these different representations of the same ideas or same information will have significant perceptual and cognitive effects.
Compare multiplying 37 x 93 and then consider using Roman numerals XXXVII x XCIII. Even if you are very well versed in Roman numerals, chances are you will still find it difficult to multiply these numbers that way.
Zhang, J., & Norman, D. A. (1995).

26. Digital vs. Analog Larry & Teddy Page, 2008, CC BY 2.0
We are all acquainted with both digital and analog clocks. If you want to know the time in a hurry, then you can quickly perceive the numbers on a digital clock. On the other hand, if you want to find how much time you’ve spent surfing the web, the analog clock provides more resources for quickly perceiving that 12 minutes have elapsed since you started surfing. On a digital clock, such an inference involves a calculation and even if it is a simple one, will necessitate the use of more resources.
Larry & Teddy Page, 2008, CC BY 2.0
Stuartpilbrow, 2010, CC BY-SA 2.0

27. Glucose and Blood Pressure Pictures
Tables can support quick and easy lookup and are a compact and efficient representational device. However, a particular external representation is likely to be effective for some population of users and not others. For example, reading a table requires a certain level of numeracy that is beyond the capabilities of certain patients with a very basic education. Numeracy refers to quantitative literacy. Research has shown that some patients who had no trouble reading out their blood pressure on their monitors could not understand the same values when expressed in a tabular form on a computer screen.
The basic point is that the same information represented differently can have very significant effects for different populations. The representational effect has significant design implications.
(Kaufman, D.R., Patel, V.L., Hilliman, C., et. Al )

28. Risk Representations (Ancker et al, 2009)
Jessica Ancker has conducted research pertaining to communicating risks to patients. For example, if you are 50 years old and smoke, what is your risk of having a heart attack over the next 10 years? How does that differ from someone of the same age who does not smoke? Let’s look at these two graphics. In the first one, dark blue stick figures who represent individuals with disease are randomly dispersed throughout a population. In the second graph on the right, the dark blue stick figures are sequentially presented at the bottom of the screen. Although they represent the same probability, people are likely to interpret them differently. Dr. Ancker has experimented with different kinds of dynamic and static representations to determine which ones work best for different populations.
(Ancker JS, Chan C, Kukafka R )

29. Cognition and Human Performance Summary – Lecture a
Introduced a cognitive engineering approach
Characterized a model of information processing
Perception and attention in relation to human performance
Representational effects
Next lecture: human memory and different ways of characterizing knowledge
This concludes lecture a of Usability and Human Factors, Cognition and Human Performance. In this lecture, we explored different facets of human cognition in the context of a cognitive engineering approach. We introduced a basic model of human information processing and introduced issues pertaining to perception and attention. We also characterized how different kinds of representations can significantly affect the kinds of inferences that people make. The same information can be expressed in very different ways. In the next section, we will discuss issues relating to human memory and characterize different ways to characterize knowledge.

30. Cognition and Human Performance References – Lecture a
Definition of GESTALT. Merriam-webster.com. Retrieved 20 June 2016, from
Goldstein, E. Bruce (2009). "Perceiving Objects and Scenes § The Gestalt Approach to Object Perception". Sensation and perception (8th ed.). Cengage Learning. ISBN 
Preece, J. Rogers, Y. & Sharp, H. (2007) Interaction Design: Beyond Human-Computer Interaction. 2nd Edition. New York, NY: John Wiley & Sons. P. 92
Zhang, J., & Norman, D. A. (1995). A representational analysis of numeration systems. Cognition, 57(3),
Images
Slide 6: Jessica FM. Steering wheel. [Online Image].Retrieved on June 20th, 2016 from
Slide 6: Yvettemn. Keyboard. [Online Image].Retrieved on June 20th, 2016 from
Slide 8: Stillings, N. A., Weisler, S. E., Chase, C. H., Feinstein, M. H., Garfield, J. L., & Rissland, E. L. (1995). Cognitive science: An introduction (2nd ed.). Cambridge, MA: MIT Press.
Slide 11-16: Goldstein, E. Bruce (2009). "Perceiving Objects and Scenes § The Gestalt Approach to Object Perception". Sensation and perception (8th ed.). Cengage Learning. 
No Audio.

31. Cognition and Human Performance References – Lecture a (Cont’d – 1)
Images
Slide 18-20: Kaufman, D., & Starren, J. (2010). Flight status. Department of Biomedical Informatics, Columbia University Medical Center.
Slide 23: Kaufman, D. (2010). Personal image of bird watching. Department of Biomedical Informatics, Columbia University Medical Center.
Slide 24: Horsky, J., Kaufman, D.R., Oppenheim, M.I. & Patel, V.L. (2003). A framework for analyzing the cognitive complexity of computer-assisted clinical ordering. Journal of Biomedical Informatics, 36, 4-22.
Slide 26: Larry & Teddy Page. (2008). 11:29 [Online Image].Retrieved on June 20th, 2016 from
Slide 26: Stuartpilbrow. (2010). 002/365 2nd January [Online Image].Retrieved on June 20th, 2016 from
Slide 27: Kaufman, D.R., Patel, V.L., Hilliman, C., Morin, P.C., Pevzner, J, Weinstock, Goland, R. Shea, S. & Starren, J. (2003). Usability in the real world: Assessing medical information technologies in patients’ homes. Journal of Biomedical Informatics, 36,
Slide 28: Ancker JS, Chan C, Kukafka R. (2009). Interactive graphics to demonstrate health risks: formative development and qualitative evaluation. Journal of Health Communication; 14:
No Audio.

32. Usability and Human Factors Cognition and Human Performance Lecture a
This material was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC This material was updated by The University of Texas Health Science Center at Houston under Award Number 90WT0006.
No Audio.
Health IT Workforce Curriculum Version 4.0