1. Engineering Psychology and Human Performance

2. Human Information Processing
Knowledge of physical performance (i.e. speed and accuracy) is important in understanding a task (or the effects upon the task from an ergonomics intervention program).
But such knowledge is not sufficient to understand WHY performance changed from a human operator perspective.

3. A MODEL for Human Information Processing
Sensory Processing
Perception
Memory and Cognition
Response Selection and Execution
Feedback
Attention

4. Sensory Processing
Information and events in the environment must gain access to the brain.
Visual, Auditory, Tactile, Smell
All sensory systems have an associated short-term sensory store
Temporary mechanism for prolonging the representation of the raw stimulus (length depends on the type of stimulus)
Can be confounded if the operator is distracted

5. Perception
Raw sensory data relayed to the brain must be interpreted (given meaning) through a stage of perception.
Decode the meaning of the signal
Generally proceeds rapidly and automatically
Driven by sensory input (rapid; bottom-up processing) or long-term memory (top-down processing)

6. Memory and Cognition
Cognitive operations generally require more “processing” time.
Rehearsal, reasoning or image transformation is carried out by working memory.
“Plans”; “Diagnose” ~ these are vulnerable to disruption when attentional resources are diverted to other mental activities.
Material that is rehearsed in working memory can get access to long-term memory – learning takes place.

7. Response Selection and Execution
The understanding of a situation, achieved through perception and augmented by cognitive transformations will trigger the selection of a response.
The selection of a response must precede the execution (requiring motor effort – which has it’s own qualities for success).

8. Feedback
Actions are directly sensed by the operator OR influence the system within which the operator functions.
Flow of information can be initiated at any point and is continuous.

9. Attention
Selective Attention
Focused Attention
Divided Attention

10. A model of human information processing stages

11. Signal Detection

12. Overview:
The process of detection may involve many states of categorization:
Operator to decide whether a “signal” is present.
Operator to attend to several signals

13. Signal Detection Paradigm ~ The Four Outcomes of Signal Detection Theory

14. Setting the Response Criterion
Hypothetical distributions underlying signal detection theory

15. Consider the circumstances!
Radiologist examining X-Rays of referred cancer patient.
More likely to say “yes” but increases the chances of a “false alarm” - CONSERVATIVE

16. Consider the circumstances!
Plant operator warned about unnecessary shutdowns and the financial implications
More likely to say “no” but increases the chances of a “missed event” - RISKY

17. Chances (probability) of false alarms or misses increases!
For what ever reasons:
When signal and noise have similar characteristics
Operator deficiencies (training, fatigue, stress)
Chances (probability) of false alarms or misses increases!
MISSES
FALSE ALARMS

19. The Probability of Signal Detection Theory
Signal detection theory is able to prescribe where the optimum beta (β) should fall:
The likelihood of observing a signal
The COSTS/BENEFITS (payoffs) of the four possible outcomes
In almost all circumstances, signal and noise will overlap.

20. Attention in Perception and Display Space
The limitations of human attention represent one of the most formidable bottlenecks in human information processing.
I failed to notice the words of a speaker because I was looking out the window
We had so many tasks to perform that some were omitted

21. SELECTIVE ATTENTION Begins with the EYE and VISUAL SAMPLING.
Seek for information/search for a target
We learn a lot by assessing visual scanning behaviour

22. SELECTIVE ATTENTION The FOVEA has about a 2 degrees of visual angle.
To keep objects in the foveal vision ~
Pursuit Movement ~ eye follows a target moving across the visual field – “smooth motion”
Saccadic Movement – jump from one stationary point to another – “jerky movements”

23. SELECTIVE ATTENTION Supervisory Control Sampling
In the aircraft cockpit or the process control console, many information sources have to be sampled periodically.
Often divide the environmental stimulus into CHANNELS along which critical EVENTS may periodically occur.

24. SELECTIVE ATTENTION
Environmental Sampling is guided by the expected cost that results when an event is missed.
The probability of missing an event is directly related to the event frequency and uncertainty.

25. SELECTIVE ATTENTION Visual Scanning
Mental Model Guides
Adjustment to Event Rate
Sampling Affected by Arrangement
Memory Imperfect; Sampling Imperfect
Preview Helps
Processing Strategies – Cognitive Tunneling

26. VISUAL SCANNING Mental Model Guides
The mental model consists of a set of expectancies of how frequently and when events will occur.
Correlation of events amongst channels.
As expertise develops, mental models become refined as do the sampling strategies.

27. VISUAL SCANNING Adjustment to Event Rate
People learn to sample channels with higher event rates more frequently (and lower event rates less frequently)

28. VISUAL SCANNING Sampling Affected by Arrangement
Operators like to scan horizontally rather than diagonally.
It is proposed that operators use simplifying rules and HEURISTICS based on channel arrangement to decrease attentional demands – this leads to systematic biases in performance.
Arrangements matter
* Heuristics – creating a model as a working hypothesis to a goal or solution

29. VISUAL SCANNING Memory Imperfect; Sampling Imperfect
People tend to sample information sources more often than they would need to if their short-term memory was better. Two consequences:
“Oversampling” occurs of channels with low event rates
People forget to sample channels with low event rates
Suggests the use of “sampling reminders”

30. VISUAL SCANNING Preview Helps
When people are given a preview of scheduled events that are likely to occur in the future, sampling a channel switching become somewhat more optimal.
Now an “external model” is featured – somewhat else guiding attention.
If the number of channels increases too much, people fail to take advantage of the preview – because of the heavy memory load.

31. VISUAL SCANNING Processing Strategies – Cognitive Tunneling
Scanning behaviours may reflect the operator’s mental model of the environment. This includes biases in the operator’s strategies.
Fixation on feedback on an event can be a substantial waste of visual attention.

32. PARALLEL PROCESSING AND DIVIDED ATTENTION
Divided attention and parallel processing is often a good thing (air traffic controllers).
However, it is sometimes very difficult to narrow the focus of attention when needed and shut out unwanted inputs.
This failure occurs when divided attention becomes mandatory rather than optional.

33. Preattentive Processing and Perceptual Organization
Preattentive phase is carried out automatically and organizes the visual world into objects and groups of objects

34. Preattentive Processing and Perceptual Organization
Grouping together of similar items on a display.
Knowledge of where one is on the display (proximity, similarity, common fate, good continuation, closure) will allow an accurate guess of where others are located.

35. Preattentive Processing and Perceptual Organization
Because all items of an organized display must be processed together to reveal organization, such parallel processing is sometimes called GLOBAL or HOLISTIC processing, in contrast to LOCAL processing.

36. Global vs Local
Global Display Organization (top) allows for faster identification of Local abnormality.
Top display likely has reduced:
Attentional demands
Risk for missed event

37. The Proximity Compatibility Principle
Display Proximity – how close together two display components are in spatial terms
Processing Proximity – the extent to which two information sources are used within the same task (compare or integrate).

38. The Proximity Compatibility Principle
Close proximity will increase the possibility of parallel processing by moving both dimensions into foveal vision. It is even more likely when objects are integrated a dimensions of a single object.
Emergent features (colour coding,symmetry) can help in this respect.

39. Visualization of Complex Dynamic Information
i.e. “the use of computer-supported, interactive, visual representations of abstract data to amplify cognition“ (Card et al., 1999)
supports cognition by
Relief of resources for memory and information processing
Facilitation of information retrieval
Recognition of hidden patterns, tendencies etc.
Reasoning at a perceptual level („direct perception“)
Improving visual attention during monitoring / supervisory tasks
Informationszugangs- und nutzungskosten entstehen durch
Suche von verteilt dargestellten Informationen (Perzeption)
Zwischenspeichern der Ergebnisse (Gedächtnis !)
Mentale Integration von Einzelinformationen (Kognition) (bspw. bei Grenzwertbetrachtungen)
Übergeordnetes Gestaltungskonzept: Ökologische Schnittstellengestaltung (Vicente & Rasmussen):
Darstellung von Rahmenbedingungen der Systemumgebung
Ermöglichen von regelbasierter Informationsverarbeitung
Gestaltung einzelner Anzeigen: Kompatibilitätsprinzip der Nähe (Wickens & Carswell, 1995):
Erhöhung der Wahrnehmungs- (Anzeigen-) Nähe bei großer Aufgabennähe (insb. bei Integrationsaufgaben – SOLL vs. IST – Vergleich  regelbasiertes Entscheiden)
Zueinander gehörende Information nah positionieren oder geometrisch integrieren
Information so darstellen, wie sie benötigt werden (Integration von Wissen!)
Goal: Reduction of effort for information access and processing

40. Proximity Compatibility Principle (Wickens & Carswell, 1995): Object Integration and Emergent Features
Idea: Utilisation of human skills for pattern-recognition by means of graphical transformation of complex data
Emergent features: Mentale Verrechnungen werden durch Wahrnehmungen ersetzt
Emergent features bei Anzeigen können durch die spezifische Konstellation mehrerer einzelner Anzeigen zueinander herbeigeführt werden. Solche Merkmale sind z. B. Symmetrie, Ausrichtung, Parallelität, Merkmale also, die die einzelnen Anzeigen alleine nicht besitzen.
Eine komplexe mentale Integration der einzelnen Daten erübrigt sich bei Vorliegen dieser emergent features.
Komplexe kognitive Aktivitäten werden durch Wahrnehmungen ersetzt.
Andererseits sind die einzelnen Daten, falls erforderlich, auch immer noch einzeln ablesbar.
Eine einzelne Anzeige über den Systemzustand, wie z.B. eine Alarmanzeige, kann also eine solche Anzeige nicht ersetzen.
Die Bedeutung der emergent features besteht darin, dass aus diesen Merkmalen direkt Hinweise (cues) für die Aufgabe abgeleitet werden können, während andernfalls bei einer getrennten Darstellung der Information Verrechnungen oder Vergleiche von individuellen Anzeigenwerten erforderlich wären.
Ein weiteres Beispiel für eine solche Anzeige wären drei parallele Balkenanzeigen mit gleichem Niveau bei Normalzustand oder Zeiger mit gleicher Ausrichtung.
Wichtig bei solchen emergent features ist, dass diese auch Variablen repräsentieren, die von wesentlicher Bedeutung für die Systemführung sind.
Ein Nachteil von emergent features – besonders, wenn sie stark hervortreten - ist allerdings, dass die Aufmerksamkeit bei Fokussierung auf die einzelnen Elemente abgelenkt und deren Diskriminierbarkeit reduziert wird.
Verbindungen.
Bei mehreren z.B. horizontal angeordneten Rundinstrumenten können emergent features auch dadurch hervorgerufen werden, dass man die Zeiger im Normalfall alle horizontal anordnet und ggf. durch Linien miteinander verbindet..
Objektintegration.
Homogene Kodes können auch zu einem einzigen Objekt integriert werden, z.B. die Fläche und Form eines Rechtecks, die durch Höhe und Breite der parallelen Seiten gebildet werden. Weitere Beispiele: ein einzelner Punkt, dessen Position in einem Koordinatensystem festliegt. Die Linie, die mehrere Punkte in einem Diagramm miteinander verbindet. Der Verlauf der Linie gibt direkt Hinweise auf die Unterschiede zwischen den Punkten, der Trends wahrnehmen lässt und somit komplexe Integrationsprozesse erübrigt.
Objektintegration und parallele Informationsverarbeitung: Die Kombination von homogenen oder heterogenen Kodes für quantitative Merkmale in einem Objekt kann einen positiven Effekt haben, der vorteilhaft für alle Grade der Aufgabennähe ist: alle unterscheidbaren Attribute eines Objektes können parallel verarbeitet werden (Object File Theorie der Aufmerksamkeit).
Objektanzeigen oder konfigurierte Anzeigen können also unter Umständen so gestaltet werden, dass sie Vorteile für Integrationsaufgaben bieten und dabei auch Nachteile für fokussierte Aufgaben vermeiden.
“Normal States” are visualized by graphical attributes like
Symmetry,
Regular Alignment or
Parallelism
Deviations from Normal States generate
Asymmetry,
Irregular Alignment,
Skewness.
(Images: Wickens, 1992)

41. Integration of attributes
Comparison of Mental Transformations: Conventional vs. “Direct Perception“ Polar Displays
Actual value
Rule
ID-Crits memory)
Result
DST 50
ALT 164
ESM Down Beat …
> ISR
ISR = 30 NM
true SU
ALT  ALTciv
ALTciv = [5000;FL 250]
false
Bewertung von Luftzielen erfolgt regelbasiert anhand vordefinierter ID-Kriterien, wobei mehrere Einzelinformationen „fusioniert“ werden. Es handelt sich also um eine n-fach-Integrationsaufgabe, die der Operateur leisten muss.
Jeder Parameter trägt einen Ist-Wert und einen „Normbereich“, der unkritische Trackzustände kennzeichnet (Integrationsaufgabe)
Jeder Track besitzt einen Parametersatz, deren regelbasiert ausgewerteten Einzelwerte das Gesamturteil ermöglichen (Integrationsaufgabe)
Das Polardisplay erzeugt unter Rückgriff auf das Regelwissen
für den Einzelparameter Anzeigennähe zwischen Ist- und Normwert auf dem einzelnen Parameter-Strahl
Der Normbereich stellt hier das in die Anzeige integrierte Situationswissen dar.
für den gesamten Parametersatz des Tracks Anzeigennähe durch die Integration in ein Objekt.
Polardisplays erzeugen Emergent features, um die Informationsinterpretation auf die Wahrnehmungsebene zu verlagern:
Emergent features können durch spezifische Konstellation mehrerer einzelner Anzeigen zueinander herbeigeführt werden.
Solche Merkmale sind z. B. Symmetrie, Ausrichtung, Parallelität.
Emergent features können durch eine einzelne Anzeige alleine nicht erzeugt werden
komplexe mentale Integration der einzelnen Daten erübrigt sich
kognitive Aktivitäten werden durch Wahrnehmungen ersetzt.
einzelne Daten sind, falls erforderlich, auch immer noch einzeln ablesbar.
EM  EMciv/FF
EMciv/FF = {…}
false
n * Integration: actual value vs. setpoint
Integration of attributes
Conventional Display
Polar Display
Air track
uncritical
Air track
critical
Reference area = situation knowledge
Emergent features

42. PCP: Display Proximity ~ Task Proximity & Integration of Knowledge
Position and flight path have to be extracted by means of several flight control displays.
Mental transformation to outside view is required.
Physical proximity of altitude and vertical speed
conventional HUD
Position and flight path have to be extracted by means of several flight control displays.
Proximity of frequently used displays
Basic-T
Integrated Display
Direct perception of position and flight path

43. Air Picture of a Combined Air Operations Centre (CAOC)

44. Visualization:
Tracks & related events and actions
A-priori information: e.g. airways, corridors, (restricted) areas etc.
Interactive ENC-chart S-57 / VMAP L1
Event- and Threat-triggered Polar Displays
Usage by toolbox using well-known methaphers:
Scrolling
Cartographic zooming
Track search
Formation editing
Identification / Classification
Display settings predefined filters, styles, etc.

45. Tactical Situation Display
Graphical Representation of Information and Integration of A-priori Knowledge for AAW Situation Analysis
Track Detail Display
Tactical Situation Display
User Support Display
Polar Displays
Toolbox

46. Secondary Display for Detailed Information
Relief of working memory:
Parameter ranges / tendencies
Predefined tolerances for non-critical values
State classification (IFF/ESM)
History plots: Values / states along time / distance
Information on demand
Utilize pattern recognition:
Heterogeneous values cause emergent line characteristics
Parameter constellations (Alt/Spd)
Graphical representation of user support system “explains” results

47. Ecological Interface Design (EID) Vicente & Rasmussen (1992)
Approach: System errors result from violations of system operation constraints…
 Unexpected errors can be managed,
if the operator possesses a valid mental model of the system AND
if the system constraints are visualized referring to environmental factors influencing system behavior.
Constraints of system operations
Physical attributes temperature, radiation, illumination etc.
System functionalities different operational phases
Structural aspects of the work system flexible vs. determinated work system
Organizational structure hierarchies, safety culture
Working climate restrictions through rules, competitors
Fundamental concepts
Abstraction hierarchies
Visualization of constraints: Invariants, system structure and interrelations
Taxonomy of human information processing (Rasmussen, 1983)

48. EID: Abstraction Hierarchies
Abstraction hierarchies offer different views which differ regarding level of detail and resulting information elements.
Hierarchy levels:
Functional purpose: Purpose which should be accomplished by the system
Abstract function: Defined process / system structure regarding mass, energy, information or information flow.
Generalized function: Basis functions of system components.
Physical function: Functional characteristics and connections of system components.
Physical form: Structure and position of system components.
Each level represent another class of constraints for system operation.

49. Influencing Factors in Adaptive Systems
Situation
dynamic: Objects / prioritized tasks, system condition
static: Knowledge (Scenario, Situation, Competences)
 needs & possibilities of actions (“What should be done?”)  advice
Operator activity
Input sequences ( supervisory control)
Focus of attention
 actual operator activity (“Which task is in focus?”)  directed assistance
Operator Functional State (OFS)
Estimation of resource capacity and operator capability (normative / individual)
 need for user support  dynamic regulation (“Overload State?”) of input load

50. Eye-based Assessment of Operator Functional State in Adaptive Systems
eye blink frequency
fixation frequency
 Indicators for visual input load
Remote / Head-based Eye Tracking
Point of gaze, scan paths
Oculomotoric parameters
 Point of interest
 Mental workload, vigilance
Assessment techniques
Corneal-retinal potential (EOG)
Cornea reflex infrared ocularography
Marker-based tracking

51. ATTENTION IN THE AUDITORY MODALITY
Auditory Sense can take input from any direction and thus there is no analog to visual scanning as an index of selective attention.
Most auditory input is transient
Thus preattentive characteristics (organize mentally) is more difficult after the event has passed (sound stops)

52. Auditory Divided Attention
A general theory states that unattended channel of auditory input remains in preattentive short-term auditory store for about 3-6 seconds.
The contents of this store can be examined if a conscious switch of attention is made.

53. Auditory Divided Attention
Even unattended auditory channels may make content with long-term memory.
Car horn honking
Your name being spoken
These are not meaningless blobs of noise!

54. It is possible to think of an “auditory object’ as a sound with several dimensions – thus parallel processing can occur.
Auditory warning alerts have been designed to capitalize on our parallel processing ability using redundant dimensions:
Pitch
Timber
Interruption Rate
All in various combinations