1. Humans in safety critical systems

2. Estimated number of “human errors”
100
The diagram shows the attribution of “human errors” as causes, which may be different from the contribution of “human errors” to incidents / accidents. 90 80 70 60
% Human action attributed as cause 50 40 30 20 10
1960
1965
1970
1975
1980
1985
1990
1995

3. What is an “error”? Actual outcomes = intended outcomes
Correctly performed actions
Detected and recovered
Detected but tolerated
Incorrect actions
Overt effects
Detected but not recovered
Latent effects
Undetected

4. Humans and system safety
Technology centred-view
Human-centred view
Humans are a major source of failure. It is therefore desirable to design the human out of the system.
Humans are the main resource during unexpected events. It is therefore necessary to keep them in the system.
Automation permits the system to function when the limits of human capability have been reached.
The conditions for transition between automation and human control are often vague and context dependent.
Automation does not use humans effectively, but leaves them with tasks that cannot be automated - because they are too complex or too trivial.
Automation is cost-effective because it reduces the skill-requirements to the operators.
Conclusion: Humans are necessary to ensure safety

5. Lisanne Bainbridge (1987), “Ironies of automation”
The basic automation “philosophy” is that the human operator is unreliable and inefficient, and therefore should be eliminated from the system. 1
“Designer errors can be a major source of operating problems.”
“The designer, who tries to eliminate the operator, still leaves the operator to do the tasks which the designer cannot think how to automate.” 2
Lisanne Bainbridge (1987), “Ironies of automation”

6. Automation double-bind
Safety critical event
Design teams are fallible, therefore humans are required in the system
Humans are fallible, and should therefore be designed “out” of the system

7. Maintaining control What can help maintain or regain control?
What causes the loss of control?
Sufficient time
Unexpected events
Good predictions of future events
Acute time pressure
Reduced task load
Not knowing what happens
Clear alternatives or procedures
Not knowing what to do
Being in control of the situation means:
Capacity to evaluate and plan
Not having the necessary resources
Knowing what will happen
Knowing what has happened

8. Cyclical HMI model Team Information / feedback Provides / produces
Goals for what to do when something unusual happens:
Goals [Identify, Diagnose, Evaluate, Action]
Modifies
Team
Next action
Current understanding
Directs / controls

9. Effects of misunderstanding
The dynamics of the process only leaves limited time for interpretation
Unexpected information / feedback
Provides / produces
Increases demands to interpretation
Operator may lose control of situation
Inadequate actions
Incorrect or incomplete understanding
Loss of
accuracy increases unexpected information
Leads to

10. Prevention and protection
Accident
Initiating event
(incorrect action)
Protection (safety barriers):
Active barrier functions that deflect consequences
Protection (boundaries):
Passive barrier functions that minimise consequences
Prevention (control barriers):
Active or passive barrier functions that prevent the initiating event from occurring.

11. Types of barrier systems
Material barriers
Physically prevents an action from being carried out, or prevents the consequences from spreading
Functional (active or dynamic) barriers
Hinders the action via preconditions (logical, physical, temporal) and interlocks (passwords, synchronisation, locks)
Symbolic barriers (perceptual, conceptual barriers)
requires an act of interpretation to work, i.e. an intelligent and perceiving agent (signs, signals alarms, warnings)
Immaterial barriers (non-material barriers)
not physically present in the situation, rely on internalised knowledge (rules, restrictions, laws)

12. Barrier system types Physical, material Functional Symbolic Immaterial
Obstructions, hindrances, ...
Functional
Mechanical (interlocks)
Logical, spatial, temporal
Symbolic
Signs & signals
Procedures
Interface design
Immaterial
Rules, laws

13. Barriers systems on the road
Symbolic: requires interpretation
Physical: works even when not seen
Symbolic: requires interpretation
Symbolic: requires interpretation

14. Classification of barriers
Containing
Walls,fences, tanks, valves
Material, physical
Restraining
Safety belts, cages
Keeping together
Safety glass
Dissipating
Air bags, sprinklers
Preventing (hard)
Locks, brakes, interlocks
Functional
Preventing (soft)
Passwords, codes, logic
Hindering
Distance, delays, synchronisation
Countering
Function coding, labels, warnings
Regulating
Instructions, procedures
Symbolic
Indicating
Signs, signals, alarms
Permitting
Work permits, passes
Communicating
Clearance, approval
Immaterial
Monitoring
Monitoring
Prescribing
Rules, restrictions, laws

15. Barrier evaluation criteria
Efficiency: how efficient the barrier is expected to be in achieving its purpose.
Robustness: how resistant the barrier is w.r.t. variability of the environment (working practices, degraded information, unexpected events, etc.).
Delay: Time from conception to implementation.
Resources required. Costs in building and maintaining the barrier.
Safety relevance: Applicability to safety critical tasks.
Evaluation: How easy it is to verify that the barrier works.