1. Testing safety-critical software systems
Marcos Mainar Lalmolda
Quality Assurance and Testing
20th November 2009 1 1

2. Contents What a safety-critical software system is Standards
Programming features and languages
Approaches on design
Testing
Conclusion 2 2

3. What a safety-critical software system is
Harm people's lives causing
deaths.
They are widely used in various
different fields, almost in everything
nowadays.
A safety-critical software system is a computer system whose failure or malfunction may severely harm people's lives, environment or equipment.
Some fields and examples:
Medicine (patient monitors)
Nuclear engineering (nuclear power station control)
Transport (railway systems, cars anti-lock brakes)
Aviation (control systems: fly-by-wire)
Aerospace (NASA space shuttle)
Civil engineering (calculate structures)
Military devices
Etc. 3 3

4. Safety-critical Standards
Industries specific
Medical device software: IEC 62304
Nuclear power stations: IEC 60880
Aerospace: AS9100A
Airbone: DO178B …
Scale of 5 safety integrity levels: 4 is very high, 0 not safety related.
Safety engineering 4 4

5. Programming features and languages (I)
General principle: Try to keep the system as simple as possible.
Programming features not recommended:
Pointers and dynamic memory allocation/deallocation.
Unstructured programming (gotos)
Variant data
Implicit declaration and initialisation
Recursion
Concurrency and interrupts 5 5

6. Programming features and languages (II)
Features which increase reliability:
Strong typing
Run time constraint checking
Parameter checking
Language to be avoided: C
Language recommended: Ada
Ada subset for safety-critical software: SPARK
Other languages: increased overhead 6 6

7. Approaches on design Formal methods
Assume that errors exist and design prevention and recovery mechanisms.
“Program verification does not mean error-proof programs […]. Mathematical proofs can also be faulty. So whereas verification might reduce the program-testing load, it cannot eliminate it” (F.P. Brooks, No Silver Bullet, 1987). 7 7

8. Testing safety-critical software systems (I)
physical condition of platform that threatens
the safety of personnel or the platform, i.e.
can lead to an accidenta condition of the
platform that, unless mitigated, can develop
into an accident through a sequence of
normal events and actions"an accident
waiting to happen"
Basic idea: Identify hazards as early as possible in the development life-cycle and try to reduce them as much as possible to an acceptable level.
Remember: Always test software against specifications!
Independent verification required
If formal methods have been used then formal mathematical proof is a verification activity.
Already known techniques used for typical systems
White box testing
Black box testing
Reviews
Static analysis
Dynamic analysis and coverage 8 8

9. Testing safety-critical software systems (II)
Specific procedures and techniques from safety engineering:
Probabilistic risk assessment (PRA)
Failure modes and effects analysis (FMEA)
Fault trees analysis (FTA)
Failure mode, effects and criticality analysis (FMECA)
Hazard and operatibility analysis (HAZOP)
Hazard and risk analysis
Cause and effect diagrams (aka fishbone diagrams or Ishikawa diagrams) 9 9

10. Probability Risk Assessment
Hazard
Severity
Probability
Risk
Risk Criteria
Tolerable?
Risk Reduction Measures No
Yes
*From Safety-Critical Computer Systems – Open Questions and Approaches presentation, Andreas Gerstinger,
February 16, 2007, Institute of Computer Technology, Wien 10 10 10 10

11. Fault tree analysis (FTA)
A graphical technique that provides a systematic description of the combinations of possible occurrences in a system which can result in an undesirable outcome (failure).
An undesired effect is taken as the root of a tree of logic
Each situation that could cause that effect is added to the tree as a series of logic expressions.
Events are labelled with actual numbers about failure probabilities.
The probability of the top-level event can be determined using mathematical techniques. 11 11

12. An example of a Fault tree
*From 12 12

13. Conclusions Complex subject
Suitably trained and experienced people are key to the success of any software development.
Main objective of testing techniques: minimise risk of implementation errors.
Above all, the best way to minimise risk both to safety, reliablity and to the timescale of a software project is to keep is simple. 13 13

14. Questions ¿? 14 14

15. References Wikipedia. http://en.wikipedia.org
IPL Information Processing Ltd, An Introduction to Safety Critical Systems, Testing Papers.
IPL Information Processing Ltd, An Introduction to Software Testing, Testing Papers.
Evangelos Nikolaropoulos, Testing safety-critical software, Hewlett-Packard Journal, June
Frederick P. Brooks, Jr. , No Silver Bullet: Essence and Accidents of Software Engineering, 1986.
Andreas Gerstinger, Safety-Critical Computer Systems – Open Questions and Approaches presentation, February 16, 2007, Institute of Computer Technology, Wien.
Fault Tree Analysis: How to understand it. 15 15