Testing safety-critical software systems Marcos Mainar Lalmolda Quality Assurance and Testing 20th November 2009 1 1

Contents What a safety-critical software system is Standards Programming features and languages Approaches on design Testing Conclusion 2 2

What a safety-critical software system is Harm people's lives causing deaths. They are widely used in various different fields, almost in everything nowadays. A safety-critical software system is a computer system whose failure or malfunction may severely harm people's lives, environment or equipment. Some fields and examples: Medicine (patient monitors) Nuclear engineering (nuclear power station control) Transport (railway systems, cars anti-lock brakes) Aviation (control systems: fly-by-wire) Aerospace (NASA space shuttle) Civil engineering (calculate structures) Military devices Etc. 3 3

Safety-critical Standards Industries specific Medical device software: IEC 62304 Nuclear power stations: IEC 60880 Aerospace: AS9100A Airbone: DO178B … Scale of 5 safety integrity levels: 4 is very high, 0 not safety related. Safety engineering 4 4

Programming features and languages (I) General principle: Try to keep the system as simple as possible. Programming features not recommended: Pointers and dynamic memory allocation/deallocation. Unstructured programming (gotos) Variant data Implicit declaration and initialisation Recursion Concurrency and interrupts 5 5

Programming features and languages (II) Features which increase reliability: Strong typing Run time constraint checking Parameter checking Language to be avoided: C Language recommended: Ada Ada subset for safety-critical software: SPARK Other languages: increased overhead 6 6

Approaches on design Formal methods Assume that errors exist and design prevention and recovery mechanisms. “Program verification does not mean error-proof programs […]. Mathematical proofs can also be faulty. So whereas verification might reduce the program-testing load, it cannot eliminate it” (F.P. Brooks, No Silver Bullet, 1987). 7 7

Testing safety-critical software systems (I) physical condition of platform that threatens the safety of personnel or the platform, i.e. can lead to an accidenta condition of the platform that, unless mitigated, can develop into an accident through a sequence of normal events and actions"an accident waiting to happen" Basic idea: Identify hazards as early as possible in the development life-cycle and try to reduce them as much as possible to an acceptable level. Remember: Always test software against specifications! Independent verification required If formal methods have been used then formal mathematical proof is a verification activity. Already known techniques used for typical systems White box testing Black box testing Reviews Static analysis Dynamic analysis and coverage 8 8

Testing safety-critical software systems (II) Specific procedures and techniques from safety engineering: Probabilistic risk assessment (PRA) Failure modes and effects analysis (FMEA) Fault trees analysis (FTA) Failure mode, effects and criticality analysis (FMECA) Hazard and operatibility analysis (HAZOP) Hazard and risk analysis Cause and effect diagrams (aka fishbone diagrams or Ishikawa diagrams) 9 9

Probability Risk Assessment Hazard Severity Probability Risk Risk Criteria Tolerable? Risk Reduction Measures No Yes *From Safety-Critical Computer Systems – Open Questions and Approaches presentation, Andreas Gerstinger, February 16, 2007, Institute of Computer Technology, Wien 10 10 10 10

Fault tree analysis (FTA) A graphical technique that provides a systematic description of the combinations of possible occurrences in a system which can result in an undesirable outcome (failure). An undesired effect is taken as the root of a tree of logic Each situation that could cause that effect is added to the tree as a series of logic expressions. Events are labelled with actual numbers about failure probabilities. The probability of the top-level event can be determined using mathematical techniques. 11 11

An example of a Fault tree *From http://syque.com/quality_tools/toolbook/FTA/how.htm 12 12

Conclusions Complex subject Suitably trained and experienced people are key to the success of any software development. Main objective of testing techniques: minimise risk of implementation errors. Above all, the best way to minimise risk both to safety, reliablity and to the timescale of a software project is to keep is simple. 13 13

Questions ¿? 14 14

References Wikipedia. http://en.wikipedia.org IPL Information Processing Ltd, An Introduction to Safety Critical Systems, Testing Papers. http://www.ipl.com/include/download/CookieRequestPage.php?FileID=p0820 IPL Information Processing Ltd, An Introduction to Software Testing, Testing Papers. http://www.ipl.com/include/download/CookieRequestPage.php?FileID=p0826 Evangelos Nikolaropoulos, Testing safety-critical software, Hewlett-Packard Journal, June 1997. http://findarticles.com/p/articles/mi_m0HPJ/is_n3_v48/ai_19540814/?tag=content;col1 Frederick P. Brooks, Jr. , No Silver Bullet: Essence and Accidents of Software Engineering, 1986. Andreas Gerstinger, Safety-Critical Computer Systems – Open Questions and Approaches presentation, February 16, 2007, Institute of Computer Technology, Wien. Fault Tree Analysis: How to understand it. http://syque.com/quality_tools/toolbook/FTA/how.htm 15 15