1 Safety - definitions Accident - an unanticipated loss of life, injury, or other cost beyond a pre-determined threshhold.  If you expect it, it’s not an accident.  Cost is high - generally higher than the value of the service provided by the system. Safety - absence of accidents.

2 What is a system? Black box view  environment Glass box view (structure)  Collection of components (which are themselves subsystems)  Software components  Computers  Sensors, actuators  Humans  Communications  System structure can be dynamic

3 Role of software in system safety Software can be a source of failures Software can deal with error diagnosis and recovery. Interaction between software and rest of system can affect safety (e.g., GUI issues).

4 System safety Principle: software safety is meaningless without consideration of system-wide effects.  GUI confuses operator.  Wrong sensor reading causes inappropriate response from software.  Bad error recovery code aggravates error.  Loading the wrong software.

5 Hazards Hazard - a condition that inevitably leads to an accident under some combination of environmental conditions (Leveson).  Hazard may or may not lead to accident ... but it is a matter of luck - assumes system cannot control environment.  Example: Two aircraft are separated by less than minimum required distance.  If exact position and velocity is “right”, they will collide.

6 Hazard identification Cannot make a system safe unless we know how it can be unsafe. Hazard identification attempts to make explicit the safety risks of the system  This requires thinking about the system globally as well as locally  Should begin in early phases of design, when problems are relatively easy to fix.

7 Hazard identification, cont After identification  System can be redesigned to remove hazard ... or reduce probability ...and/or reduce effects of hazard. Hazards should be tracked  Hazard analysis should be updated  To reflect design and implementation changes  Increased understanding and experience (e.g. from observed accidents or malfunctions).  Also, remedies for hazards should be logged.

8 Hazard identification methods There is no magic bullet Draw on past experience  Study problems from previous systems.  There may be checklists of common hazards. “Structured brainstorming”  Form a committee of experienced people with a variety of expertise and viewpoints  Present the system in various ways  Try to think of possible problems

9 Preliminary Hazard Analysis (PHA) Goal: Early identification of hazards so they can be taken into account in system specification and design.  However, analysis should be updated to reflect increased understanding and design changes. Inputs  System design objectives  equipment specs  environmental conditions  regulations

10 >

11 Hazard Level: severity and likelihood Example DoD MIL-STD-882B Category I: Catastrophic: may cause death or loss of system. Category II: Critical: May cause severe injury, severe occupational illnusess, or major system damage. Category III: Marginal: may cause minor injury, minor occupational illness, or minor system damage. Category IV: Negligable: will not result in injury, occupational illness, or system damage.

12 Hazard likelihood British Std Frequent: Likely to be continually experienced. Probable: Likely to occur often. Occasional: Likely to occur several times. Remote: Likely to occur some time. Improbable: Unlikely, but may exceptionally occur. Incredible: Extremely unlikely that the event will ever occur.

13 Hazard Aversion/Containment Hazard elimination: e.g., robot arm is too slow to break a hole in the wall. Hazard reduction: e.g., add interlocks Hazard control: e.g., pressure valve Damage reduction: e.g., evacuation procedures.

14 PHA evaluation Advantages Focuses on hazards Disadvantages Determining hazard categories may be hard. May make unrealistic assumptions about engineering, testing, operational procedures, etc.

15 HAZOP Goal: To establish, by systematic examination of system components and operations, failure modes which can lead to hazards When: during design Inputs: results of previous safety analysis Outputs: identification and elimination of events and failure modes leading to hazards.

16 HAZOP guide words (00-58) No: Complete negation of design intention. No part of the intention is achieved, nothing else happens More: a quantitative increase Less: a quantitative decrease As well as: Design intent achieved, with additions Part of: Only some of design intention is achieved. Reverse: The logical opposite of the intention is achieved.

17 HAZOP evaluation Advantages Simplicity and ease of application Combines members from different teams Disadvantages Labor intensive Results vary depending on quality of team.

18 FMECA (FMEA) “Failure Modes, Effects (and Criticality) Analysis” Goal: Systematically analyze the components of the target system with respect to attributes relevant to safety. When: Early design, after components identified. Input: Schematics, functional diagrams, component relationships Output: List of failure modes and effects.

19 FMECA example/more info

20 FMECA evaluation Advantages Effective for analyzing single units Completeness Used to identify redundancy. Disadvantages Multiple failures/common mode failures overlooked. Time consuming and costly Failure modes must be known in advance.  Hard for complex components