Part 1 Introduction to Goal Structured Notation (GSN) Safety Assurance for the Warfighter Part 1 Introduction to Goal Structured Notation (GSN) © 2016 Nova Systems

Safety Arguments The Safety Argument should form the ‘spine’ of the Safety Case showing how these elements are related and combined to provide assurance of safety e.g. within the limits defined [Scope], the system [System Description] is SAFE because all identified hazards [System Hazards] and requirements [Safety Requirements] have been addressed. Hazards have been sufficiently controlled and mitigated [Hazard Control / Risk Reduction Measures] according to the safety risk posed [Risk Assessment]. Evidence [Safety Analysis / Test] is provided that demonstrates the effectiveness and sufficiency of these measures. Appropriate roles, responsibilities and methods were defined throughout the development of this system [Development Process Justification] [Safety Management System] and defined future operation. Argument needs to link claims to the evidence © 2016 Nova Systems

Typical Safety Arguments Probably the most common high level structure in the safety case Hazards are identified from Hazard Assessment Activity and Hazard Log Standards defined by domain e.g. ‘Safety Assessment Principles and Safety Criteria for the Naval Nuclear Programme’

Typical Safety Arguments Process and Product safe system = safe development process + safe finished product Functional Breakdown safe system = all system functions are safe SFARP safe system = all risks reduced SFARP At Least As Safe safe NEW system = at least as safe as existing system Many safety cases use combinations of these forms

Elements of an Assurance Case Basic argument structure claim – what we want to show – statement of system argument – why we believe the claim is met – links evidence to the claim evidence – information demonstrating validity of claim - test results, analysis results, etc. In general, argument broken down hierarchically claim, argument, sub-claims, sub-arguments, evidence easy to show graphically, although can be done in document structure (sub-section numbering, etc.) In practice, other concepts useful e.g. system model, assumption

Elements of an Assurance Case It is possible in text – at least sometimes use simple language and short sentences use bullet points for key statements break down the argument one step at a time and refer to following sub-sections structure document sub-sections around separate concepts e.g. Section 6.2 – Control of Hazard – ‘Inadvertent Chaff Release’ But it might be easier with pictures or tables! Use a tabular notation to ensure you have an effective argument or use a graphical notation to summarise argument Claim Structures Goal Structuring Notation (GSN)

SFARP Argument Usefulness Establishes the arguments the project intends to use to achieve a Safe System at an early stage Identify gaps or high risk areas Allows early review and agreement with multiple stakeholders: Regulators Operators In-service systems Reduces nugatory work, provides context for evidence development Each item of evidence has a clear purpose Impacts of failed/omitted activities are easy to see

Module 01: Course Introduction and Overview Saturday, 28 March 2020 Basic GSN Usage Copyright Nova Systems 2018

Graphical Safety Arguments e.g GSN We won’t teach this, but here is an example to understand the general concept. Not hard to be self taught, and can be drawn in Visio if useful to a project. Often useful to simply architect a safety argument at the beginning of a safety case report or even in the system safety program plan, as a basis of activity that follows.

Goal Structured Notation Goals [Claims] A claim forming part of the argument. Solutions [Evidence] Presents a reference to an evidence item (or items).

Goals/Claims What is the overall objective of the argument? Reading the goal structure should convince someone that ... e.g. “System X is Acceptably Safe” Things to watch out for: Jumping ahead Oversimplification

Identification of Goals Goals should be phrased as propositions: Statements said to be TRUE / FALSE NB: not limited to statements that can be objectively proven Statement should be expressed as a single statement identifying the subject of the goal and establishes the definition of the subject Eg [subject] {All identified hazards for System Y} [definition]{have been sufficiently mitigated} Goals should be phrased as positive statements of objectives achieved - not requirements or aspirations Define basis of goal unambiguously

Expanded Notations Undeveloped Goal A claim which is intentionally left undeveloped in the argument Strategy Describes the nature of the inference that exists between a goal and its supporting goals. Example 1: Goal: The lasers on my system are safe because they comply with the ARPANSA Act. However, don’t know how to achieve this yet because I haven’t read the ARPANSA Act in detail. It might be good enough for the state of the program (e.g. PDR), but we all recognise we might have to go further during DDR. It retains it’s spot in the structure. Example 2: Typically used where the sub-goals or evidence do not obviously solve the goal. e.g. All hazards have been treated, which might be solved by the “Hazard Log” – you would probably need a ‘strategy’ here to show how that worked (e.g. hazards are not closed until they are treated SFARP). © 2016 Nova Systems

Identify the Strategy Substantiate the stated goal Aim for statements that are easier to support than the larger goal ie break top level goal into number of smaller goals Relate goals more closely to specific application Clearly explain the relationship between a goal and a set of sub-goals Strategy node required whenever: Explanation of the relationship between a goal and its sub-goals is required Strategy requires additional (contextual) information, justification or assumptions

Identify the Strategy Strategies should: be expressed from the perspective of the argument approach, not the design, testing, or analysis approach not contain claims be possible to remove strategy nodes and not affect the argument being made Rationale: Assumptions - assumptions on which the strategy goal is being put forward as a solution to the parent goal Justification - why the strategy goal is being put forward as a solution to the parent goal Both assumptions and justifications are statements (like goals)

Expanded Notations Context Applicable context/scope for the claim made. Assumptions and Justifications Assumption: Intentionally unsubstantiated statement. Justification: Statement of rationale. Context: e.g. The CRE of the system. System might be safe for testing, but not operation – Safety Case context makes this clear. Assumptions/Justifications: Assumptions: Generally used where information had to be assumed and you want to highlight this. For example, you haven’t been given the population set for the anthropometric range of operators – so you assume the size, weights, etc. from a standard. Justification: Used to provide additional information in a structure (without resorting to the words underneath it) that is critical to understanding it. Typically used in conjunction with a strategy box. For example, you might claim that all hazards from a system are reduced SFARP without doing an explicit hazard identification activity with evidence of an Australian OEM’s certificate of compliance. You might think this is justified as the OEM is an Australian OEM, it’s a simple tool and you’ve ensured the CRE is identical to the OEM – thus they needed to do this work and you think it’s appropriate to rely on their activities rather than conduct your own. The justification box would be used to show this relationship. © 2016 Nova Systems

Identify Solutions When a goal doesn’t need further expansion, refinement, explanation Reference out to information that supports claim Common mistakes: Jumping ahead to a solution too soon Relationship between claim and referenced information should be clear and obvious to both writer and reader

Undeveloped Goal Operational testing is later on, so undeveloped goal at this stage. © 2016 Nova Systems

Strategy How these solutions satisfy this goal is not immediately obvious, thus the ‘strategy’ box describes how this occurs. © 2016 Nova Systems

Module 01: Course Introduction and Overview Saturday, 28 March 2020 Templates (Patterns) High Level Argument This pattern represents the basic safety argument.  It should be customised for each project, for example: What ‘part’ of the system is safe? How were all hazards identified? How was the risk assessment conducted for hazards? Was there a hazard log used? How were the safety requirements identified and verified? Has/Will the evidence been validated in-service? These patterns can be used more than once in a system, they’re just to give an idea of common arguments. Copyright Nova Systems 2018

Templates (Patterns) Lifecycle View This view takes a three-phase lifecycle view and demonstrates how safety is managed in each phase. Customisation will take the form of: What SMSs currently exist How the system will be handled in-service Any information that will need to be transferred between “design”, “in-service” and “disposal”

Configuration, Role and Environment Templates (Patterns) Configuration, Role and Environment Prior evidence shouldn’t be used without understanding the limitations of that evidence. Customisation will take the form of: What evidence is being used Whether a formal delta assessment is necessary The competence of the external authority Types/number of additional evidence which might be necessary based on expected CRE differences.

First Principles Approach Templates (Patterns) First Principles Approach Can be used when applying a pure SSPP to the capture and mitigation of hazards inherent in a program. Customisation will take the form of: Simplification where detail is unnecessary Contractor SSP for various components

A Simple Goal Structure Example We won’t teach this, but here is an example to understand the general concept. Not hard to be self taught, and can be drawn in Visio if useful to a project. Often useful to simply architect a safety argument at the beginning of a safety case report or even in the system safety program plan, as a basis of activity that follows.

Module 01: Course Introduction and Overview Saturday, 28 March 2020 Questions? Overview of the week’s activities. This Photo by Unknown Author is licensed under CC BY-SA Copyright Nova Systems 2018

Part 2 A Software Assurance Case using GSN Safety Assurance for the Warfighter Part 2 A Software Assurance Case using GSN

Software Our focus will be EO procured by Foreign Military Sales (FMS) containing software Little to zero influence on the OEM software and safety engineering processes Sparse evidence driven by security and intellectual property concerns No software knowledge assumed; GSN from Part 1 F-22 firing AIM-120 AMRAAM By U.S. Air Force/Master Sgt. Michael Ammons - Combat Archer, Public Domain, https://commons.wikimedia.org/w/index.php?curid=48048974 How do we provide assurance to the warfighter that FMS EO software is suitably safe?

Module 01: Course Introduction and Overview Saturday, 28 March 2020 Software Assurance GSN From Part 1: CRE Template Inc story on src code here! Each GSN solution on the bottom row will be described. The two undeveloped sub-goals will be developed. Copyright Nova Systems 2018

Prior Evidence - Software Source Code? I want the source code! You can’t handle the source code!!! Columbia Pictures 1992

Prior Evidence (OEM) Process Question (assess the process used to engineer the software) Evidence (example document name) Safety Functions? Safety Related Functions List Software Safety Criticality Index (SSCI)? Software Development Plan Safety Goals (specified)? System Safety Program Plan System Requirements Specification Software Engineering Process (consistent with the allocated SSCI & standard? Eg. AOP-52)? Configuration Management? Software Configuration Management Plan Residual Software Safety Hazards? System Hazard Analysis Safety Goals (achieved)? Test Reports, Certificate Notes: Distilled from AOP-52 (2008) “GUIDANCE ON SOFTWARE SAFETY DESIGN AND ASSESSMENT OF MUNITION-RELATED COMPUTING SYSTEMS”; non-exhaustive. FMS => no design specification or software source code!

Prior Evidence Develop the undeveloped goal for “Prior Evidence”:

Competence Assessment A report (eg. “Competence Assessment Report”) written by the EO acquirer, not the OEM: Who is the external authority certifying the software? In which country are they based? What are their credentials? Used by other projects? Trusted?

CRE Delta Assessment A report (eg. “CRE Delta Assessment Report”) written by the acquisition team, not the OEM. Certified Configuration vs Actual Configuration? Certified Role vs Actual Role? Regulatory Environment in country of origin vs Actual Regulatory Environment Physical Environment N/A for software

Module 01: Course Introduction and Overview Saturday, 28 March 2020 Additional Evidence Further Testing – different CRE? V&V OT&E Modelling OEM Issue Tracking Modelling – MATLAB, Mathematica, GNU Octave, Copyright Nova Systems 2018

Special Case – Certificate Only

Module 01: Course Introduction and Overview Saturday, 28 March 2020 Questions? Overview of the week’s activities. This Photo by Unknown Author is licensed under CC BY-SA Copyright Nova Systems 2018