LDRA Technology Pvt. Ltd Software Technology A Practitioner's Guide to DO-178B, Certification and the Emerging DO-178C Standard Shinto Joseph Operations Director, LDRA Technology Pvt. Ltd Bangalore 1

Verification Activities Agenda Introduction DO-178B Overview Verification Activities Review Testing Analysis What’s Coming with DO-178C? DO-178C Structure Software Development Landscape Traceability Indian Scenario Summary 2

Agenda DO-178B Overview

DO-178’s Timeline DO-178, November 1981 DO-178A, March 1985 Basic guidance DO-178A, March 1985 3 failure conditions / software levels critical/1, essential/2, non-essential/3 Development/verification steps DO-178B, December 1992 5 failure conditions / software levels Catastrophic/A, Hazardous/B, Major/C, Minor/D, no effect/E Objectives-based DO-178C, 2011? A modest update to DO-178B (If C based development) Adds guidance on model-based development, formal methods, object-oriented technology & tool qualification

F-16 Falcon Unstable airframe

F-16 Falcon Unstable airframe Flipped crossing 0o

? What failures can occur Severity Probability Hazard Analysis Result - System level Safety Integrity Level (SIL)

Probability of Failure DO-178B Safety Integrity Levels Software Level Impact of Failure Probability of Failure (per operating hour)* A Catastrophic 10-9 B Hazardous 10-7 C Major 10-5 D Minor 10-3 E No effect N/A *FAA System Safety Handbook, Chapter 3: Principles of System Safety; December 30, 2000

DO-178B process Safety Assessment Process Guidelines & Methods (ARP 4761) Intended Aircraft Functions Safety Information System Design Information Aircraft and System Development Processes (ARP 4754) Guidance for Integrated Modular Avionics (DO-297) Electronics Hardware Development Lifecycle (DO-254) Electronics Hardware Development Lifecycle (DO-254) Electronics Hardware Development Lifecycle (DO-254) Software Development Lifecycle (DO-178B)

…..DO-178B process Intended to ensure that avionics software performs its intended function with an appropriate level of confidence in safety. Defines 5 processes: Planning, development, verification, configuration management and quality assurance Defines 5 levels of design assurance and 66 objectives: Level A: 66 objectives (25 with independence) Level B: 65 objectives (14 with independence) Level C: 57 objectives Level D: 28 objectives Level E: no objectives Provides guidelines for implementing these processes and meeting these objectives. 10

Structural Coverage Technique DO-178B (cont.) Certifiable Software became central goal Deterministic Verification Techniques Software Level Impact of Failure Structural Coverage Technique A Catastrophic MC/DC B Hazardous Decision C Major Statement MC/DC code coverage ensures that all conditions that independently affect a programmatic result have been tested

Verification Process Purpose: Detect and report errors that have been introduced during the software development process. Objectives: System Requirements satisfies Software Requirements satisfies Software Architecture satisfies satisfies satisfies Executable Object Code Source Code 12

Verification Activities Review A qualitative assessment of accuracy, completeness consistency and correctness. Testing Demonstrate that the software satisfies its requirements. Demonstrate, to an appropriate degree of confidence, that errors that could lead to unacceptable failure conditions have been removed. Analysis A quantitative assessment of accuracy, completeness consistency and correctness. 13

Review A review provides a qualitative assessment of accuracy, completeness consistency and correctness. IP boilerplate Comments Indentation Complexity … - Compliance with requirements Compliance with architecture Verifiability Accuracy and consistency … standards checklist if (x < 0) then z = y – 2; else z = y + 2; Review Result source code 14

Testing Testing demonstrates, to an appropriate degree of confidence, that software satisfies its requirements and that errors that could lead to unacceptable failure conditions have been removed. Requirements-based tests: verify implementation of requirements. HW/SW integration tests: verify correct operation in the target computer environment. SW/SW integration tests: verify software interfaces and interrelationships. 15

Test success Test failure Test Result Analysis Proceed Incorrect software behavior Incorrect requirement Incorrect test case/procedure Incorrect test environment/setup 16

Traceability Analysis Requirements Code Tests Objectives Verify that every requirement is implemented. Verify that every requirement is tested. Verify that every line of code has “a reason to be”. Common gaps Requirement has no associated tests: Missing trace information, missing tests. Requirement has no associated source code: Missing trace information, missing code, extraneous requirement. Source code doesn’t trace to requirements: Missing trace information, extraneous code. 17

Moving from DO-178B to C: The Essentials 18

DO-178C Core Document Including DO-178B & Revised Processes

Core Document DO-178C Including DO-178B & Revised Processes Formal Methods Supplement Model-Based Development Supplement Object- Oriented Technologies Supplement

Core Document DO-178C Including DO-178B & Revised Processes Formal Methods Supplement Model-Based Development Supplement Object- Oriented Technologies Supplement Tools Supplement

Software Development Landscape DO-178C Software Development Landscape

High Level Requirements Tier 1

High Level Requirements Modelling Tools Formal Methods Software Specs Tier 1 Modelling Tools Formal Methods Software Specs Hand Code Tier 2

(Source Code / Assembly ) High Level Requirements Tier 1 Modelling Tools Formal Methods Software Specs Hand Code Tier 2 Implementation (Source Code / Assembly ) Tier 3

(Source Code / Assembly ) High Level Requirements Tier 1 Modelling Tools Formal Methods Software Specs Hand Code Tier 2 Implementation (Source Code / Assembly ) Tier 3 Host Tier (Node 1 – n) Tier 4

(Source Code / Assembly ) High Level Requirements Tier 1 Modelling Tools Formal Methods Software Specs Hand Code Tier 2 Implementation (Source Code / Assembly ) Tier 3 Host Tier (Node 1 – n) Tier 4 Target Tier (Node 1 – n) Tier 5

Requirements Traceability Matrix High Level Requirements Tier 1 Requirements Traceability Matrix Design Review defects LL Reqs to HL Reqs Modelling Tools Formal Methods Software Specs Hand Code Tier 2 Requirements Traceability Matrix Code & Quality Review defects Code to LL Reqs Implementation (Source Code / Assembly ) Tier 3 Test Results & Defects Requirements Traceability Matrix Test Cases to LL Reqs Host Tier (Node 1 – n) Tier 4 Test Results & Defects Requirements Traceability Matrix Test Cases to LL Reqs Target Tier (Node 1 – n) Tier 5

Traceability: Complex

Low Level Requirements, or design? Complexity: Sources Formal Methods Model-Based Development Object- Oriented Technologies Low Level Requirements, or design? Dynamic aspects: Coverage must be performed on target & combined with static traces to assure completeness

DO-178C Traces Level A, B,C and D System requirements allocated to Software High-Level Requirements Test Cases Low-Level Requirements Test Procedures Review and Analysis Results Source Code Test Results Executable Object Code SW Architecture

DO-178C Traces Level A, B,C and D System requirements allocated to Software Level A, B and C High-Level Requirements Test Cases Low-Level Requirements Test Procedures Review and Analysis Results Source Code Test Results Executable Object Code SW Architecture

DO-178C Traces Level A, B,C and D System requirements allocated to Software Level A, B and C Level A High-Level Requirements Test Cases Low-Level Requirements Test Procedures Review and Analysis Results Source Code Test Results Executable Object Code SW Architecture

Requirements Traceability Matrix IBM® Rational® DOORS® & Visure IRQA... High Level Requirements TBreq® Requirements Traceability TBmanager® System Test Management Unit Test Requirements Traceability Matrix LDRA Testbed® Design Review Defects Modelling Tools Formal Methods Software Specs Hand Code Requirements Traceability Matrix TBvision® Code Review Defects Implementation (Source Code / Assembly) Requirements Traceability Matrix TBrun® Host Testing Host Tier (Node 1 – n) Requirements Traceability Matrix TBrun® Target Testing Target Tier (Node 1 – n)

-Lack of safety awareness -Gap between local and global practices Indian Scenario -Lack of safety awareness -Gap between local and global practices -Sudden demand for aerospace skills -Need for a healthy ecosystem, backed by long term govt. policies -Committed engineers ready to work on Indian projects -Role of technology vendors -Regulatory framework- Defense and Civilian

Verification is an important component of DO-178 Summary Summary Verification is an important component of DO-178 Review Testing Analysis Bottom line Detect and report errors that have been introduced during the software development process. Ensure that the software performs its intended function to an appropriate degree of confidence. 36

Future: ......Summary …Requirements management / traceability paradigm no longer adequate Future: Should accommodate emerging technologies, methodologies Requires distributed, collaborative, bidirectional traceability mechanism Security India- an aerospace powerhouse…..

Copyright © 2011 Liverpool Data Research Associates Limited Software Technology www.ldra.com india@ldra.com shinto.joseph@ldra.com Thank you for your time today. If you require any further information please visit the LDRA website, or e-mail me direct with any questions you may have. Copyright © 2011 Liverpool Data Research Associates Limited 38