2. Safety Critical Systems and Certification Issues DO-178B Airborne Standard

3. SC190 / WG-52 Application Guidelines For RTCA DO-178b/ED-12b
EUROCAE
SC-167
WG-12
DO-178B
ED-12B
CNS/ATM
Community
Avionics Industry
Cast Position
Papers
SC-190 / WG-52
RTCA DO-178B is the joint development of:
RTCA SC-167 and EUROCAE WG-12
It’s history is:
RTCA SC-145 -> EUROCAE WG-12 ED-35 developed in parallel.
ED-12 was withheld and RTCA and E. worked jointly on SC-145.
1982 => SC-145 produced the DO-178 document.
1985 => SC-152 produced DO-178A
1992 => EUROCAE WG-12 & RTCA SC-167 produced DO-178B
WG = Working Group
SC = Special Committee
RCTA = Radio Tech. Comm. On Aeronautics
EUROCAE = E. Org. for Civil Aviation Equipment
CAST
(Certification
Authority
Software
Team)
SC-190 Products 3

4. DO-178B in a Nutshell Highly Process Oriented Requires Traceability
Requirements
High Level Design
Detailed Design
Source Code
Test Procedures
Test results
Test & Test & Test & Test and ….. 4

5. DO-178B Safety Levels Level A Level B Level C Level D Catastrophic
Failure Prevents Continued Safe Flight and Landing
Level B
Hazardous/Severe-Major
Potential Fatal Injuries to a Small Number of Occupants
Level C
Major
Discomfort to Occupants or Possible Injuries
Level D
Minor
Increased Crew Workload 5

6. Certification Evidence
Development Team
What is COTS?
How can objectives of DO-178B be satisfied, using COTS?
There is much variation in applicants for COTS certification credit
Is DO-178B clear on the interpretations?
Certification Evidence
Available as COTS
Product is COTS +
Together these satisfy safety objectives 6

7. Testing without Source Code
Wrappers to
Validate Parameters
Commercial O/S
Application
This cannot be trusted
unless O/S is verified 7

8. Special Considerations
Use of Service History for Certification
Air Traffic Control system
Worked Correctly in US for years
Transferred to U.K.
Actual Plane 2
Plane 1
Displayed Plane 2
Greenwich Meridian 8

9. Use of Service History
Developed under a less stringent standard (Military?)
Used for 4 years
Problems tracked
Quality Good!
Safety
Critical
System
Residual Error
Dead Code(Unintended Function)
Is this system safe for the next 4 years? At Level A, B, C?
We can bound inputs, but we cannot check internal states
without “looking inside” 9

10. Black Box Testing
No single failure should prevent “Continuous safe flight and landing.”
Statistical testing cannot show absence of a single state that will cause a failure
Software has discontinuities
Software does not follow Gaus/Normal Distribution
There is no foundation for statistical reasoning
about software faults or safety 10

11. Verification Team examples of issues
What are low level requirements? How can they be tested
Data and Control flow coupling
Use of higher level test results for lower level requirements
What is the intent of structural coverage?
Traceability of source to object code for structural coverage
What is statement, decision, condition and MCDC coverage testing (Modified Condition/Decision Code)
Verification tool qualification
etc..
MCDC = Modified Condition/Decision Code 11

12. Coverage Analysis at Level B and C
Statement Coverage
Decision Coverage
Entry Points
Exit Points
All Decisions
All Outcomes 12

13. Fixing anomalies example for level B/Library
Filter
B := 3;
A := Filter (B);
X := X + A;
Compiler
Object Code
Source level coverage required
Even for Filter 13

14. Boundary Level testing not enough!
A := B; -- A and B are packed Boolean arrays
Run-time call to:
Bit_Block_Move (From, To, Size); -- size in bits
Min, Mid, Max in combination gives 27 (useless) test cases
5 Bits size
32 Bit size
67 Bit Size
From overlaps To
To overlaps From
Interesting test cases based on
actual code
i.e. White Box Testing 14

15. Coverage at Level A Coverage required at Machine Code level or
Show source to object code traceability
and test at source level or
Use different language/compilers
and use voting system
MCDC testing required
each condition must have effect on outcome
Tools which modify source for traceability
problem at level A
Mitigation method : use 3 different compilers
(Now Look At Conditional Statements) 15

16. Conditions/Decisions
if A=B and C or D<3 then
Decision
Boolean Variable
Boolean Operators 16

17. What are Conditions/Decisions
if (A=B and C) or D<3 then
Ada :
if ((A==B) & C ) | (D<3) then
C :
if ((A==B) * C) + (D<3) then
C :
if ((A==B) and C) or (D<3) then
C++ :
MCDC Coverage Requires all Branches AND all Conditions Be Covered 17

18. More Boolean Conditions
X := (A=B and C) or D<3;
if X then -- X is Boolean
Ada :
Cannot hide from Testing Obligations
X = ((A==B) * C) + (D<3));
if X then /* X can be any Integer
C :
‘*’ and ‘+’ are Boolean Operators! 18

19. Condition coverage X := (A=B and C) or D<3;
if X then -- X is Boolean
Ada :
Coverage of “Basic-Block”
may not capture condition
results 19

20. Avoiding MCDC Testing Use Ada’s short-circuit conditions:
Modified Condition/Decision Code
Use Ada’s short-circuit conditions:
if A=0 and then B< 2 and then C>5 then
Or in C write:
if A== 0 && B < 2 && C < 5 { 20

21. Why short-circuit conditions eliminate MCDC
if A=0 and then B< 2 and then C>5 then
if A=0 then
if B<2 then
if C>5 then P;
end if;
MCDC not required for this code 21

22. Testing strategy must evaluate conditions
if A=0 and then B< 2 and then C>5 then
if A=0 then
if B<2 then
if C>5 then P;
end if;
BUT !!!
Testing must show that each ‘then’ part has
been tested True and False
MCDC not required for this code 22

23. Inlining code If decisions/conditions introduced
Decisions must be identified and verified (level B)
Conditions must be identified and verified (level A)
Verification may be done by analysis
Traced to derived requirements
ensure safety is not compromised
Code may be “deactivated”
As inlined code depends on local state it may be very hard to test the conditions in accordance with standards requirements
Intent - absence of unintended funtion
Dead code not allowed 23

24. Use of Tools
Tool Qualification is required if tool replaces a step of development process
Development tools
Examples - Compiler, Design to code generator
May introduce an error
In general - NOT qualified, not trusted
Verification tools
Examples - Coverage analyser
May conceal an error
May be qualified - Trusted for verification purposes
Additional verification process required if the tool is not trusted 24

25. CNS/ATM Process Integration
Information matrix
Regulators
Committees
Standards Bodies
Standard
Software Integrity Assurance Standards vs. Software Development Standards
Relationships between DO-178B
and IEC 61508 25

26. Ground Based Community
Communications/ Navigation/Surveilance and Air Traffic Management - in the loop
Ground Based Systems affect airborne software
DO-178B addresses airborne only
Guidance being prepared to encompass needs of CNS/ATM community (SC-190 committee)
Standards tightening up 26

27. Certification Standards are Improving
“Holes” in documents are being fixed
Understanding of Certification Requirements is spreading
Industry and Certification Authorities are collaborating on Guidance materials
It will get more difficult to “shop around” for a more lenient signature 27

28. ( Legal) Safety Systems
Laws
Regulations
Standards
Guidelines
Case Law
Precedence
Interpretations
Standards
Guidelines
PROCESS
Visibility
Traceability
EVIDENCE / RECORD
Confidence / Safety 28

29. When Is Software Safe 29

30. When Is Software Safe
We Don’t Know !! 30

31. What is our best guess about the safety
When applicable processes have been followed
When we have verified the code “from within”
When this has been checked
and checked 31

32. The FAA/JAA Rules are Strict
To Date:
“no fatalities have been attributed to Software Failure”
Have we been lucky?
Have we been safe? 32