1. SEDS Research GroupSchool of EECS, Washington State University Annual Reliability & Maintainability Symposium January 30, 2002 Frederick T. Sheldon and Hye Yeon Kim Software Engineering for Dependable Systems (SEDS) Research Laboratory School of Electrical Engineering and Computer Science Washington State University Validation of Guidance Control Software Requirements Specification for Reliability and Fault-Tolerance

2. SEDS Research LaboratorySchool of EECS, Washington State University Overview  Goal: Show the feasibility of this analysis approach using a industrial strength SRS to ensure:  Completeness and Consistency  Fault-tolerance  Specification Under Study  A NASA provided Guidance and Control Software (GCS) development specification for the Viking Mars Lander.  Analysis Approach  Using Zed to specify the data  Using Statecharts : Statemate for dynamical analysis  Summary and Future study

3. SEDS Research LaboratorySchool of EECS, Washington State University Introduction  Why Requirements Specification?  Cost, Time, and Effort Defect detected phaseTypical cost of correction Requirements Specification$100 - $1,000 Coding/Unit Testing$1,000 or more System Testing$7,000 - $8,000 Acceptance Testing$1,000 - $100,000 After ImplementationUp to millions of dollars

4. SEDS Research LaboratorySchool of EECS, Washington State University Reliable Specification  Is Correct  Complete, consistent and robust  Can the specification be trusted while minimizing the risk of costly errors?  How to analyze the specification to prevent the propagation of errors into the downstream activities?

5. SEDS Research LaboratorySchool of EECS, Washington State University Consistency and Completeness  Completeness: The lack of ambiguity  Incomplete if …  … the system behavior is not specified precisely because the required behavior for some events or conditions is omitted or is subject to more than one interpretation.  Consistency  The Specification is free from conflicting requirements and undesired nondeterminism.

6. SEDS Research LaboratorySchool of EECS, Washington State University Fault Tolerance  Faults  A fault is a feature of a system that precludes it from operating according to its specification –H. Ammar, B. Cukic, C. Fuhrman, and A. Mili, A comparative Analysis of HW and SW fault tolerance: Impact on software reliability engineering, 1999  Fault Tolerance  The ability to respond to unexpected system failure (detection and mask/recover)

7. SEDS Research LaboratorySchool of EECS, Washington State University Guidance and Control Software  Software Requirements – GCS Dev. Spec.  The system was designed to provide software control of the embedded sensors and actuators of the Viking Mars Lander during the terminal decent (landing) phase.  ARSP (Altimeter Radar Sensor Processing)  The ARSP module reads data provided by the altimeter radar sensor to determine the lander’s altitude from the Mars surface.

8. SEDS Research LaboratorySchool of EECS, Washington State University Mars Lander trajectories

9. SEDS Research LaboratorySchool of EECS, Washington State University Velocity – Altitude Contour

10. SEDS Research LaboratorySchool of EECS, Washington State University

11. SEDS Research LaboratorySchool of EECS, Washington State University

12. SEDS Research LaboratorySchool of EECS, Washington State University Zed Overview  Clarifying ambiguities  Identify assumptions  Correctness checking  Mathematical proofs  Giving an in-depth understanding of the SRS

13. SEDS Research LaboratorySchool of EECS, Washington State University Statecharts  Visual formalism: Diagrammatic in nature  Testability is provided through symbolic simulation  Predevelopment evaluation through  Fault Injection  Statemate consists of …  Activity chart  Statecharts

14. SEDS Research LaboratorySchool of EECS, Washington State University Natural Language based SRS  Inherently ambiguous risking the possibility of multiple interpretations

15. SEDS Research LaboratorySchool of EECS, Washington State University Zed Schema

16. SEDS Research LaboratorySchool of EECS, Washington State University From NL to Zed  Discovered Ambiguities  The confusing definition of variable “Rotation”, and direction of the rotation.  The type assigned to the AR_COUNTER variable was unclear.  An undefined 3 rd order polynomial.  Where the AR_COUNTER should be modified?

17. SEDS Research LaboratorySchool of EECS, Washington State University Statecharts Model: Activity chart

18. SEDS Research LaboratorySchool of EECS, Washington State University Statecharts Model: Statechart 1

19. SEDS Research LaboratorySchool of EECS, Washington State University Statecharts Model: Statechart 2

20. SEDS Research LaboratorySchool of EECS, Washington State University Some Theory … Set of Inputs (  ) Set of Outputs Unknowns (  ) Known Safe Unsafe Assumed Safe Sources: Normal Operation Hardware Failures Human Intervention Models/Simulators

21. SEDS Research LaboratorySchool of EECS, Washington State University

22. SEDS Research LaboratorySchool of EECS, Washington State University Paradigms …  Software Failures: “Software does not fail - it just does not perform as intended” Professor Nancy Leveson, MIT  Design and test for functionality: Also specify what the system should not do...... then test it!

23. SEDS Research LaboratorySchool of EECS, Washington State University Some Theory… lets take a second look Set of Inputs (  ) Set of Outputs Unknowns (  ) Known Safe Unsafe Assumed Safe Sources: Normal Operation Hardware Failures Human Intervention Models/Simulators Fault Injection (added known)

24. SEDS Research LaboratorySchool of EECS, Washington State University Testing and Fault Injection  By using symbolic simulation in Statemate  Boundary Testing  Input that is inside of the variable range  Input that is outside of the variable range  Fault Injection  State variable alternation  State transition redirection

25. SEDS Research LaboratorySchool of EECS, Washington State University Testing Results ARSP Specification Test Input and Output VariableCase 1Case 2Case 3Case 4Case 5 Input FRAME_COUNTER 22113 AR_STATUS --[0, 0, 0, 0]-[0, 1, 0, 0] AR_COUNTER 1990020000 Output AR_STATUS KP [1, 0, 0, 0][0, -, -, -][1, 0, 1, 0] K_ALT KP [1, 1, 1, 1][1, -, -, -][0, 1, -, 1] AR_ALTITUDE KP [*, -, -, -][2000,-,-,-]KP

26. SEDS Research LaboratorySchool of EECS, Washington State University Detailed Testing Results - Case 1  Initial values  Final values  Initial variable values are being calculated based on the given equations. VariableCase 1 Input FRAME_COUNTER 2 AR_STATUS - AR_COUNTER Output AR_STATUS [1, 0, 0, 0] K_ALT [1, 1, 1, 1] AR_ALTITUDE [2000, -, -, -] [1, 1, 0, 0] [1, 1, 1, 1] [2000, 2000, -, -]

27. SEDS Research LaboratorySchool of EECS, Washington State University Fault Injection Results

28. SEDS Research LaboratorySchool of EECS, Washington State University Detailed Fault Injection Results  Change FRAME_COUNTER at CURRENT_STATE from 2 to 3 VariableCase 1 Input FRAME_COUNTER 2 AR_STATUS - AR_COUNTER Output AR_STATUS [1, 0, 0, 0] K_ALT [1, 1, 1, 1] AR_ALTITUDE [2000, -, -, -] [1/0, 1, 0, 0] [1, 1, 1, 1] [*, 2000, -, -]

29. SEDS Research LaboratorySchool of EECS, Washington State University Summary  Interpretation from NL to Zed  Clarifying ambiguities  Interpretation from Zed to Statecharts  Clarifying misinterpreted Zed specification  Iterative process  Boundary Testing, Fault Injection analysis  Reveals weak point(s) in the system  Fault Tolerance validation

30. SEDS Research LaboratorySchool of EECS, Washington State University Conclusion  Using this combination of FMs we could:  Clarify ambiguities  Validate Correctness, Completeness, and Consistency  Validate Fault tolerance features of the SRS  This approach enabled us to:  Avoid the problems that result when incorrectly specified artifacts force corrective rework.  Minimize the risk of costly errors being propagated into downstream activities

31. SEDS Research LaboratorySchool of EECS, Washington State University Future Study  Build concrete translation rules between the methods  Find an effective algorithm to automate the process  Validate the algorithm for the different (domain/ application specific) critical software requirements  In depth comparative study with other approaches