1. Copyright Critical Software S.A. 1998-2007 All Rights Reserved. G-SWFIT A Technique for Fault Injection in Executable Code SAS’07, Morgantown Sept. 25th-27th Nuno Silva Ricardo Barbosa VV&RAMS Engineering Unit

2. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 2 Presentation Overview Last Year’s Presentation Current R&D at CSW SpaceAQua Project G-SWFIT Technique and Xception Plugin Other Research Initiatives Future Work

3. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 3 Last Year Presentation COTS vs. Custom Made “Evaluation of COTS through Fault Injection?” Three techniques for assessing COTS components Robustness and Stress Testing Double phased technique using API fault injection and SWIFI Fault Injection for Risk Assessment Assessment of real time (COTS) kernels through fault injection Presentation of the case studies and results

4. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 4 Current R&D at CSW Main stream R&D is divided into two different (but related) trends SpaceAQua Project G-SWFIT Technique Further development (improvement and optimization) of current Xception plugins Mainly based on industrial usage feedback Other Research Initiatives Xpy – Non intrusive monitoring tool for space applications TestOO – Static assessment of object oriented real time applications

5. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 5 SpaceAQua Project Automated Qualification Framework SpaceAQua will integrate several techniques and tools for assessing COTS used in space Qualify each assessed product according to a predefined qualification scheme Generic Test Plan for Reuse Automate qualification steps (including the definition, generation and execution of test cases)

6. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 6 G-SWFIT Technique G-SWFIT Assessment and research of the technique Applicability analysis of technique in space systems evaluation, namely COTS Implementation of plug-in for Xception toolset (for Intel IA32 architecture) Experimentation and validation of toolset

7. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 7 G-SWFIT Technique PhD work of João Durães Durães, J., Madeira, H., “Definition of Software Fault Emulation Operators: a Field Data Study”, DSN 2003 Durães, J., Madeira, H., “Generic Faultloads Based on Software Faults for Dependability Benchmarking” Durães, J., Madeira, H., “Multidimensional Characterization of the Impact of Faulty Drivers on the Operating Systems Behavior”, IEICE 2003 Durães, J., Madeira, H., “Emulation of Software Faults by Educated Mutations at Machine-Code Level”, ISSRE 2002 Fault injection technique Used on binary code No source code needed No recompilation or development environment required Applicable virtually to any software Need to investigate legal aspects for COTS software Check impacts on SW with Checksums

8. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 8 G-SWFIT Technique – Fault Types Analyses of opensource projects bug lists Classification of faults using: ODC associated to the type of correction needed Extra variable ‘nature’ defines if there is something ‘missing’, ‘wrong’ or ‘extraneous’ Selection of the most representative faults

9. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 9 G-SWFIT Technique - Fault Injection Example

10. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 10 G-SWFIT Technique - Fault Creation and Injection Process Operator Fault Operator Original executable Mutated executable Assembly representation Fault 1. Disassembling 2. Fault generation 3. Fault injection Hard drive Memory

11. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 11 G-SWFIT Technique - Operator example Fault Operator for Missing Function Call 1. Locates function callsPattern 2. Function call not alone in blockConstraint 3. Function Returned value (if any) not usedConstraint 4. Removes function callInjection CALL removal is made by replacing the CALL instruction by NOP instructions ExampleSearch patternCode change function(...);CALL target-addressremove ‘CALL’ instruction

12. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 12 Other Research Initiatives - XPY XPY - Monitoring and Profiling Tool for Space Software XPY provides the end user (VV Engineers) with a automated code coverage analysis of the user software in a non-intrusive way through boundary scan. XPY calculates coverage metrics over the original non-instrumented code, based on the low level monitoring of the target system. SCEEC DCBCC MC/DC SC (Statement Coverage), EEC (Entry and Exit Point Coverage), DC (Decision Coverage), BCC (Branch Condition Coverage), MC/DC (Modified Condition/Decision Coverage)

13. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 13 XPY - Monitoring and Profiling Tool for Space Software Modular architecture design allows to be integrated in different development environments (e.g. Eclipse) and target processors (e.g. LEON – Sparc V8). The XPY components are: XPY Core – metrics calculation, timing statistics, execution control; XPY DB Interface – interface with the pre-defined Database, XPY Interface – integration with application specific plug-ins, Target System Abstraction Layer – interface layer to other processor specific boundary scan libraries. Other Research Initiatives - XPY

14. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 14 XPY - Monitoring and Profiling Tool for Space Software The XPY external components are described hereafter: GDBIF – access to GDB functionalities in the application environment. MDSProtocol API – MDS JTAG API to the ERC32 XceptionTM XPY Plug-In – XPY GUI as a new XceptionTM Plug-In Other Research Initiatives - XPY

15. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 15 Other Research Initiatives - XPY

16. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 16 Testing Object Orientation (TestOO) Can we relay on Object Oriented Software in Critical Systems? Development a Coding Rules Checker Tool to support a set of coding rules enhancing the testability and verifiability of Object Oriented Software for Critical Systems; General Best Coding Practices, ESA Standard Coding Rules, BSSC(98)3 Issue 1 Ada95 Coding Standards, BSSC 2005(2) Issue 1 Java Coding Standards, Object Oriented guidelines provided within the scope of the project. Checking Ada and JAVA source code for the right use of: General Best Coding Practices, ESA Standard Coding Rules, BSSC(98)3 Issue 1 Ada95 Coding Standards, BSSC 2005(2) Issue 1 Java Coding Standards, Object Oriented guidelines provided within the scope of the project. Other Research Initiatives - TESTOO

17. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 17 Testing Object Orientation (TestOO) TestOO Checker Architecture Modular Architecture Design, Windows XP and Linux OS Portability, Integrates Open Source Tools and Rules DB (DataBase) TestOO Checker Features Generic IDE Layout (GUI) Explorer File System Browser; Profile Manager (Ada and JAVA Profiles / Rules Navigators) Text Editor Output Console Reporting Facilities Command Line Interface (CLI) Other Research Initiatives - TESTOO

18. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 18 Other Research Initiatives - TESTOO

19. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 19 Future Work Evaluation of Xception G-SWFIT Further validation of toolset Development of other fault operators Optimization of pattern search Assessment of disassembling capabilities Assessment of processor architecture and compiler dependencies OK for OSS, needs to be carefully thought for “other” COTS

20. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 20 Future Work SpaceAQua Project Kick Off of SpaceAQua Project (late ’07) Cooperation between CSW, NASA IVV, WVU and CISUC

21. © Copyright Critical Software S.A. 1998-2007 All Rights Reserved. 21 Questions? Thank You! www.criticalsoftware.com VV&RAMS Engineering Unit Engineering Manager Nuno Silva nsilva@criticalsoftware.com Project Manager Ricardo Barbosa rbarbosa@criticalsoftware.com Critical Software SA Critical Software, LimitedCritical Software Technologies Ltd Parque Industrial de Taveiro, Lote 48 3045-504 Coimbra, PORTUGAL Tel: +351 239 989 100 Fax: +351 239 989 119 Pólo Tecnológico de Lisboa, Lote 1, Estrada do Paço do Lumiar 1600-546 Lisboa, PORTUGAL Tel: +351 21 7101192 Fax: +351 21 7101103 Tecmaia - Rua Eng.º Frederico Ulrich, nº 2650 4470-605 Moreira da Maia, Porto, PORTUGAL Tel.: +351 229446927 Fax: +351 229446929 111 North Market Street, Suite 670 San Jose, California, USA, 95113 Tel: +1(408) 9711231 Fax: +1(408) 3513330 Suite 19-21 - 2 Venture Road Southampton Science Park Chilworth - Southampton SO16 7NP - United Kingdom Tel: +44 (0)23 8076 3853