Department of Computer Engineering PROPANE An Environment for Examining the Propagation of Errors in Software Martin Hiller, Arshad Jhumka, Neeraj Suri Chalmers University of Technology Göteborg, Sweden {hiller, arshad,

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Overall Objectives A D C E B F Software reliability can be provided/increased by adding mechanisms that detect and correct data errors  wrappers, assertions, etc. (especially for black-box software) Given a software system and limited resources one would like to concentrate work on the most vulnerable/exposed parts of the software, i.e. … Where do upcoming errors propagate?  Where do upcoming errors propagate?

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Outline Error Propagation & Basic Approach Error Propagation & Basic Approach Examples of results obtained by using PROPANE Examples of results obtained by using PROPANE –Aircraft arrestment system Overview of PROPANE Overview of PROPANE –Tool suite –Requirements & limitations Conclusions & some future directions Conclusions & some future directions

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Error Propagation B F Error = A system state which is different from the state in a ”correct” execution of the system (i.e, not mutations or software defects) Modules in a software system have different levels of exposure and different ”ability” to break error propagation (i.e., different levels of error containment) – examining error propagation gives the developer a picture/profile of these levels. D C E A

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Basic Approach  Error Injection A D C E B F 1.Generate Golden Run, i.e., an error free reference run 2.Generate Injection Run, i.e., a run in which an error (i.e. erroneous system state) is injected 3.Compare Injection Run with Golden Run to see which parts of the system were corrupted by the injected error PROPANE – Propagation Analysis Environment  By instrumenting the target software, PROPANE can, during execution, log individual variables and events and inject errors into individual variables. = probe = injection location

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Example: Aircraft Arrestment System CLOCK CALC DIST_S PRES_A V_REG PRES_S ms_slot_nbr i mscnt pulscnt slow_speed stopped IsValue OutValueTOC2ADC TCNT TIC1 PACNT SetValue Rotation sensor Pressure sensor Pressure valve Computer Pressure valve Tape drum (original) Tape drum (mirror) Cable Target system overview Target software overview

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Results Generated by PROPANE From low level to high level information From low level to high level information –Signal/Variable Plots –Propagation Signatures –Propagation Graphs –Propagation Summaries Data compilations that can be used for further analysis Data compilations that can be used for further analysis

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Signal/Variable Plots One-cycle single-bit upset at t = 1500 ms SetValue erroneous at t = 1539 ms

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Propagation Signatures Each injected error generates a Propagation Signature showing when and where the error propagated Each injected error generates a Propagation Signature showing when and where the error propagated

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Propagation Graphs The Propagation Graph is generated in three different formats –PROPANE native format –GML (Graph Markup Language) – dot (part of the graphviz tool suite from AT&T Research) Probed location Incoming errors from locations earlier in the propagation path Outgoing errors to locations later in the propagation path error count t min / t avg / t max

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Propagation Graph for PACNT Starting point Propagation path Each arc carries information regarding number of propagated errors and propagation time Variables along the propagation trajectory

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Propagation Summary for PACNT Signal Error count Error rate t min t avg t max PACNT pulscnt i SetValue OutValue TOC ADC IsValue slow_speed mscnt ms_slot_nbr TCNT TIC Probed location (variable) For all incoming arcs to one node: Total error count and error rate Total error count and error rate Combined t min / t avg / t max Combined t min / t avg / t max Propagation Graphs and Propagation Summaries indicate which parts of the observed system state that are most vulnerable to the injected errors  perhaps a good location for error detection/recovery mechanisms?

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Error Propagation from PACNT Highest Error Rate Lowest Error Rate ms_slot_nbri mscnt pulscnt slow_speed stopped IsValue OutValue TOC2ADC TCNT TIC1 PACNT SetValue CLOCK PRES_S V_REG PRES_A CALC DIST_S

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Overview

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA PL PROPANE Library Static C Library providing the injection/logging API Static C Library providing the injection/logging API –propane_inject() /* Injects an error */ –propane_log_var() /* Variable probe */ –propane_log_event() /* Event probe */ Target must be instrumented with these functions and linked with the library Target must be instrumented with these functions and linked with the library Experiments are then set up using description files Experiments are then set up using description files – Error types (what errors to inject, e.g., bit-flips) – Error triggers (when to inject errors, e.g., once/periodically) – Active probes (perhaps not all instrumented variables/events have to be logged)

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Instrumentation Example int spherical_volume( double radius ) { double volume; volume = 4.0 * (PI * pow(radius, 3.0)) / 3.0; return volume; } int spherical_volume( double radius ) { double volume; /* Injection location for radius */ propane_inject( IL_SPHERE_VOL, &radius, PROPANE_DOUBLE ); /* Probe the value of radius */ propane_log_var( P_RADIUS, &radius ); volume = 4.0 * (PI * pow(radius, 3.0)) / 3.0; /* Probe the value of volume */ propane_log_var( P_VOLUME, &volume ); return volume; } Original code Instrumented code At this point, instrumentation is unfortunately still a manual task. However, automation is the next step in the development of PROPANE.

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA PSC PROPANE Setup Creator Once the target is instrumented, experiments can be set up Once the target is instrumented, experiments can be set up Given information on injection locations, probes, error types and occurrences, PSC will create Given information on injection locations, probes, error types and occurrences, PSC will create –description files for running experiments –description files for data extraction and analysis

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA PCD PROPANE Campaign Driver After instrumentation and setup, the PCD runs the experiments After instrumentation and setup, the PCD runs the experiments –Invokes instrumented target executable containing the PL and all links to external modules, e.g., environment simulator Provides continuous information on experiment status and remaining work Provides continuous information on experiment status and remaining work User control User control –Pause –Abort –Skip

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA PDE PROPANE Data Extractor Extracts and analyses data from the raw readouts produced during experiments Extracts and analyses data from the raw readouts produced during experiments –Perform Golden Run Comparisons, i.e., compare an injection run with a ”clean” reference run to trace errors, generating a Propagation Signature –Collapse multiple propagation signatures into Propagation Graphs and Propagation Summaries –Prepare/compile data for further external analysis (using e.g. MatLab)

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Requirements & Limitations Programming language Programming language –The library itself is written in C  the language in the target system must be able to interface with C-libraries Environment simulator Environment simulator –Stimuli to the target system must be provided by an environment simulator –PROPANE provides basic interfacing functionality such that control of the EnvSim can be made part of the experiment setup (~plug-in’s) Target instrumentation Target instrumentation –Difficulties with real-time applications that cannot be made to run in simulated time Current implementation focused towards single-node SW Current implementation focused towards single-node SW –Distributed SW not considered at this point

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Conclusions PROPANE enables experimental evaluation of error propagation in SW PROPANE enables experimental evaluation of error propagation in SW –Propagation profiles indicate which parts of system state that are vulnerable  aid for equipping SW with error detection and recovery mechanisms We envision that PROPANE can be used as a design stage aid for analysis of single node SW We envision that PROPANE can be used as a design stage aid for analysis of single node SW Limitations apply for some apsects of real-time and distributed SW Limitations apply for some apsects of real-time and distributed SW

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Some Future Directions Increased automation Increased automation –E.g. automatic instrumentation of target software Extended analysis capabilities Extended analysis capabilities –E.g. provide your own analysis plug-ins Open source Open source –Possibly in the near future

Department of Computer Engineering July 22, 2002 Martin Hiller PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA Further Information (and future download area)