1. What makes software dependable? Martyn Thomas CBE FREng independent consultant / expert witness http://www.thomas-associates.co.uk

2. Software projects are not dependable Standish “Chaos Chronicles” (2004 edition): 18% of projects “failed”; (cancelled before completion) 53% of projects “challenged” (operational, but over budget and/or over time with fewer features or functions than initially specified…) Typical Standish figures: Cost overruns on 43% of projects; and Time overruns on 82% of projects. Oxford University/Computer Weekly 2003 study: 10% of UK projects “failed”; and 75% of UK projects “challenged”.

3. Software products are often not dependable Security vulnerabilities Safety-critical faults Requirements errors Programming mistakes

4. Security vulnerabilities Usually poor programming, such as allowing a program to read over-length data and overwrite itself. Weak languages such as C allow such errors. The Code Red and Slammer worms exploited this sort of error in Microsoft products, and did more than $3B of damage worldwide.

5. Safety Critical Faults Erroneous signal de-activation. Data not sent or lost Inadequate defensive programming with respected to untrusted input data Warnings not sent Display of misleading data Stale values inconsistently treated Undefined array, local data and output parameters

6. More safety critical faults -Incorrect data message formats -Ambiguous variable process update -Incorrect initialisation of variables -Inadequate RAM test -Indefinite timeouts after test failure -RAM corruption -Timing issues - system runs backwards -Process does not disengage when required -Switches not operated when required -System does not close down after failure -Safety check not conducted within a suitable time frame -Use of exception handling and continuous resets -Invalid aircraft transition states used -Incorrect aircraft direction data -Incorrect Magic numbers used -Reliance on a single bit to prevent erroneous operation Errors found in C130J software after certification. Source: Andy German, Qinetiq. Personal communication.

7. Requirements Errors what happened Airbus A320, Warsaw 1993 aircraft landed on wet runway aquaplaned, so brakes didn’t work pilot applied reverse thrust, but disabled why airborne ⇔ disabled airborne ⇔ not WheelPulse ⇔ disabled simplified; for full analysis, see [Ladkin 96]

8. Coding Errors type Alert is (Warning, Caution, Advisory); function RingBell(Event : Alert) return Boolean -- return True for Event = Warning or Event = Caution, -- return False for Event = Advisory is Result : Boolean; begin if Event = Warning then Result := True; elsif Event = Advisory then Result := False; end if; return Result; end RingBell; -- C130J code: Caution returns uninitialised (usually TRUE, as required).

9. What makes a Project dependable? BUSINESS ISSUES The business requirements must be properly understood The business changes and risks must be planned, budgeted and managed competently Requirement changes must be kept under control and budgets and timescales must be adjusted to reflect any changes Stakeholder conflicts must be resolved before the computing project starts

10. What makes a Project dependable? REQUIREMENTS ISSUES The computing requirements must be correctly derived from the business, formalised and analysed for omissions and contradictions Most requirements changes are NOT changes to the business needs, but things that were overlooked when the contract was placed. Every requirements change adds cost and delay, and transfers risk back from the supplier to the customer. Beware Output Based Specifications and Agile Methods. They should only be used where the risks are well understood and accepted by the customer.

11. What makes a Project dependable? COMPUTING ISSUES The computing development and risks are planned and managed competently Strong software engineering methods and tools are used to develop the software and show that it has the required properties. Testing is used to show that the assumptions are correct not that the software is correct!

12. Testing software tells you that the tests work – not that the software works Continuous behaviour means you can interpolate between test results Discrete behaviour means that you can’t!

13. What makes a product dependable? Explicit, unambiguous requirements, with the critical properties clearly identified Development using strong, science-based software engineering Rigorous quality management Auditable evidence that the critical requirements have been met Software built around COTS components will often fail these criteria Cheap, dependable and a good fit to your requirements. Pick two.

14. How do you get the right technical solution to a business requirement? USE AN ARCHITECT! See the Royal Academy of Engineering report on complex IT Systems.

15. Role of the Systems Architect Help the customer to understand the business requirements and possibilities Propose appropriate and technically feasible high- level solutions (architectures) Help resolve stakeholder conflicts and agree requirements and architecture Complete and FORMALISE the technical specification This will eliminate most requirements risk. Manage supplier selection Manage the supply contract for the customer Manage requirement changes Manage the user acceptance phase

16. Then use Correct by Construction development SPARK Implementation INFORMED Design Formal Design Formal Specification Security Properties Proof of Security Properties (Z) Proof of Formal Specification (Z) Refinement Proof of Formal Design (Z) Proof of Security Properties (SPARK Proof) Proof of Functional Properties (SPARK Proof) Static Analysis Assurance Activity Key System Test Specification

17. Highly productive Ada Source Lines Spark annotations LOC/day Core9,93916,56438 Support3,6972,24088

18. Other Statistics from NSA Tokeneer experiment Total effort 260 man days Total cost – $250k Total schedule – 9 months Team – 3 people part-time Testing criterion – 99.99% reliability with 90% degree of confidence Total critical failures – 0 [Yes, zero!]

19. WHY use Correct by Construction S/W Engineering ? “Meets Common Criteria and ITSEC security requirements for EAL5 +” “Produces code more quickly and reliably and at lower cost than traditional methods” “Is commercially supported (ORA Canada, Praxis HIS, Pyrrhus Software, SPRE Inc.)” “Reasonable learning curve” “C by C is proven and practical” These are NSA quotes.

20. CONCLUSION Software projects can be dependable and can deliver dependable software. But this depends on a robust specification and strong correct by construction software engineering. Plan, budget and manage the business changes Use an independent system architect Prefer proof above testing for critical properties

21. Useful references http://www.praxis-his.com/publications/ http://www.raeng.org.uk/news/publications/list/repor ts/Complex_IT_Projects.pdf http://www7.nationalacademies.org/CSTB/project_de pendable.html http://www.ukcrc.org.uk/resource/reports/index.cfm