1. Computational Logic QMUL 26 Mar 04 Verification of Safety Critical Software Nick Tudor tel: +44 1684 894489 email: njtudor@qinetiq.com

2. Computational Logic QMUL 26 Mar 04 The Agenda The NDI Control Law A Path Finding Experiment Benefits Resistance Questions

3. Computational Logic QMUL 26 Mar 04 The NDI Control Law

4. Computational Logic QMUL 26 Mar 04 Control software Example of successful application Verification of autocoded Non-linear Dynamic Inversion Control Laws embedded in Vectored thrust Aircraft Advanced flight Control (VAAC) Harrier

5. Computational Logic QMUL 26 Mar 04 Part of NDI Control Law

6. Computational Logic QMUL 26 Mar 04 Year 1999 One man ; 3 months Used RTW Ada autocoder –Produced 3 procedures, Step, Control Law & End –800 LOC Used manual refinement Interactive proof to discharge the 36 VCs Print out of instructions to ProofPower took ~180 pages

7. Computational Logic QMUL 26 Mar 04 Year 2000 Outstanding MSc Student at the world renowned Computer Science Dept, University of York Modules in the Simulink could be replicated in the autocode –5 Modules –Used packages to get 3 procedures per package –1200 LOC –43 VCs (not proven) Now meant that effort could be divided and system upgraded in modular fashion (modular certification)

8. Computational Logic QMUL 26 Mar 04 Meanwhile – Reverse Engineered Safety Evidence Fortran not used in development for 25 procedures Procedure results for remaining 331 procedures –Positive compliance: 88% –Negative compliance: 2% –Tool problems: 2% –Inconclusive: 7% Verification condition results (16,000 VCs) –Totally automatic proofs: 95.7% –Part-automatic, part-interactive proofs: 3.1% –Unproven: 1.2%

9. Computational Logic QMUL 26 Mar 04 Year 2003 4 people; 1 week Still using RTW Ada autocoder –Produced 8 procedures –850 LOC Used refinement script to drive automatic refinement Automatic proof using Supertac to discharge 94% of 373 VCs (21 remained) –Improvements since then

10. Computational Logic QMUL 26 Mar 04 A Path Finding Experiment

11. Computational Logic QMUL 26 Mar 04 Why do an experiment? The embryonic technique has been applied to experimental control laws (…….and it worked!!) No metrics were gathered, therefore: “How good is it for my project?” No independent assessment by industry or MOD on a real project Safety/certification issues to be addressed Applicability: Safety/non-safety critical?

12. Computational Logic QMUL 26 Mar 04 The Comparison Translation to Simulink {Done in 2001} Manually Code into SPARK Ada Confirmed equivalent Requirement - Fortran Iterate Unit test Autocode/Autoprove 100% pass

13. Computational Logic QMUL 26 Mar 04 Manhours comparison

14. Computational Logic QMUL 26 Mar 04 Conventiona l PRICE-S ROM Comparison Based on one result extrapolated to 1KLOC – Dates are irrelevant

15. Computational Logic QMUL 26 Mar 04 Results Interpretation CAVEAT: THIS IS ONE EXPERIMENT WITH CONSTRAINTS Two separate analysis were carried out on the results: –BAES/York University and PFG SW Cost Forecasting Represents 2 1 / 2 - 4 1 / 2 times faster than existing process for Design, Code & Unit Test (BAES/York) Based on a nominal 1000LOCs, code development effort reduced to 28% (ie 72% savings) (PFG) Typically would expect 0.33 LOC per person per hour; CLawZ is at worst 40 and at best 100 times faster (PFG) Translates to approx 30-40% savings in software life cycle costs (CADMID) (PFG)

16. Computational Logic QMUL 26 Mar 04 Benefits

17. Computational Logic QMUL 26 Mar 04 Model development and proof V&V vs Traditional development and V&V Concept/Req Design Flight Test Rig Tests Proof and limited tests Mathematical Specification, Simulink autocode

18. Computational Logic QMUL 26 Mar 04 Resistance “…is futile” – The Borg Collective

19. Computational Logic QMUL 26 Mar 04 Barriers to be overcome Industrial investment in existing tools, processes, people, training NIH Not C – yet! Certification and tool qualification How do I know I have got the right Simulink……? ….and are safety properties in the Simulink reflected in the code…and can I demonstrate that to certifier?

20. Computational Logic QMUL 26 Mar 04 Proving Properties - Certification G{S}H{S} Property needs to be provable in the code Safety Case

21. Computational Logic QMUL 26 Mar 04 Safety gap Mind the Gap!

22. Computational Logic QMUL 26 Mar 04 Verification of Safety Critical Software Nick Tudor tel: +44 1684 894489 email: njtudor@qinetiq.com Any Questions?