1. 8/9/2005Kestrel Technology LLC Page 1 C Global Surveyor Arnaud Venet Kestrel Technology, LLC 3260 Hillview Avenue Palo Alto, CA 94304 arnaud@kestreltechnology.com Static Analysis of Large NASA Flight Software: Experience, Lessons and Perspectives

2. 8/9/2005Kestrel Technology LLC Page 2 Motivations At the starting point of our study are two questions: Can we achieve the precise verification of pointer- intensive applications automatically? Can we do this for the whole program at once? No existing tool met both requirements We designed and developed C Global Surveyor Context of our study: Conducting research at NASA Ames Available software from the Mars Exploration Program

3. 8/9/2005Kestrel Technology LLC Page 3 Verification of Array Manipulations Arrays are the basic data structures in embedded programs Out-of-bounds array access: One of the most common runtime errors One the most difficult to trace back double a[10]; for (i = 0; i < 10; i++) a[i] =...; if (...) a[i] =...; 0 <= i < 10 i = 10

4. 8/9/2005Kestrel Technology LLC Page 4 Roadmap 1.The structure of flight software for Mars missions 2.Initial design of C Global Surveyor 3.Reviewing the design of the analyzer 4.Experiments on existing flight codes 5.What next?

5. 8/9/2005Kestrel Technology LLC Page 5 Roadmap 1.The structure of flight software for Mars missions 2.Initial design of C Global Surveyor 3.Reviewing the design of the analyzer 4.Experiments on existing flight codes 5.What next?

6. 8/9/2005Kestrel Technology LLC Page 6 The MPF Family Mars Path Finder (MPF): Experimental mission for testing new technologies (airbag landing) New software architecture Subsequent missions shared the architecture and programming style inherited from MPF: Mars Path Finder: 140 KLOC, 20 threads Deep Space 1 (DS1): 280 KLOC, 40 threads Mars Exploration Rovers (MER): 550 KLOC, 100 threads

7. 8/9/2005Kestrel Technology LLC Page 7 Object-Oriented Design assign (double *p, double *q, int n) { int i; for (i = 0; i < n; i++) p[i] = q[i]; } assign (&A, &B, 10)assign (&pS->f, &A[2], m) 10...1000 call sites Thousands of such functions Almost all of them contain loops

8. 8/9/2005Kestrel Technology LLC Page 8 Runtime Structure Thread Queue Heap Queue Shallow Large

9. 8/9/2005Kestrel Technology LLC Page 9 Roadmap 1.The structure of flight software for Mars missions 2.Initial design of C Global Surveyor 3.Reviewing the design of the analyzer 4.Experiments on existing flight codes 5.What next?

10. 8/9/2005Kestrel Technology LLC Page 10 Design Choices Symbolic information (access paths) is bulky and difficult to mix with numerical information (array indices) All-numerical representation Context-sensitivity is required We can’t afford performing 1000 fixpoint iterations with widening and narrowing for a single function Compute a summary of the function using a relational numerical lattice &S.f[2][3]  &S + offset( f ) + 2 * size(row) + 3 * size(elem)

11. 8/9/2005Kestrel Technology LLC Page 11 Design Choices The structure of the memory graph is shallow and stable over time Use Steensgaard & Das’ pointer analysis Precision is required for loop invariants and array indices Convex polyhedra have exponential complexity Use Difference-Bound Matrices: O(n 3 ) Relevant numerical information is mostly carried by function parameters Abstract away all integers in the heap

12. 8/9/2005Kestrel Technology LLC Page 12 Memory Graph Construction Abstract Heap (sound approxim ation) thr1 f thr2 init g Refined Abstract Heap (sound approxim ation) READWRITE ITERATE

13. 8/9/2005Kestrel Technology LLC Page 13 Database Equations for file1.c Equations for file2.c Cluster of machines Analyze function f Analyze function g Distributed Architecture PostgreSQL PVM

14. 8/9/2005Kestrel Technology LLC Page 14 Roadmap 1.The structure of flight software for Mars missions 2.Initial design of C Global Surveyor 3.Reviewing the design of the analyzer 4.Experiments on existing flight codes 5.What next?

15. 8/9/2005Kestrel Technology LLC Page 15 First Experiments The execution times were very long (tens of hours) The difference-bound matrices were large and dense The cubic time complexity was always attained The memory graph was very large and imprecise: A lot of pointers were transmitted between threads through message queues The approximation of message queues by Steensgaard’s analysis was too coarse

16. 8/9/2005Kestrel Technology LLC Page 16 CGS Tune-Up Adaptive clustering of variables in difference-bound matrices: Variables are grouped in small-size packets (average size: 4) Packets are dynamically constructed during the analysis Significant speedup (15 min  5 sec for a function) Extending Das one-level flow optimization to an arbitrary depth within data structures: Spectrum of pointer analyses between Steensgaard and Andersen Depth 3 analysis was sufficient to recover enough precision

17. 8/9/2005Kestrel Technology LLC Page 17 Roadmap 1.The structure of flight software for Mars missions 2.Initial design of C Global Surveyor 3.Reviewing the design of the analyzer 4.Experiments on existing flight codes 5.What next?

18. 8/9/2005Kestrel Technology LLC Page 18 Performance Results Overall precision: 80% of all array accesses statically checked for MPF, DS1 and MER Performances: Over 100 KLOC/hour for MPF and DS1 20 hours for MER Main issue: Massive amount of artifacts clogs up the database The database architecture is difficult to optimize (B- trees) A standard relational database is not adequate

19. 8/9/2005Kestrel Technology LLC Page 19 Impact of Parallelization

20. 8/9/2005Kestrel Technology LLC Page 20 Main Conclusions Experiments conducted on dual-processor machines Significant speedup when the network is not used, negligible otherwise Main source of imprecision: important data passing across low-level structures Message queues EEPROM Recovering a high-level abstraction from a low-level representation is extremely difficult

21. 8/9/2005Kestrel Technology LLC Page 21 Experiments with CGS CGS is currently used at: JPL Marshall Space Center Ames Research Center It has been applied to a variety of codes including: The Advanced Video Guidance Sensor (Shuttle) The Boot Loader for the Shuttle engine controller The Urine Processor Assembly of the ISS The Habitat Holding Rack (ISS) The Materials Science Research Rack (ISS)

22. 8/9/2005Kestrel Technology LLC Page 22 Roadmap 1.The structure of flight software for Mars missions 2.Initial design of C Global Surveyor 3.Reviewing the design of the analyzer 4.Experiments on existing flight codes 5.What next?

23. 8/9/2005Kestrel Technology LLC Page 23 Static Analysis at the Spec Level SpecsCode Static Analysis Implementation Synthesis Refinement Code Certification Functional Validation

24. 8/9/2005Kestrel Technology LLC Page 24 Model-Centric Safety-Critical Java for Exploration (NASA ESMD) DSL SC Java Handwritten Java Power Management Guidance & Control … Verification of System Requirements Static Analysis Verification of Real-Time Requirements Static Analysis Provably Correct Code Generation

25. 8/9/2005Kestrel Technology LLC Page 25 Whole System Analysis Model of the Evironment Model of the User Model of the System Static Analysis System-Level verification Automated test generation System reengineering …

26. 8/9/2005Kestrel Technology LLC Page 26 More Information Online papers MXJ Project: “Model-Centric Safety-Critical Java for Exploration” Visit our web site: www.kestreltechnology.com