1. Lifecycle Verification of the NASA Ames K9 Rover Executive Dimitra Giannakopoulou Mike Lowry Corina Păsăreanu (pcorina@email.arc.nasa.gov) Rich Washington Collaborators: Howard Cannon, Ray Garcia – ARA Group Summer Students: Colin Blundell (University of Pennsylvania), Jamie Cobleigh (University of Massachusetts)

2. Objectives Motivation / NASA Relevance: Verification is essential for autonomy insertion into missions Traditional testing of autonomy software is hard due to high complexity and unpredictable environments Integration problems are difficult to detect and expensive to fix; verification should be addressed early in the software lifecycle Objectives: Use a combination of formal analysis techniques (model checking) and testing to analyze autonomous systems throughout their lifecycle Provide support for compositional (“divide and conquer”) verification to address scalability Use design level artifacts to guide the implementation and to enable efficient source code verification Requirements & Design CodingDeployment Compositional Verification Model checking/ Testing Properties, assumptions cost of detecting / fixing bugs increases integration issues handled early

3. Accomplishments/Contributions Development of novel, fully automated compositional verification techniques Integrated lifecycle approach to verification of autonomy software Analysis of K9 Rover Executive (35KLOC of C++) Design level modeling –Design models: describe concurrent architecture and autonomy features –Requirements: concurrency and plan execution properties Design level analysis –Model checking used for exhaustive verification of design models –10x improvement over monolithic (non-compositional) verification Code level analysis –Model checking: 3x improvement over monolithic verification –Advanced testing (automated plan input generation and run time verification) Several integration problems discovered Change in design as a result of our analysis –Simplified architecture with increased modularity

4. Outline Introduction The K9 Rover Executive Compositional Techniques for Design- and Code- Level Verification Analysis of the Rover Executive Conclusions and Future Work

5. Ames K9 Rover Executive Executes flexible plans for autonomy –branching on state / temporal conditions Multi-threaded system –communication through shared variables –synchronization through mutexes and condition variables Architecture changed as a result of our analysis –Simplified inter-thread communication through event queue Original architecture Updated architecture

6. Outline Introduction The K9 Rover Executive Compositional Techniques for Design- and Code- Level Verification Analysis of the Rover Executive Conclusions and Future Work

7. Model Checking for Design Level Analysis OK Finite-state model Specification Verification tool  or Error trace Line 5: … Line 12: … Line 15:… Line 21:… Line 25:… Line 27:… … Line 41:… Line 47:…

8.  Check P on entire system: too many states  Check one component at a time: need abstraction of other modules M2M2 M1M1 Does system made up of M 1 and M 2 satisfy property P? Our work: the first incremental and automated approach for assume-guarantee reasoning Solution 1 (2002) algorithmic generation of assumption as a controller knowledge of environment is not required applied to Rover Executive (ASE’02) (2003) extended to deal with deadlock applied to DS-1 remote agent executive (Journal of ASE, 2003) Solution 2 (2003) incremental assumption computation based on learning and knowledge of environment applied to Rover Executive (TACAS’03, Journal on STTT, submitted October 2003) extended framework for symmetric assume-guarantee rules (SAVCBS’03) Compositional Verification for Increased Scalability finite state models safety properties P holds in system A  Assume-guarantee reasoning: 1. check P on M 1 with assumption A for M 2 2. check that M 2 satisfies A  Assumption generation is a manual process

9. Learning Framework for Automated Compositional Verification P violated Process is iterative We use a learning algorithm to infer the smallest assumption at each stage Assumptions are generated by querying the system, and are gradually refined Queries are answered by model checking Refinement is based on counterexamples obtained by model checking Termination is guaranteed Algorithm is “any-time” Learning Model Checking AiAi real error? counterexample – refine A i M 2 satisfies A i M 1 satisfies P, under A i P holds true false true false NY TECHNOLOGY Extended LTSA verification tool to support automated assume guarantee reasoning at design level

10. PROBLEM  Suppose we have a (finite) design model for a piece of software and safety properties  Can use our automated assume-guarantee reasoning on the design model to verify the safety properties  Can we use this knowledge to improve the performance of verifying the properties on the actual implementation? TECHNOLOGY  Developed methodology for using design-level assumptions for reasoning about source code, to improve scalability For source code verification, we used  Model checking Extended Java PathFinder model checker to support assume-guarantee reasoning at the source code level; applied approach to K9 Rover (ICSE’04)  Advanced testing Automated plan input generation based on plan language grammar Property monitoring (Eagle) Design models Component implementations M1M1 M2M2 A C1C1 C2C2 A P P SOLUTION  Use assumption A generated at the design level to check the implementation C 1 || C 2 in an assume- guarantee style Assume-guarantee Verification of Code with Design-Level Assumptions

11. Testing Framework Input plan & property generation event stream K9 executive instrumentation Observer (Eagle) Input plans Behavioural properties reports instrumentation

12. Outline Introduction The K9 Rover Executive Compositional Techniques for Design- and Code- Level Verification Analysis of the Rover Executive Conclusions and Future Work

13. Analysis of the Executive Properties Concurrency: deadlocks, correct lock usage / data races, sequence of interactions between modules Plan execution properties Analyzed several versions Original architecture: components (threads), locks, condition variables New architecture: queuing and event handling, floating branch execution (execution of synchronous and asynchronous alternate plans) Design level analysis Several synchronization issues discovered (also present in actual implementation) 10x improvement over monolithic (non-compositional) verification Code level analysis Model checking: 3x improvement over monolithic verification Testing –Automatic generation of hundreds of plans in a few seconds –Detection of integration problems early (during unit testing) –Better coverage (predict, by mathematical inference, properties about all possible inter-leavings)

14. Analyzed Properties P1: If the last task in the plan terminates successfully, then the only possible outcome for the plan is successful termination. P2: When a task fails, the continue-on-failure flag on the block will always be checked before any outcome is produced; moreover, if continue-on-failure is true, the outcome is success, otherwise it is failed. P3: The Executive only receives ExecCondChecker events if it has registered for them. P4: The ExecCondChecker only puts events in the queue if the Executive registered for them. P5: When a task fails, it will always check its continue-on-failure flag; moreover, if the continue-on-failure flag is false, no subsequent task in the block will be started; new tasks can be started after the parent block reports the results (i.e. other block is expanded). P6: If a task fails, then the parent block’s continue-on-failure flag will be checked: if it is true, then the block succeeds, otherwise it fails. P7: If the Executive thread reads the value of the shared variable savedWakeupStruct, then the ExexcCondChecker thread should not read it until the Executive clears it first. P8: (Race condition) All accesses to shared structure conditionSetChanged by the Executive and the ExecCondChecker threads will be protected by locks. P9: (Race condition) All accesses to shared structure existConditions by the Executive and the ExecCondChecker threads will be protected by locks. P10: Absence of local and global deadlocks. P11: No irrelevant action execution events can happen. P12: No irrelevant condition checker events can happen. P13: Alternate and principal plans

15. Example property (P7) ExecTimerChecker Database (state of system) ActionExecution DBMonitor Internal Executive ExecCondChecker savedWakeupStruct If the Executive thread reads the value of variable savedWakeupStruct, the ExecCondChecker thread should not read this value unless the Executive clears it first. plan

16. Example Property (P7) 0 error executive.savedWakeupStruct.read[0..1] 1 execCondCh.savedWakeupStruct.read[0..1] executive.savedWakeupStruct.assign[0] execCondCh.savedWakeupStruct.assign[0..1]

17. Generated Assumption Assumption = Q0, Q0 = ( executive.exec.lock -> Q2), Q2 = (executive.exec.unlock -> Q0 | executive.savedWakeupStruct.read[1] -> Q3 | executive.savedWakeupStruct.assign[0] -> Q4 | executive.savedWakeupStruct.read[0] -> Q5), Q3 = ( executive.savedWakeupStruct.read[1] -> Q3 | executive.savedWakeupStruct.assign[0] -> Q4), Q4 = ( executive.exec.unlock -> Q0 | executive.savedWakeupStruct.assign[0] -> Q4 | executive.savedWakeupStruct.read[0] -> Q5), Q5 = ( executive.savedWakeupStruct.assign[0] -> Q4 | executive.savedWakeupStruct.read[0] -> Q5). not displaying sink state Q1

18. Analysis Results for P7 AnalysisToolLOCMonolithic model checking Modular verification Design levelLTSA700 FSP4,672 states541 states Code levelJPF7.2K Java183K states53K states

19. Conclusions and Future Work Lifecycle verification and validation of K9 executive –Modeled and checked key autonomy features at design level –Found error in design (before coding); error fixed by developer –Influenced developer to improve/simplify design Developed testing framework for code verification –Tool for automated input plan generation, property monitoring Developed novel compositional verification techniques for increased scalability Future plans –Verification and validation for future universal executive (PLEXIL) –Automated verification of plans generated by Ames planner