1. Model Checking Large-Scale Software Natasha Sharygina Carnegie Mellon University Software Engineering Institute

2. 2 Outline Model Checking Software Integrated Design and Verification Approach Integrated State Space Reduction Approach Conclusions Directions for Future Work

3. 3 Formal Verification by Model Checking Continuously operating concurrent systems (e.g. operating systems, hardware controllers and network protocols) Ongoing, reactive semantics Non-terminating, infinite computations Manifest non-determinism Temporal Logic Model checking [Clarke,Emerson 81][Queille,Sifakis 82]: Formal model, M (state transition system - finite state machines) Specification, P (temporal logic) Algorithm to check if M satisfies P, M |= P exhaustive enumeration of all states reachable by the system when the design fails to satisfy a desired property, a counterexample is generated

4. 4 State Space Explosion Problem: Size of the state graph can be exponential in size of the program text (both in the number of the program variables and the number of program components) M = M 1 || … || M n If each M i has just 2 local states, potentially 2 n global states Research Directions: State space reduction Principal Approaches: Abstraction (elimination of details irrelevant to verification) Compositional reasoning (reasoning about parts of the system)

5. 5 Application of Model Checking to Hardware Verification Simple data structures are used Systems are modular Mostly finite-state systems System components have well defined interfaces Mostly synchronous execution

6. 6 Application of Model Checking to Software Verification Complex data structures are used Procedural or OO design Non-finite state systems System components do not have well defined interfaces Complex coordination between SW components Synchronous or asynchronous execution

7. 7 Model Checking Software (current practice)  1. Design/Implementation/Testing  2. Modeling/Property Specification Finite-state model extraction Simplifications Restrictions  3. Verification Abstractions Divide-and-conquer techniques (when applicable) Limitations Final (expensive) stage in the program development Consistency between code and model Limited to simplified systems

8. 8 Outline Model Checking Software Integrated Design and Verification Approach Integrated State Space Reduction Approach Conclusions Directions for Future Work

9. 9 Model Checking Software (UT Austin – Bell Labs Project)  3. Verification State space reduction techniques Advantages: Earlier bug detection Direct translation of informal program into formal syntax (no simplifications) Separation of concerns: abstraction of control from data Domain-specific property specification  4. Code Generation (last stage)  1. Executable Design Specifications Abstraction from low-level to high-level operations  2. Modeling/Property Specification Finite-state model extraction

10. 10 xUML: An Executable OO Modeling Language Executable dialect of UML A system consists of interacting sequential programs (class instances) System and class hierarchies Class instances communicate mainly through asynchronous event passing with buffering Behavior of class instances is defined as State Models State Models are extended with state actions Each action is run-to-completion State transitions are enabled by events

11. 11 Executable OO Modeling Language (textual xUML) An xUML system is a parallel composition of individual sequential program, P = p 1 || … || p n, Sample commands of xUML programs: Assignments: x: = exp | x := any{exp 1, …, exp n } Communication: : Generate e i (ID,exp) - Event generation Receive e i (ID,x) - Event consumption Compounds: if then else, while do od, switch

12. 12 A Sample xUML State Model State Action State State Transition Message Type

13. 13 Model Checking xUML Programs (UT-Bell Labs Project) xUML ModelxUML Query xUML-to-S/R Translation S/R ModelS/R Query Model Checking with COSPAN Model Checker S/R QueryCOSPAN Error Track Error Report Generation xUML Level Error Report Data Process InputOutput Legend:

14. 14 COSPAN Model Checker and S/R Automaton Language COSPAN is an  - automata-based model checker and input models and queries are formulated in S/R In S/R, a system is a synchronous parallel composition of processes

15. 15 xUML-to-S/R Model Translation Maps xUML class instances to S/R processes Models asynchrony with synchrony: –An S/R process as global execution scheduler –Message buffers by separate S/R processes Simulates dynamic creation of class instances Bounds infinite state spaces of xUML models Reuses modules from SDLCheck [Kurshan, Levin’01], such as optimization modules that conduct Static Partial Order Reduction

16. 16 xUML Level Query Formulation DECLARE Joint_2_in_Move_EE > $Move_EE; DECLARE Recovery_Called > recovery_status = 1; NEVER (Joint_2_in_Move_EE AND Recovery_Called); Proposition Instantiation of Temporal Template Semantic Constructs of xUML Model

17. 17 Demonstration Property Specification InterfacexUML IDEError Visualizer xUML-to-S/R TranslatorError Report Generator COSPAN Model Checker S/R ModelS/R Query Error ReportError TrackDesigner xUML Model Property

18. 18 Step-by-Step Demonstration Designer Property Specification Interface xUML IDEError Visualizer Error ReportxUML ModelProperty xUML-to-S/R TranslatorError Report Generator Error TrackS/R ModelS/R Query COSPAN Model Checker

19. 19

20. 20

21. 21

22. 22

23. 23

24. 24

25. 25 Step-by-Step Demonstration Designer Property Specification Interface xUML IDEError Visualizer Error ReportxUML ModelProperty xUML-to-S/R TranslatorError Report Generator Error TrackS/R ModelS/R Query COSPAN Model Checker

26. 26

27. 27 Step-by-Step Demonstration Designer Property Specification Interface xUML IDEError Visualizer Error ReportxUML ModelProperty xUML-to-S/R TranslatorError Report Generator Error TrackS/R ModelS/R Query COSPAN Model Checker

28. 28

29. 29

30. 30 Step-by-Step Demonstration Designer Property Specification Interface xUML IDEError Visualizer Error ReportxUML ModelProperty xUML-to-S/R TranslatorError Report Generator Error TrackS/R ModelS/R Query COSPAN Model Checker

31. 31

32. 32 Step-by-Step Demonstration Designer Property Specification Interface xUML IDEError Visualizer Error ReportxUML ModelProperty xUML-to-S/R TranslatorError Report Generator Error TrackS/R ModelS/R Query COSPAN Model Checker

33. 33

34. 34 Step-by-Step Demonstration Designer Property Specification Interface xUML IDEError Visualizer Error ReportxUML ModelProperty xUML-to-S/R TranslatorError Report Generator Error TrackS/R ModelS/R Query COSPAN Model Checker

35. 35

36. 36

37. 37

38. 38 NASA Robot Controller System v EEF

39. 39 NASA Robot Controller System Actual robot control system deployed in space (Canadian arm) Complex functionality (kinematics control, obstacle avoidance, fault tolerance, performance evaluation, etc.) Complex coordination (hand shaking) between functional units Stringent reliability requirements Distributed control xUML framework for design and validation of robot control algorithms Discrete real-time modeling ~800K LOC of textual xUML Abstract interfaces to the computational libraries Computational libraries (C++) ~300K LOC, 150+ classes, 500+ methods of C++ 34 functional properties, 20 implementation properties 6 logical errors were found and reported to NASA v EEF

40. 40 Class Information Model of the Robot Controller Systems

41. 41 xUML Modeling of the NASA Robot Controller System EndEffector xUML program Arm xUML program ||

42. 42 Examples of the Robot Control Properties Control Termination: Eventually the robot control terminates EventuallyAlways(abort_var=1) Safety Operation: If the EndEffector reaches an undesired position than the program terminates prior to a new move of the EndEffector AfterAlwaysUntil(undesired_position =1,ee_reference=1,abort_var=1) Configuration Validity Check: If an instance of the EndEffector class is in the “FollowingDesiredTrajectory” state than the instance of the corresponding Arm class is in the ‘Valid” state Always((ee_reference=1) ->(arm_status=1)) Proper Program Termination: The program terminates when it either completes the task or reaches the state where there is no solution for the fault recovery AlwaysUntil(abort_var=0,end_position=1 OR (recovery_status=1 AND number_joints=1))

43. 43 Outline Model Checking Software Integrated Design and Verification Approach Integrated State Space Reduction Approach Conclusions Directions for Future Work

44. 44 Efficient Model Checking of xUML Programs State Space Reduction Approach: –Exploit design modularity for modular verification –Reduce complexity of the program prior to generation of state graph (by syntactic analysis and program transformation) –Use domain specific knowledge to derive abstractions and other reduction techniques –Use existing state space reduction techniques

45. 45 Integrated State Space Reduction Approach Methodological Approaches Algorithmic Approaches

46. 46 Assume-Guarantee Compositional Reasoning In the assume-guarantee paradigm: each component guarantees properties based on assumptions about other components via proof rules M1 || T2 |= T1 M2 || T1 |= T2 * Circularity during verification of different blocks is broken by induction over time [AbadiLamport95][AlurHenzinger96][HenzingerQadeerRajamani99][Kurshan94][McMillan98][Stark85] A trace is a sequence of states -Composition of FSM (||) is the intersection of traces -Consistency check (|=) is the trace containment check M1 || M2 |= T1 || T2* Constraining environment

47. 47 Scalability of Multi-Process Model Checking Limitation:Properties when used as constraints commonly do not give sufficient details about the verifiable processes Approach: Add abstraction constraints Abstraction constraints ( cf. Kurshan 94, Henzinger 98) are added to the property specifications: M 1 || T2 |= T1 || T 1 abs || …|| T n abs T 1 abs, …, T k abs – abstraction constraints Abstraction constraints specification: - temporal logic formulae constraining the external variables of the verifiable processes - translation of the formulae to processes

48. 48 Limitations Observation: Conventional software structure and behaviors preclude system decomposition and applicability of assume/guarantee reasoning Hardware (Spatial Modularity): Natural division into components System components have well defined interfaces Precisely defined communication protocols Software (Not Spatially Modular): Monolithic designs (even for OO programs) Difficulties in property decomposition No clean interfaces Complex interaction between SW elements Difficulties in identifying environment assumptions Inability to specify meaningful abstraction constraints

49. 49 Design for Verification Approach Enforce software design discipline: apply design constraints analogous to the physical constraints of hardware designs Principle 1: Perform encapsulation at the component level (not an object) Principle 2:Design software components to spatial modularity What to do? Make state spaces of components rigorously disjoint Channel communication through precisely defined interfaces Specify precise communication protocols

50. 50 Assume/Guarantee Reasoning of Software Spatial Designs T4T2 T3 Component 3: Design model GATE Component 1: formal model T1 |= ? Component 1: Design model Component 4: Design model Component 2: Design model Property decomposition: Temporal logic specification M1 || T2 || T2 || T3 |= T1 Environment constraints

51. 51 Assume/Guarantee Reasoning of Software Spatial Designs Component 2: Design model GATE T2 abs Component 1: formal model, M1 T1 |= ? Component 2: formal model, M2 T2 |= ? Component 1: Design model GATE T1 abs Abstraction Constraints High-level environment specs environment constraints abstraction constraints M1 || M2 |= T1 || T2 M1 || T2 |= T1 M2 || T1 |= T2 || T2 abs || T1 abs

52. 52 Verification Statistics: Robot Controller System i(M) Monolithic verification, i – number of joints i(C) Compositional verification of Kinematics component Platform: HP9000 (440MHz) with 6144 MB RAM Memory/Time exhaustion Memory, B Models

53. 53 Abstractions in Model Checking Data Abstraction (abstraction of details irrelevant to verification of a property) Approach: M abs  T  M conc  T To Prove: soundness and completeness Systematic Construction of Abstractions (Predicate Abstraction) [Saidi,Graf 97] : Define an abstraction function as a predicate over concrete data Specify decision procedures to compute a set of abstraction predicates Demonstrate the soundness and completeness of the abstraction Refinement–based abstraction [MSR SLAM Project],[Clarke et. al. ‘00], [Saidi 00], [Visser et. al 00]

54. 54 Abstractions in Model Checking: Limitations Abstraction might be expensive! Complex computation procedures to define a set of the abstraction predicates Memory exhaustion during computation of predicates for large systems and complex properties Behavioral over-approximations caused by the abstraction might lead to the state space explosion Can we do better? Abstract and verify selectively Example: Abstract with respect to a subset of properties (not a single property) Simplify decision procedures Example: Eliminate some predicates from consideration

55. 55 Model Checking Programs with Loops Control intensive systems typically execute loops of control functions. Each decision point in the loop typically depends on a small number of variables and each possible branch is commonly taken for a wide range of data values of these variables. Let X control  X be the control effect variables (variables that determine the control flow). The global state transition graph incorporates all of the loops for all values of the control effect variables.

56. 56 Example of a Program with Loops EndEffector xUML programArm xUML program v EEF ||

57. 57 Control Abstraction Goal: Verify Control Properties (properties defined over states that input events) of programs with loops Approach: Abstract execution of loops from valuation of data - Construct an abstract program which contains all of the control flow paths in the concrete program Atomicity of Program Structure: Basic Block is a sequence of statements which can be entered by an event only at the head statement and which always run to completion.

58. 58 Key Ideas 1.Exploit atomicity of the program structure (not predicates) to identify control flow sequences 2. Compute output ranges of each basic block that has loop predicates 3. Transform control flow predicates of each basic block into a multi-way selector expression (non-deterministic choice) that controls all possible sequences of events. Each output of an abstract basic block is controlled by a single value 4. Perform dependency analysis and source-to-source transformation

59. 59 Loop Abstraction for Model Checking of Control Properties

60. 60 Verification Statistics: Robot Controller Memory/Time exhaustion 6(C) 5(C) i(C) Concrete program, i – number of joints i(A) Abstract program Platform: HP9000 (440MHz) with 6144 MB RAM Memory Models

61. 61 Lessons Learned Integration design and verification environment enables verification of non- trivial software systems –Translation of software into a formal language of model checkers is simplified –Model checker is used as a debugging tool during software design –Software design management enhances the applicability of the existing state space reduction techniques –Testing of design-level specifications is helpful for specification of useful abstractions Integrated state space reduction supports verifying larger systems –State space reduction techniques conducted by the syntactic program transformation are applicable without a change to the model checking tools and can be combined with the existing reduction techniques –Effectiveness of reduction algorithms depends on interaction (combination, order of application, etc.) among reduction algorithms

62. 62 Some Ideas for Future Work Integration of abstraction techniques Combined abstraction approach: syntactic program transformation and reductions of the state transition graphs Systematic design and analysis of refinement procedures for design-level specifications Predictable component assembly Abstraction techniques to check consistency of the component interfaces Integration of model checking and testing techniques Model checking to support interactive simulation during analysis of errors -To track multiple processes -To allow analysis backwards and forward during the execution Testing of data dependent parts of the code which can not be handled by finite-state model checking techniques Engineering of software systems to enable symbolic verification to be efficient