2. Grand Challenge Problem: Model Check Concurrent Software Edmund M. Clarke Department of Computer Science Carnegie Mellon University

3. Outline of Talk 1.Explain what model checking is 2.Some successes of model checking 3.What makes software different 4.Approaches to software model checking 5.Some of my projects

4. What is Model Checking? “The are Glitch Project” Bad pun for cult movie “The Blair Witch Project” ?

5. Temporal Logic Model Checking Model checking is an automatic verification technique for finite state concurrent systems. Developed independently by Clarke, Emerson, and Sistla and by Queille and Sifakis in early 1980’s. Specifications are written in propositional temporal logic. Verification procedure is an exhaustive search of the state space of the design.

6. Advantages of Model Checking No proofs!!! Fast (compared to other rigorous methods such as theorem proving) Diagnostic counterexamples No problem with partial specifications Logics can easily express many concurrency properties

7. Main Disadvantage State Explosion Problem: Too many processes Data Paths Much progress has been made on this problem recently!

8. Model of computation stst ~ Start ~ Close ~ Heat ~ Error Start ~ Close ~ Heat Error ~ Start Close ~ Heat ~ Error ~ Start Close Heat ~ Error Start Close Heat ~ Error Start Close ~ Heat ~ Error Start Close ~ Heat Error Microwave Oven Example State-transition graph describes system evolving over time. No concurrency in simple examples.

9. Temporal Logic The oven doesn’t heat up until the door is closed. Not heat_up holds until door_closed (~ heat_up) U door_closed

10. Basic Temporal Operators Fp - p holds sometime in the future. Gp - p holds globally in the future. Xp - p holds next time. pUq - p holds until q holds. The symbol “p” is an atomic proposition, e.g. “Device Enabled”.

11. Model Checking Problem Let M be a state-transition graph. Let ƒ be the specification in temporal logic. Find all states s of M such that M, s  ƒ. Efficient Algorithms: CE81, CES83

12. The EMC System Preprocessor Model Checker (EMC) Model Checker (EMC) State Transition Graph 10 4 to 10 5 states State Transition Graph 10 4 to 10 5 states Specification True or Counterexamples

13. Breakthrough! Ken McMillan implemented our model checking algorithm using Binary Decision Diagrams in 1987. Model Checker SMV Now able to handle much larger examples!!

14. Combating the State Explosion Problem Binary Decision Diagrams can be used to represent state transition systems more efficiently. The partial order reduction can be used to reduce the number of states that must be enumerated. Many techniques for alleviating state explosion: –Abstraction. –Compositional reasoning. –Symmetry. –Cone of influence reduction. –Semantic minimization.

15. Model Checker Performance Model checkers today can routinely handle systems with between 100 and 1000 state variables. Systems with 10 120 reachable states have been checked. (Compare approx. 10 78 atoms in universe.) By using appropriate abstraction techniques, systems with an essentially unlimited number of states can be checked.

16. Model Checking Systems There are many successful examples of the use of model checking in hardware and protocol verification. The fact that industry (INTEL, IBM, MOTOROLA) is starting to use model checking is encouraging. Below are some well-known model checkers.

17. Temporal Logic Model Checkers The first two model checkers were EMC (Clarke, Emerson, Sistla) and Caesar (Queille, Sifakis). SMV is the first model checker to use BDDs. Spin uses the partial order reduction to reduce the state explosion problem. Verus and Kronos check properties of real-time systems. HyTech is designed for reasoning about hybrid systems.

18. Notable Examples- IEEE Futurebus + In 1992 Clarke and his students at CMU used SMV to verify the IEEE Future+ cache coherence protocol. They found a number of previously undetected errors in the design of the protocol. This was the first time that formal methods have been used to find errors in an IEEE standard. Although the development of the protocol began in 1988, all previous attempts to validate it were based entirely on informal techniques.

19. Notable Examples-IEEE SCI In 1992 Dill and his students at Stanford used Murphi to verify the cache coherence protocol of the IEEE Scalable Coherent Interface. They found several errors, ranging from uninitialized variables to subtle logical errors. The errors also existed in the complete protocol, although it had been extensively discussed, simulated, and even implemented.

20. Notable Examples-PowerScale In 1995 researchers from Bull and Verimag used LOTOS to describe the processors, memory controller, and bus arbiter of the PowerScale multiprocessor architecture. They identified four correctness requirements for proper functioning of the arbiter. The properties were formalized using bisimulation relations between finite labeled transition systems. Correctness was established automatically in a few minutes using the CÆSAR/ ALDÉBARAN toolbox.

21. Notable Examples - HDLC A High-level Data Link Controller was being designed at AT&T in Madrid in 1996. Researchers at Bell Labs offered to check some properties of the design using the FormalCheck verifier. Within five hours, six properties were specified and five were verified. The sixth property failed, uncovering a bug that would have reduced throughput or caused lost transmissions!

22. Notable Examples PowerPC 620 Microprocessor Richard Raimi used Motorola’s Verdict model checker to debug a hardware laboratory failure. Initial silicon of the PowerPC 620 microprocessor crashed during boot of an operating system. In a matter of seconds, Verdict found a BIU deadlock causing the failure.

23. Notable Examples-Analog Circuits In 1994 Bosscher, Polak, and Vaandrager won a best-paper award for proving manually the correctness of a control protocol used in Philips stereo components. In 1995 Ho and Wong-Toi verified an abstraction of this protocol automatically using HyTech. Later in 1995 Daws and Yovine used Kronos to check all the properties stated and hand proved by Bosscher, et al.

24. Notable Examples-ISDN/ISUP The NewCoRe Project (89-92) was the first application of formal verification in a software project within AT&T. A special purpose model checker was used in the development of the CCITT ISDN User Part Protocol. Five “verification engineers” analyzed 145 requirements. A total of 7,500 lines of SDL source code was verified. 112 errors were found; about 55% of the original design requirements were logically inconsistent.

25. Notable Examples-Edifices In 1995 the Concurrency Workbench was used to analyze an active structural control system to make buildings more resistant to earthquakes. The control system sampled the forces being applied to the structure and used hydraulic actuators to exert countervailing forces. A timing error was discovered that could have caused the controller to worsen, rather than dampen, the vibration experienced during earthquakes.

26. Grand Challenge: Model Check Software ! What makes Software Model Checking different ?

27. What Makes Software Model Checking Different ? Large/unbounded base types: int, float, string User-defined types/classes Pointers/aliasing + unbounded #’s of heap-allocated cells Procedure calls/recursion/calls through pointers/dynamic method lookup/overloading Concurrency + unbounded #’s of threads

28. What Makes Software Model Checking Different ? Templates/generics/include files Interrupts/exceptions/callbacks Use of secondary storage: files, databases Absent source code for: libraries, system calls, mobile code Esoteric features: continuations, self-modifying code Size (e.g., MS Word = 1.4 MLOC)

29. What Does It Mean to Model Check Software? 1.Combine static analysis and model checking Use static analysis to extract a finite state synchronization skeleton K from the program. Then check that K models f (K ² f), where f is the specification of the program. ² Bandera (Kansas State), ² Java PathFinder (NASA Ames)

30. What Does It Mean to Model Check Software? 2.Simulate program along all paths in computation tree ² Source code + backtracking (e.g., Verisoft) ² Source code + symbolic execution + backtracking (e.g., MS/Intrinsa Prefix) 3.Use finite-state machine to look for patterns in control-flow graph [Engler]

31. What Does It Mean to Model Check Software? 4. Design with Finite-State Software Models Finite state software models can act as “missing link” between transition graphs and complex software. ² Statecharts ² Esterel ² Lustre ² OO Software Models

32. Example Statechart for Brake Control BRAKE_CONTROL DYNAMIC DYNAMIC_ PARK_BRAKE_ ABS_ON DYNAMIC_ PARK_BRAKE_ ABS_OFF [VEHICLE_SPEED<5 or WHEELSLIP<=THRESHOLD or (VEHICLE_RUN_MODE and not PARKING_BRAKE_BUTTON and in(ACTUATOR_REGULATION))] [WHEELSLIP>THRESHOLD and in(RR_ACTUATOR_REGULATION) and VEHICLE_SPEED>=5] ACTUATOR ACTUATOR_ REGULATION ACTUATOR_ HOME BRAKE_ LATCHED [VEHICLE_SPEED>=5 and PARKING_BRAKE_BUTTON] [VEHICLE_RUN_MODE and not PARKING_BRAKE_BUTTON] [VEHICLE_RUN_MODE and not PARKING_BRAKE_BUTTON] DYNAMIC PARK_BRAKE_ ABS_ON PARK_BRAKE_ ABS_OFF ACTUATOR ACTUATOR_ REGULATION ACTUATOR_ HOME BRAKE_ LATCHED or (VEHICLE_RUN_MODE and not PARKING_BRAKE_BUTTON and in(RR_ACTUATOR_REGULATION) [VEHICLE_SPEED<5 and PARKING_BRAKE_BUTTON] PARKING_BRAKE_BUTTON] [VEHICLE_RUN_MODE and not PARKING_BRAKE_BUTTON] [VEHICLE_RUN_MODE and not PARKING_BRAKE_BUTTON] [VEHICLE_SPEED<5]

33. New Projects (Joint research with Tom Reps, Somesh Jha, and Helmut Veith) Objective: Develop mechanisms and build tools for establishing properties of concurrent software: e.g. absence of deadlock, correctness of security features, correct response times, etc.

34. Project 1 From Shape-Analysis to Model Checking Obtaining useful models of software in the presence of User-defined types/classes Pointers/aliasing + unbounded #’s of heap-allocated cells Procedure calls/recursion Concurrency + unbounded #’s of threads

35. Shape Analysis: Formalizing “...” Formal: y {r[y],r[x]}{p[y],r[x]} x {p[x]} {r[x]} Informal: x y A list: x y

36. Java Threads Are Heap-Allocated Objects  Thread Analysis  Shape Analysis A memory configuration: thread3 inCritical lock1 isAcquired thread1 atStart thread2 atStart thread4 atStart csLock heldBy

37. An abstract memory configuration: thread inCritical lock1 isAcquired thread ’ atStart csLock heldBy Java Threads Are Heap-Allocated Objects  Thread Analysis  Shape Analysis

38. Here, model checking means: Explore the space of possible transitions among abstract memory configurations Java Threads Are Heap-Allocated Objects  Thread Analysis  Shape Analysis

39. Project 2 Verification of Real Time Embedded Systems Real time embedded systems: Set of tasks, to be scheduled on 1 or more processors Execute periodically Share resources and data Hard deadlines Can be safety critical Want to verify: Schedulability Deadlocks Data flow (variables used before defined...) Sound communication Response times,.....

40. Verification of Real Time Embedded Systems TargetedCompiler WCET analysis of basic blocks assembly code inspection WCET of complex tasks BDD based max. path exploration Schedulability Analysis rate monotonic analysis Model checking RT model checking SMV, UPPAAL, VERUS,... Verification results (counterexample) Embedded System Correctness requirements Statecharts, MATLAB (  temporal logic, RT logic) Global finite state RT model tasks + scheduler + synchronization no recursion, pointers, complex data types,... producer(int p) { bool prod; p = 0; prod = false; periodic(0,10,10) { wait(3); prod = true; p = p+1; wait(1); prod = false; };} main() { int p,c; process producer(p); consumer(c);... producer(int p) { bool prod; p = 0; prod = false; periodic(0,10,10) { wait(3); prod = true; p = p+1; wait(1); prod = false; };} main() { int p,c; process producer(p); consumer(c);... abstract model

41. Project 3 Verifying Security Protocol Interaction Security protocols run on top of other protocols –the SSL record protocol runs on top of TCP Interactions between these protocols can cause subtle bugs Consequence: cannot analyze these protocols in isolation

42. Source code Abstract specification of the protocol stack Model checker capable of handling security and non-security related properties Attacks! Relate back to the code

43. Project 4 Bounded Model-Checking Problem: How to compute set of reachable states? Fixpoint computation is too expensive. Idea: Restrict search to states that are reachable from initial state within fixed number n of transitions Implemented by unwinding program

44. Bounded Model-Checking (cont.) Advantages: + Avoids expensive fixpoint computation + Number n can be adjusted to available computational power + Finds all bugs if program terminates quickly enough + No need for manual loop invariants Disadvantages –Runtime grows with bound, might be too small for large examples

45. Bounded Model-Checking (cont.) void f(...) {... while(cond) { Body; } Rest; } void f(...) {... while(cond) { Body; } Rest; }

46. Bounded Model-Checking (cont.) while() loops are unwound iteratively void f(...) {... if(cond) { Body; while(cond) { Body; } Rest; } void f(...) {... if(cond) { Body; while(cond) { Body; } Rest; }

47. Bounded Model-Checking (cont.) while() loops are unwound iteratively Use a combination of decision procedures to check spec. void f(...) {... if(cond) { Body; if(cond) { Body; while(cond) { Body; } Rest; } void f(...) {... if(cond) { Body; if(cond) { Body; while(cond) { Body; } Rest; }

48. Bounded Model-Checking (cont.) while() loops are unwound iteratively Use a combination of decision procedures to check spec. Assertion may be inserted after last iteration: violated if program runs longer than bound permits void f(...) {... if(cond) { Body; if(cond) { Body; if(cond) { Body; while(cond) { Body; } Rest; } void f(...) {... if(cond) { Body; if(cond) { Body; if(cond) { Body; while(cond) { Body; } Rest; }

49. Bounded Model-Checking (cont.) while() loops are unwound iteratively Use a combination of decision procedures to check spec. Assertion may be inserted after last iteration: violated if program runs longer than bound permits If program terminates “fast enough“, one gets a positive correctness result! void f(...) {... if(cond) { Body; if(cond) { Body; if(cond) { Body; if(cond) { assert(FALSE); } Rest; } void f(...) {... if(cond) { Body; if(cond) { Body; if(cond) { Body; if(cond) { assert(FALSE); } Rest; }

50. Project 5: Alternative Approach: Automated Theorem Proving for Code Proof System Language Modules ANSI-C VHDL PVS Abstraction Data Types Predicate Decision Procedures CMU SMV SAT Presburger Bitvector Arith. Simplex SyMP Proof Manager SyMP User Interface EMacs

51. Project Homepage http://www.cs.cmu.edu/~emc/cip