1. A Tutorial on Automated Verification Tevfik Bultan

2. Who are these people and what do they have in common? 2007 Clarke, Edmund M 2007 Emerson, E Allen 2007 Sifakis, Joseph 1996 Pnueli, Amir 1991 Milner, Robin 1980 Hoare, C. Antony R. 1978 Floyd, Robert W 1972 Dijkstra, E. W.

3. Outline Software’s Chronic Crisis Temporal Logics and Model Checking Problem Symbolic Model Checking Automata Theoretic Model Checking Software Verification Using Explicit State Model Checking with Java Path Finder Bounded Model Checking Symbolic Software Model Checking with Predicate Abstraction and Counter-Example Guided Abstraction Refinement

4. Software’s Chronic Crisis Large software systems often: Do not provide the desired functionality Take too long to build Cost too much to build Require too much resources (time, space) to run Cannot evolve to meet changing needs –For every 6 large software projects that become operational, 2 of them are canceled –On the average software development projects overshoot their schedule by half –3 quarters of the large systems do not provide required functionality

5. Software Failures There is a long list of failed software projects and software failures You can find a list of famous software bugs at: http://www5.in.tum.de/~huckle/bugse.html I will talk about two famous and interesting software bugs

6. Ariane 5 Failure A software bug caused European Space Agency’s Ariane 5 rocket to crash 40 seconds into its first flight in 1996 (cost: half billion dollars) The bug was caused because of a software component that was being reused from Ariane 4 A software exception occurred during execution of a data conversion from 64-bit floating point to 16-bit signed integer value –The value was larger than 32,767, the largest integer storable in a 16 bit signed integer, and thus the conversion failed and an exception was raised by the program When the primary computer system failed due to this problem, the secondary system started running. –The secondary system was running the same software, so it failed too!

7. Ariane 5 Failure The programmers for Ariane 4 had decided that this particular velocity figure would never be large enough to raise this exception. –Ariane 5 was a faster rocket than Ariane 4! The calculation containing the bug actually served no purpose once the rocket was in the air. –Engineers chose long ago, in an earlier version of the Ariane rocket, to leave this function running for the first 40 seconds of flight to make it easy to restart the system in the event of a brief hold in the countdown. You can read the report of Ariane 5 failure at: http://www.ima.umn.edu/~arnold/disasters/ariane5rep.html

8. Mars Pathfinder Priority inversion occurs when –a thread that has higher priority is waiting for a resource held by thread with a lower priority Pathfinder contained a data bus shared among multiple threads and protected by a mutex lock Two threads that accessed the data bus were: a high-priority bus management thread and a low-priority meteorological data gathering thread Yet another thread with medium-priority was a long running communications thread (which did not access the data bus) A few days into its mission, NASA’s Mars Pathfinder computer system started rebooting itself –Cause: Priority inversion during preemptive priority scheduling of threads

9. Mars Pathfinder The scenario that caused the reboot was: –The meteorological data gathering thread accesses the bus and obtains the mutex lock –While the meteorological data gathering thread is accessing the bus, an interrupt causes the high-priority bus management thread to be scheduled –Bus management thread tries to access the bus and blocks on the mutex lock –Scheduler starts running the meteorological thread again –Before the meteorological thread finishes its task yet another interrupt occurs and the medium-priority (and long running) communications thread gets scheduled –At this point high-priority bus management thread is waiting for the low- priority meteorological data gathering thread, and the low-priority meteorological data gathering thread is waiting for the medium-priority communications thread –Since communications thread had long-running tasks, after a while a watchdog timer would go off and notice that the high-priority bus management thread has not been executed for some time and conclude that something was wrong and reboot the system

10. Software’s Chronic Crisis Software product size is increasing exponentially –faster, smaller, cheaper hardware Software is everywhere: from TV sets to cell-phones Software is in safety-critical systems –cars, airplanes, nuclear-power plants We are seeing more of –distributed systems –embedded systems –real-time systems These kinds of systems are harder to build Software requirements change –software evolves rather than being built

11. Summary Software’s chronic crisis: Development of large software systems is a challenging task –Large software systems often: Do not provide the desired functionality; Take too long to build; Cost too much to build Require too much resources (time, space) to run; Cannot evolve to meet changing needs

12. What is this?

13. First Computer Bug In 1947, Grace Murray Hopper was working on the Harvard University Mark II Aiken Relay Calculator (a primitive computer). On the 9th of September, 1947, when the machine was experiencing problems, an investigation showed that there was a moth trapped between the points of Relay #70, in Panel F. The operators removed the moth and affixed it to the log. The entry reads: "First actual case of bug being found." The word went out that they had "debugged" the machine and the term "debugging a computer program" was born.

14. Can Model Checking Help The question is: Can the automated verification techniques we have been discussing be used in finding bugs in software systems? Today I will discuss some automated verification techniques that have been successful in identifying bugs

15. State of the art in automated verification: Model Checking What is model checking? –Automated verification technique –Focuses on bug finding rather than proving correctness –The basic idea is to exhaustively search for bugs in software –Has many flavors Explicit-state model checking Symbolic model checking Bounded model checking

16. Model Checking Evolution Earlier model checkers had their own input specification languages –For example Spin, SMV This requires translation of the system to be verified to the input langauge of the model checker –Most of the time these translations are not automated and use ad-hoc simplifications and abstractions More recently several researchers developed tools for model checking programs –These model checkers work directly on programs, i.e., their input language is a programming language –These model checkers use well-defined techniques for restricting the state space or use automated abstraction techniques

17. Explicit-State Model Checking Programs Verisoft from Bell Labs –C programs, handles concurrency, bounded search, bounded recursion. –Uses stateless search and partial order reduction. Java Path Finder (JPF) at NASA Ames –Explicit state model checking for Java programs, bounded search, bounded recursion, handles concurrency. –Uses techniques similar to the techniques used in Spin. CMC from Stanford for checking systems code written in C

18. Symbolic Model Checking of Programs CBMC –This is the bounded model checker we discussed earlier, bounds the loop iterations and recursion depth. –Uses a SAT solver. SLAM project at Microsoft Research –Symbolic model checking for C programs. Can handle unbounded recursion but does not handle concurrency. –Uses predicate abstraction and BDDs.

19. Beyond Model Checking Promising results obtained in the model checking area created a new interest in automated verification Nowadays, there is a wide spectrum of verification/analysis/testing techniques with varying levels of power and scalability –Bounded verification using SAT solvers –Symbolic execution using combinations of decision procedures –Dynamic symbolic execution (aka concolic execution) –Various types of symbolic analysis: shape analysis, string analysis, size analysis, etc.

20. What to Verify? Before we start talking about automated verification techniques, we need to identify what we want to verify It turns out that this is not a very simple question First we will discuss issues related to this question

21. Temporal Logics and Model Checking Problem

22. A Mutual Exclusion Protocol Process 1: while (true) { out: a := true; turn := true; wait: await (!b or !turn); cs: a := false; } || Process 2: while (true) { out: b := true; turn := false; wait: await (!a or turn); cs: b := false; } Two concurrently executing processes are trying to enter a critical section without violating mutual exclusion

23. Reactive Systems: A Very Simple Model We will use a very simple model for reactive systems A reactive system generates a set of execution paths An execution path is a concatenation of the states (configurations) of the system, starting from some initial state There is a transition relation which specifies the next-state relation, i.e., given a state what are the states that can follow that state

24. State Space The state space of a program can be captured by the valuations of the variables and the program counters For our example, we have –two program counters: pc1, pc2 domains of the program counters: {out, wait, cs} –three boolean variables: turn, a, b boolean domain: {True, False} Each state of the program is a valuation of all the variables

25. State Space Each state can be written as a tuple (pc1,pc2,turn,a,b) Initial states: {(o,o,F,F,F), (o,o,F,F,T), (o,o,F,T,F), (o,o,F,T,T), (o,o,T,F,F), (o,o,T,F,T), (o,o,T,T,F), (o,o,T,T,T)} –initially: pc1=o and pc2=o How many states total? 3 * 3 * 2 * 2 * 2 = 72 exponential in the number of variables and the number of concurrent components

26. Transition Relation Transition Relation specifies the next-state relation, i.e., given a state what are the states that can come immediately after that state For example, given the initial state (o,o,F,F,F) Process 1 can execute: out: a := true; turn := true; or Process 2 can execute: out: b := true; turn := false; If process 1 executes, the next state is (w,o,T,T,F) If process 2 executes, the next state is (o,w,F,F,T) So the state pairs ((o,o,F,F,F),(w,o,T,T,F)) and ((o,o,F,F,F),(o,w,F,F,T)) are included in the transition relation

27. Transition Relation The transition relation is like a graph, edges represent the next-state relation (o,o,F,F,F) (o,w,F,F,T) (w,o,T,T,F) (o,c,F,F,T) (w,w,T,T,T)

28. Transition System A transition system T = (S, I, R) consists of –a set of statesS –a set of initial statesI  S –and a transition relationR  S  S A common assumption in model checking –R is total, i.e., for all s  S, there exists s’ such that (s,s’)  R

29. Execution Paths An execution path is an infinite sequence of states x = s 0, s 1, s 2,... such that s 0  I and for all i  0, (s i,s i+1 )  R Notation: For any path x x i denotes the i’th state on the path (i.e., s i ) x i denotes the i’th suffix of the path (i.e., s i, s i+1, s i+2,... )

30. Execution Paths A possible execution path: ((o,o,F,F,F), (o,w,F,F,T), (o,c,F,F,T))  (  means repeat the above three states infinitely many times) (o,o,F,F,F) (o,w,F,F,T) (w,o,T,T,F) (o,c,F,F,T) (w,w,T,T,T)

31. Temporal Logics Pnueli proposed using temporal logics for reasoning about the properties of reactive systems Temporal logics are a type of modal logics –Modal logics were developed to express modalities such as “necessity” or “possibility” –Temporal logics focus on the modality of temporal progression Temporal logics can be used to express, for example, that: –an assertion is an invariant (i.e., it is true all the time) –an assertion eventually becomes true (i.e., it will become true sometime in the future)

32. Temporal Logics We will assume that there is a set of basic (atomic) properties called AP –These are used to write the basic (non-temporal) assertions about the program –Examples: a=true, pc0=c, x=y+1 We will use the usual boolean connectives: , ,  We will also use four temporal operators: Invariant p : G p (aka p) (Globally) Eventually p : F p (aka p)(Future) Next p : X p (aka p)(neXt) p Until q : p U q

33. LTL Properties p... pppppp p ppppq X p G p F p p U q

34. Example Properties mutual exclusion: G (  (pc1=c  pc2=c)) starvation freedom: G(pc1=w  F(pc1=c))  G(pc2=w  F(pc2=c)) Given the execution path: x = ((o,o,F,F,F), (o,w,F,F,T), (o,c,F,F,T))  x |= pc1=o x |= X (pc2=w) x |= F (pc2=c) x |= (  turn) U (pc2=c  b) x |= G (  (pc1=c  pc2=c)) x |= G(pc1=w  F(pc1=c))  G(pc2=w  F(pc2=c))

35. LTL Model Checking Given a transition system T and an LTL property p T |= pifffor all execution paths x in T, x |= p For example: T |=? G (  (pc1=c  pc2=c)) T |=? G(pc1=w  F(pc1=c))  G(pc2=w  F(pc2=c)) Model checking problem: Given a transition system T and an LTL property p, determine if T is a model for p (i.e., if T |=p) Complexity: (|S|+|R|)  2 O(|f|)

36. Linear Time vs. Branching Time In linear time logics we look at the execution paths individually In branching time logics we view the computation as a tree –computation tree: unroll the transition relation s2s1s4s3 Transition SystemExecution PathsComputation Tree s3 s4 s3 s1 s2...... s3 s4 s3 s1 s2...... s3 s4s1......... s4s1...

37. Computation Tree Logic (CTL) In CTL we quantify over the paths in the computation tree We use the same four temporal operators: X, G, F, U However we attach path quantifiers to these temporal operators: –A : for all paths –E : there exists a path We end up with eight temporal operators: –AX, EX, AG, EG, AF, EF, AU, EU

38. CTL Properties s2s1s4s3 Transition SystemComputation Tree s4 s3 s1 s2...... s3 s4s1......... s4s1... pp p p p p s3 |= p s4 |= p s1 |=  p s2 |=  p s3 |= EX p s3 |= EX  p s3 |=  AX p s3 |=  AX  p s3 |= EG p s3 |=  EG  p s3 |= AF p s3 |= EF  p s3 |=  AF  p p p

39. CTL Model Checking Given a transition system T= (S, I, R) and a CTL property p T |= pifffor all initial state s  I, s |= p Model checking problem: Given a transition system T and a CTL property p, determine if T is a model for p (i.e., if T |=p) Complexity: O(|f|  (|S|+|R|)) For example: T |=? AG (  (pc1=c  pc2=c)) T |=? AG(pc1=w  AF(pc1=c))  AG(pc2=w  AF(pc2=c)) Question: Are CTL and LTL equivalent?

40. CTL vs. LTL CTL and LTL are not equivalent –There are properties that can be expressed in LTL but cannot be expressed in CTL For example: FG p –There are properties that can be expressed in CTL but cannot be expressed in LTL For example: AG(EF p) Hence, expressive power of CTL and LTL are not comparable

41. Symbolic Model Checking

42. Temporal Properties  Fixpoints [Emerson and Clarke 80] Here are some interesting CTL equivalences: AG p = p  AX AG p EG p = p  EX EG p AF p = p  AX AF p EF p = p  EX EF p p AU q = q  (p  AX (p AU q)) p EU q = q  (p  EX (p EU q)) Note that we wrote the CTL temporal operators in terms of themselves and EX and AX operators

43. Fixpoint Characterizations Fixpoint CharacterizationEquivalences AG p = y. p  AX y AG p = p  AX AG p EG p = y. p  EX y EG p = p  EX EG p AF p =  y. p  AX y AF p = p  AX AF p EF p =  y. p  EX y EF p = p  EX EF p p AU q =  y. q  (p  AX (y)) p AU q=q  (p  AX (p AU q)) p EU q =  y. q  (p  EX (y)) p EU q = q  (p  EX (p EU q))

44. Least Fixpoint Given a monotonic function F, the least fixpoint  y. F y is the limit of the following sequence (assuming F is  - continuous): , F , F 2 , F 3 ,... If S is finite, then we can compute the least fixpoint using the above sequence

45. EF Fixpoint Computation EF p =  y. p  EX y is the limit of the sequence: , p  EX , p  EX(p  EX  ), p  EX(p  EX(p  EX  )),... which is equivalent to , p, p  EX p, p  EX (p  EX (p) ),...

46. EF Fixpoint Computation s2s1s4s3 p p Start  1 st iteration p  EX  = {s1,s4}  EX(  )= {s1,s4}   ={s1,s4} 2 nd iteration p  EX(p  EX  ) = {s1,s4}  EX({s1,s4})= {s1,s4}  {s3}={s1,s3,s4} 3 rd iteration p  EX(p  EX(p  EX  )) = {s1,s4}  EX({s1,s3,s4})= {s1,s4}  {s2,s3,s4}={s1,s2,s3,s4} 4 th iteration p  EX(p  EX(p  EX(p  EX  ))) = {s1,s4}  EX({s1,s2,s3,s4})= {s1,s4}  {s1,s2,s3,s4} = {s1,s2,s3,s4}

47. EF Fixpoint Computation p EF(p)states that can reach p p EX(p) EX(EX(p))... EF(p)  states that can reach p  p  EX(p)  EX(EX(p)) ... EF(p)

48. Greatest Fixpoint Given a monotonic function F, the greatest fixpoint y. F y is the limit of the following sequence (assuming F is  - continuous): S, F S, F 2 S, F 3 S,... If S is finite, then we can compute the greatest fixpoint using the above sequence

49. EG Fixpoint Computation Similarly, EG p = y. p  EX y is the limit of the sequence: S, p  EX S, p  EX(p  EX S), p  EX(p  EX (p  EX S)),... which is equivalent to S, p, p  EX p, p  EX (p  EX (p) ),...

50. EG Fixpoint Computation s2s1s4s3 pp p Start S = {s1,s2,s3,s4} 1 st iteration p  EX S = {s1,s3,s4}  EX({s1,s2,s3,s4})= {s1,s3,s4}  {s1,s2,s3,s4}={s1,s3,s4} 2 nd iteration p  EX(p  EX S) = {s1,s3,s4}  EX({s1,s3,s4})= {s1,s3,s4}  {s2,s3,s4}={s3,s4} 3 rd iteration p  EX(p  EX(p  EX S)) = {s1,s3,s4}  EX({s3,s4})= {s1,s3,s4}  {s2,s3,s4}={s3,s4}

51. EG Fixpoint Computation EG(p) EG(p) states that can avoid reaching  pp EX(p) EX(EX(p))... EG(p)  states that can avoid reaching  p  p  EX(p)  EX(EX(p)) ...

52. Symbolic Model Checking [McMillan et al. LICS 90] Basic idea: Represent sets of states and the transition relation as Boolean logic formulas Fixpoint computation becomes formula manipulation –pre-condition (EX) computation: Existential variable elimination –conjunction (intersection), disjunction (union) and negation (set difference), and equivalence check Use an efficient data structure for boolean logic formulas –Binary Decision Diagrams (BDDs)

53. Symbolic Pre-condition Computation Remember the function EX : 2 S  2 S which is defined as: EX(p) = { s | (s,s’)  R and s’  p } We can symbolically compute pre as follows EX(p)   V’ R  p[V’ / V] –V : current-state boolean variables –V’ : next-state boolean variables –p[V’ / V] : rename variables in p by replacing current- state variables with the corresponding next-state variables –  V’ f : existentially quantify out all the variables in V’ from f

54. An Extremely Simple Example Variables: x, y: boolean Set of states: S = {(F,F), (F,T), (T,F), (T,T)} S  True Initial condition: I   x   y Transition relation (negates one variable at a time): R  x’=  x  y’=y  x’=x  y’=  y (= means  ) F,T F,F T,T T,F

55. An Extremely Simple Example Given p  x  y, compute EX(p) EX(p)   V’ R  p[V’ / V]   V’ R  x’  y’   V’ (x’=  x  y’=y  x’=x  y’=  y )  x’  y’   V’ (x’=  x  y’=y)  x’  y’  (x’=x  y’=  y)  x’  y’  V’  x  y  x’  y’  x   y  x’  y’  x  y  x   y EX(x  y)   x  y  x   y In other words EX({(T,T)})  {(F,T), (T,F)} F,T F,F T,T T,F

56. An Extremely Simple Example Let’s compute compute EF(x  y) The fixpoint sequence is False, x  y, x  y  EX(x  y), x  y  EX (x  y  EX(x  y)),... If we do the EX computations, we get: False, x  y, x  y   x  y  x   y, True EF(x  y)  True In other words EF({(T,T)})  {(F,F),(F,T), (T,F),(T,T)} F,T F,F T,T T,F 0123 1 2 3

57. An Extremely Simple Example Based on our results, for our extremely simple transition system T=(S,I,R) we have I  EF(x  y) hence: T |= EF(x  y) (i.e., there exists a path from each initial state where eventually x and y both become true at the same time) I  EX(x  y) hence: T |= EX(x  y) (i.e., there does not exist a path from each initial state where in the next state x and y both become true)

58. An Extremely Simple Example Let’s try one more property AF(x  y) To check this property we first convert it to a formula which uses only the temporal operators in our basis: AF(x  y)   EG(  (x  y)) If we can find an initial state which satisfies EG(  (x  y)), then we know that the transition system T, does not satisfy the property AF(x  y)

59. An Extremely Simple Example Let’s compute compute EG(  (x  y)) The fixpoint sequence is True,  x   y, (  x   y)  EX(  x   y), … If we do the EX computations, we get: True,  x   y,  x   y, EG(  (x  y))   x   y Since I  EG(  (x  y))   we conclude that T |= AF(x  y) F,T F,F T,T T,F 012 0 1

60. Symbolic CTL Model Checking Algorithm Translate the formula to a formula which uses the basis –EX p, EG p, p EU q Atomic formulas can be interpreted directly on the state representation For EX p compute the precondition using existential variable elimination as we discussed For EG and EU compute the fixpoints iteratively

61. SMV [McMillan 93] BDD-based symbolic model checker Finite state Temporal logic: CTL Focus: hardware verification –Later applied to software specifications, protocols, etc. SMV has its own input specification language –concurrency: synchronous, asynchronous –shared variables –boolean and enumerated variables –bounded integer variables (binary encoding) SMV is not efficient for integers, but that can be fixed –fixed size arrays

62. SMV Language An SMV specification consists of a set of modules (one of them must be called main) Modules can have access to shared variables Modules can be composed asynchronously using the process keyword Module behaviors can be specified using the ASSIGN statement which assigns values to next values of variables in parallel Module behaviors can also be specified using the TRANS statements which allow specification of the transition relation as a logic formula where next state values are identified using the next keyword

63. Example Mutual Exclusion Protocol Process 1: while (true) { out: a := true; turn := true; wait: await (b = false or turn = false); cs: a := false; } || Process 2: while (true) { out: b := true; turn := false; wait: await (a = false or turn); cs: b := false; } Two concurrently executing processes are trying to enter a critical section without violating mutual exclusion

64. Example Mutual Exclusion Protocol in SMV MODULE process1(a,b,turn) VAR pc: {out, wait, cs}; ASSIGN init(pc) := out; next(pc) := case pc=out : wait; pc=wait & (!b | !turn) : cs; pc=cs : out; 1 : pc; esac; next(turn) := case pc=out : 1; 1 : turn; esac; next(a) := case pc=out : 1; pc=cs : 0; 1 : a; esac; next(b) := b; FAIRNESS running MODULE process2(a,b,turn) VAR pc: {out, wait, cs}; ASSIGN init(pc) := out; next(pc) := case pc=out : wait; pc=wait & (!a | turn) : cs; pc=cs : out; 1 : pc; esac; next(turn) := case pc=out : 0; 1 : turn; esac; next(b) := case pc=out : 1; pc=cs : 0; 1 : b; esac; next(a) := a; FAIRNESS running

65. Example Mutual Exclusion Protocol in SMV MODULE main VAR a : boolean; b : boolean; turn : boolean; p1 : process process1(a,b,turn); p2 : process process2(a,b,turn); SPEC AG(!(p1.pc=cs & p2.pc=cs)) -- AG(p1.pc=wait -> AF(p1.pc=cs)) & AG(p2.pc=wait -> AF(p2.pc=cs)) Here is the output when I run SMV on this example to check the mutual exclusion property % smv mutex.smv -- specification AG (!(p1.pc = cs & p2.pc = cs)) is true resources used: user time: 0.01 s, system time: 0 s BDD nodes allocated: 692 Bytes allocated: 1245184 BDD nodes representing transition relation: 143 + 6

66. Example Mutual Exclusion Protocol in SMV The output for the starvation freedom property: % smv mutex.smv -- specification AG (p1.pc = wait -> AF p1.pc = cs) & AG... is true resources used: user time: 0 s, system time: 0 s BDD nodes allocated: 1251 Bytes allocated: 1245184 BDD nodes representing transition relation: 143 + 6

67. Example Mutual Exclusion Protocol in SMV Let’s insert an error change pc=wait & (!b | !turn) : cs; to pc=wait & (!b | turn) : cs;

68. % smv mutex.smv -- specification AG (!(p1.pc = cs & p2.pc = cs)) is false -- as demonstrated by the following execution sequence state 1.1: a = 0 b = 0 turn = 0 p1.pc = out p2.pc = out [stuttering] state 1.2: [executing process p2] state 1.3: b = 1 p2.pc = wait [executing process p2] state 1.4: p2.pc = cs [executing process p1] state 1.5: a = 1 turn = 1 p1.pc = wait [executing process p1] state 1.6: p1.pc = cs [stuttering] resources used: user time: 0.01 s, system time: 0 s BDD nodes allocated: 1878 Bytes allocated: 1245184 BDD nodes representing transition relation: 143 + 6

69. Symbolic Model Checking with BDDs BDDs are used as a data structure for encoding trust sets of Boolean logic formulas in symbolic model checking One can use BDD-based symbolic model checking for any finite state system using a Boolean encoding of the state space and the transition relation Why are we using symbolic model checking? –We hope that the symbolic representations will be more compact than the explicit state representation on the average –In the worst case we may not gain anything

70. Problems with BDDs The BDD for the transition relation could be huge –Remember that the BDD could be exponential in the number of disjuncts and conjuncts –Since we are using a Boolean encoding there could be a large number of conjuncts and disjuncts The EX computation could result in exponential blow-up –Exponential in the number of existentially quantified variables

71. Heuristics Instead of computing a monolithic BDD for the whole transition system partition the transition relation in order to keep the BDD size small Use good variable ordering in order to keep the BDD sizes small –Use heuristics to find good variable orderings, –Use dynamic variable ordering heuristics that change the variable ordering dynamically if the BDD size grows too much Use other data structures (such as multi-terminal decision diagrams)

72. Counter-Example Generation Remember: Given a transition system T= (S, I, R) and a CTL property p T |= p ifffor all initial state s  I, s |= p Verification vs. Falsification –Verification: Show: initial states  truth set of p –Falsification: Find: a state  initial states  truth set of  p Generate a counter-example starting from that state The ability to find counter-examples is one of the biggest strengths of the model checkers

73. An Example pppp EF(  p)states that can reach  p p EX(  p) EX(EX(  p))... EF(  p)  states that can reach  p   p  EX(  p)  EX(EX(  p)) ... EF(  p) We want to check the property AG(p) We compute the fixpoint for EF(  p) We check if the intersection of the set of initial states I and the truth set of EF(  p) is empty –If it is not empty we generate a counter-example path starting from the intersection I Generate a counter-example path starting from a state here In order to generate the counter-example path, save the fixpoint iterations. After the fixpoint computation converges, do a second pass to generate the counter-example path.

74. Automata Theoretic Model Checking

75. LTL Properties  Büchi automata [Vardi and Wolper LICS 86] Büchi automata: Finite state automata that accept infinite strings –The better known variant of finite state automata accept finite strings (used in lexical analysis for example) A Büchi automaton accepts a string when the corresponding run visits an accepting state infinitely often –Note that an infinite run never ends, so we cannot say that an accepting run ends at an accepting state LTL properties can be translated to Büchi automata –The automaton accepts a path if and only if the path satisfies the corresponding LTL property

76. LTL Properties  Büchi automata G p p pp true F p p pp true G (F p) p The size of the property automaton can be exponential in the size of the LTL formula (recall the complexity of LTL model checking) pp pp p

77. Büchi Automata: Language Emptiness Check Given a Buchi automaton, one interesting question is: –Is the language accepted by the automaton empty? i.e., does it accept any string? A Büchi automaton accepts a string when the corresponding run visits an accepting state infinitely often To check emptiness: –Look for a cycle which contains an accepting state and is reachable from the initial state Find a strongly connected component that contains an accepting state, and is reachable from the initial state –If no such cycle can be found the language accepted by the automaton is empty

78. LTL Model Checking Generate the property automaton from the negated LTL property Generate the product of the property automaton and the transition system Show that there is no accepting cycle in the product automaton (check language emptiness) –i.e., show that the intersection of the paths generated by the transition system and the paths accepted by the (negated) property automaton is empty If there is a cycle, it corresponds to a counterexample behavior that demonstrates the bug

79. LTL Model Checking Example G q Each state is labeled with the propositions that hold in that state Example transition system Property to be verified Negation of the property  G q  F  q Property automaton for the negated property q qq true 2 1 3 p,q qp {q},{p,q} , {p} ,{p},{q}, {p,q} 12 Equivalently

80. Transition System to Buchi Automaton Translation 2 1 3 Each state is labeled with the propositions that hold in that state Example transition systemCorresponding Buchi automaton {p,q} {p} {q} {p,q} {q} i 1 23 p,q qp

81. {p,q} {p} {q} {p,q} {q} {q},{p,q} , {p} ,{p},{q}, {p,q} 1 2 34 12 Product automaton {p,q} {p} {q} {p,q} 1,1 2,1 3,1 4,2 {q} 3,2 {p} Buchi automaton for the transition system (every state is accepting) Property Automaton Accepting cycle: (1,1), (2,1), (3,1), ((4,2), (3,2))  Corresponds to a counter-example path for the property G q

82. SPIN [Holzmann 91, TSE 97] Explicit state model checker Finite state Temporal logic: LTL Input language: PROMELA –Asynchronous processes –Shared variables –Message passing through (bounded) communication channels –Variables: boolean, char, integer (bounded), arrays (fixed size) –Structured data types

83. SPIN Verification in SPIN Uses the LTL model checking approach Constructs the product automaton on-the-fly –It is possible to find an accepting cycle (i.e. a counter- example) without constructing the whole state space Uses a nested depth-first search algorithm to look for an accepting cycle Uses various heuristics to improve the efficiency of the nested depth first search: –partial order reduction –state compression

84. Example Mutual Exclusion Protocol Process 1: while (true) { out: a := true; turn := true; wait: await (b = false or turn = false); cs: a := false; } || Process 2: while (true) { out: b := true; turn := false; wait: await (a = false or turn); cs: b := false; } Two concurrently executing processes are trying to enter a critical section without violating mutual exclusion

85. Example Mutual Exclusion Protocol in Promela #define cs1 process1@cs #define cs2 process2@cs #define wait1 process1@wait #define wait2 process2@wait #define true 1 #define false 0 bool a; bool b; bool turn; proctype process1() { out: a = true; turn = true; wait: (b == false || turn == false); cs: a = false; goto out; } proctype process2() { out: b = true; turn = false; wait: (a == false || turn == true); cs: b = false; goto out; } init { run process1(); run process2() }

86. Property automaton generation % spin -f "! [] (! (cs1 && cs2))“ never { /* ! [] (! (cs1 && cs2)) */ T0_init: if :: ((cs1) && (cs2)) -> goto accept_all :: (1) -> goto T0_init fi; accept_all: skip } % spin -f "!([](wait1 -> <>(cs1)))“ never { /* !([](wait1 -> <>(cs1))) */ T0_init: if :: ( !((cs1)) && (wait1) ) -> goto accept_S4 :: (1) -> goto T0_init fi; accept_S4: if :: (! ((cs1))) -> goto accept_S4 fi; } Concatanate the generated never claims to the end of the specification file Input formula “[]” means G “<>” means F “spin –f” option generates a Buchi automaton for the input LTL formula

87. SPIN “spin –a mutex.spin” generates a C program “pan.c” from the specification file –This C program implements the on-the-fly nested-depth first search algorithm –You compile “pan.c” and run it to the model checking Spin generates a counter-example trace if it finds out that a property is violated

88. %mutex -a warning: for p.o. reduction to be valid the never claim must be stutter-invariant (never claims generated from LTL formulae are stutter-invariant) (Spin Version 4.2.6 -- 27 October 2005) + Partial Order Reduction Full statespace search for: never claim + assertion violations + (if within scope of claim) acceptance cycles + (fairness disabled) invalid end states - (disabled by never claim) State-vector 28 byte, depth reached 33, errors: 0 22 states, stored 15 states, matched 37 transitions (= stored+matched) 0 atomic steps hash conflicts: 0 (resolved) 2.622 memory usage (Mbyte) unreached in proctype process1 line 18, state 6, "-end-" (1 of 6 states) unreached in proctype process2 line 27, state 6, "-end-" (1 of 6 states) unreached in proctype :init: (0 of 3 states)

89. Problems/Heuristics for explicit state model checking State space explosion: Number of states can be exponential in the number of variables and concurrent components Heuristics used by Spin: –On the fly checking: use a depth first search that computes the product of the property automaton and the transition relation while looking for a violation –Bit-state hashing do not store the full state information might skip some unvisited states so it is not sound –Partial order reduction only explore a representative subset of interleavings among the concurrent processes this can be done in a sound manner

90. Software Verification Using Explicit State Model Checking with Java Path Finder

91. Java Path Finder Program checker for Java Properties to be verified –Properties can be specified as assertions static checking of assertions –It can also verify LTL properties Implements both depth-first and breadth-first search and looks for assertion violations statically Uses static analysis techniques to improve the efficiency of the search Requires a complete Java program It can only handle pure Java, it cannot handle native code

92. Java Path Finder, First Version First version –A translator from Java to PROMELA –Use SPIN for model checking Since SPIN cannot handle unbounded data –Restrict the program to finite domains A fixed number of objects from each class Fixed bounds for array sizes Does not scale well if these fixed bounds are increased Java source code is required for translation

93. Java Path Finder, Current Version Current version of the JPF has its own virtual machine: JPF-JVM –Executes Java bytecode can handle pure Java but can not handle native code –Has its own garbage collection –Stores the visited states and stores current path –Offers some methods to the user to optimize verification Traversal algorithm –Traverses the state-graph of the program –Tells JPF-JVM to move forward, backward in the state space, and evaluate the assertion The rest of the slides are on the current version of JPF: W. Visser, K. Havelund, G. Brat, S. Park and F. Lerda. "Model Checking Programs." Automated Software Engineering Journal Volume 10, Number 2, April 2003.

94. Storing the States JPF implements a depth-first search on the state space of the given Java program –To do depth first search we need to store the visited states There are also verification tools which use stateless search such as Verisoft The state of the program consists of –information for each thread in the Java program a stack of frames, one for each method called –the static variables in classes locks and fields for the classes –the dynamic variables (fields) in objects locks and fields for the objects

95. Storing States Efficiently Since different states can have common parts each state is divided to a set of components which are stored separately –locks, frames, fields Keep a pool for each component –A table of field values, lock values, frame values Instead of storing the value of a component in a state store an index at which the component is stored in the table in the state –The whole state becomes an integer vector JPF collapses states to integer vectors using this idea

96. State Space Explosion State space explosion if one of the major challenges in model checking The idea is to reduce the number of states that have to be visited during state space exploration Here are some approaches used to attack state space explosion –Symmetry reduction search equivalent states only once –Partial order reduction do not search thread interleavings that generate equivalent behavior –Abstraction Abstract parts of the state to reduce the size of the state space

97. Symmetry Reduction Some states of the program may be equivalent –Equivalent states should be searched only once Some states may differ only in their memory layout, the order objects are created, etc. –these may not have any effect on the behavior of the program JPF makes sure that the order which the classes are loaded does not effect the state –There is a canonical ordering of the classes in the memory

98. Symmetry Reduction A similar problem occurs for location of dynamically allocated objects in the heap –If we store the memory location as the state, then we can miss equivalent states which have different memory layouts –JPF tries to remedy this problem by storing some information about the new statements that create an object and the number of times they are executed

99. Partial Order Reduction Statements of concurrently executing threads can generate many different interleavings –all these different interleavings are allowable behavior of the program A model checker has to check all possible interleavings that the behavior of the program is correct in all cases –However different interleavings may generate equivalent behaviors In such cases it is sufficient to check just one interleaving without exhausting all the possibilities –This is called partial order reduction

100. class S1 { int x;} class FirstTask extends Thread { public void run() { S1 s1; int x = 1; s1 = new S!(); x = 3; }} class Main { public static void main(String[] args) { FirstTask task1 = new FirstTask(); SecondTask task2 = new SecondTask(); task1.statr(); task2.start(); }} class S2 { int y;} class SecondTask extends Thread { public void run() { S2 s2; int x = 1; s2 = new S2(); x = 3; }} state space search generates 258 states with symmetry reduction: 105 states with partial order reduction: 68 states with symmetry reduction + partial order reduction : 38 states

101. Static Analysis JPF uses following static analysis techniques for reducing the state space: –slicing –partial evaluation Given a slicing criterion slicing reduces the size of a program by removing the parts of the program that have no effect on the slicing criterion –A slicing criterion could be a program point –Program slices are computed using dependency analysis Partial evaluation propagates constant values and simplifies expressions

102. Abstraction vs. Restriction JPF also uses abstraction techniques such as predicate abstraction to reduce the state space Still, in order to check a program with JPF, typically, you need to restrict the domains of the variables, the sizes of the arrays, etc. Abstraction over approximates the program behavior –causes spurious counter-examples Restriction under approximates the program behavior –may result in missed errors If both under and over approximation techniques are used then the resulting verification technique is neither sound nor complete –However, it is still useful as a debugging tool and it is helpful in finding bugs

103. JPF Java Modeling Primitives Atomicity (used to reduce the state space) –beginAtomic(), endAtomic() Nondeterminism (used to model non-determinism caused by abstraction) –int random(int); boolean randomBool(); Object randomObject(String cname); Properties (used to specify properties to be verified) –AssertTrue(boolean cond)

104. Annotated Java Code for a Reader-Writer Lock import gov.nasa.arc.ase.jpf.jvm.Verify; class ReaderWriter { private int nr; private boolean busy; private Object Condr_enter; private Object Condw_enter; public ReaderWriter() { Verify.beginAtomic(); nr = 0; busy=false ; Condr_enter =new Object(); Condw_enter =new Object(); Verify.endAtomic(); } public boolean read_exit(){ boolean result=false; synchronized(this){ nr = (nr - 1); result=true; } Verify.assertTrue(!busy || nr==0 ); return result; } private boolean Guarded_r_enter(){ boolean result=false; synchronized(this){ if(!busy){nr = (nr + 1);result=true;}} return result; } public void read_enter(){ synchronized(Condr_enter){ while (! Guarded_r_enter()){ try{Condr_enter.wait();} catch(InterruptedException e){} }} Verify.assertTrue(!busy || nr==0 ); } private boolean Guarded_w_enter(){…} public void write_enter(){…} public boolean write_exit(){…} };

105. JPF Output >java gov.nasa.arc.ase.jpf.jvm.Main rwmain JPF 2.1 - (C) 1999,2002 RIACS/NASA Ames Research Center JVM 2.1 - (C) 1999,2002 RIACS/NASA Ames Research Center Loading class gov.nasa.arc.ase.jpf.jvm.reflection.JavaLangObjectReflection Loading class gov.nasa.arc.ase.jpf.jvm.reflection.JavaLangThreadReflection ============================== No Errors Found ============================== ----------------------------------- States visited : 36,999 Transitions executed : 68,759 Instructions executed: 213,462 Maximum stack depth : 9,010 Intermediate steps : 2,774 Memory used : 22.1MB Memory used after gc : 14.49MB Storage memory : 7.33MB Collected objects : 51 Mark and sweep runs : 55,302 Execution time : 20.401s Speed : 3,370tr/s -----------------------------------

106. Example Error Trace 1 error found: Deadlock ======================== *** Path to error: *** ======================== Steps to error: 2521 Step #0 Thread #0 Step #1 Thread #0 rwmain.java:4 ReaderWriter monitor=new ReaderWriter(); Step #2 Thread #0 ReaderWriter.java:10 public ReaderWriter( ) { … Step #2519 Thread #2 ReaderWriter.java:71 while (! Guarded_w_enter()){ Step #2520 Thread #2 ReaderWriter.java:73 Condw_enter.wait();

107. Bounded Model Checking

108. Represent sets of states and the transition relation as Boolean logic formulas Instead of computing the fixpoints, unroll the transition relation up to certain fixed bound and search for violations of the property within that bound Transform this search to a Boolean satisfiability problem and solve it using a SAT solver

109. What Can We Guarantee? Note that in bounded model checking we are checking only for bounded paths (paths which have at most k+1 distinct states) –So if the property is violated by only paths with more than k+1 distinct states, we would not find a counter- example using bounded model checking –Hence if we do not find a counter-example using bounded model checking we are not sure that the property holds However, if we find a counter-example, then we are sure that the property is violated since the generated counter- example is never spurious (i.e., it is always a concrete counter-example)

110. Bounded Model Checking: Proving Correctness One can also show that given an LTL property f, if E f holds for a finite state transition system, then E f also holds for that transition system using bounded semantics for some bound k So if we keep increasing the bound, then we are guaranteed to find a path that satisfies the formula –And, if we do not find a path that satisfies the formula, then we decide that the formula is not satisfied by the transition system –Is there a problem here?

111. Proving Correctness We can modify the bounded model checking algorithm as follows: –Start from an initial bound. –If no counter-examples are found using the current bound, increment the bound and try again. The problem is: We do not know when to stop

112. Proving Correctness If we can find a way to figure out when we should stop then we would be able to provide guarantee of correctness. There is a way to define a diameter of a transition system so that a property holds for the transition system if and only if it is not violated on a path bounded by the diameter. So if we do bounded model checking using the diameter of the system as our bound, then we can guarantee correctness if no counter-example is found.

113. Bounded Model Checking What are the differences between bounded model checking and BDD-based symbolic model checking? –In bounded model checking we are using a SAT solver instead of a BDD library –In symbolic model checking we do not unroll the transition relation as in bounded model checking –In bounded model checking we do not execute the iterative fixpoint computations as in symbolic model checking –In symbolic model checking for finite state systems both verification and falsification results are guaranteed In bounded model checking we can only guarantee the falsification results, in order to guarantee the verification results we need to know the diameter of the system

114. Bounded Model Checking A bounded model checker needs an efficient SAT solver –zChaff SAT solver is one of the most commonly used ones Most SAT solvers require their input to be in Conjunctive Normal Form (CNF) –So the final formula has to be converted to CNF Similar to BDD-based symbolic model checking, bounded model checking was also first used for hardware verification More recently, it has been applied to verification of software

115. Bounded Model Checking for Software CBMC is a bounded model checker for ANSI-C programs Handles function calls using inlining Unwinds the loops a fixed number of times Allows user input to be modeled using non-determinism –So that a program can be checked for a set of inputs rather than a single input Allows specification of assertions which are checked using the bounded model checking

116. Loops Unwind the loop n times by duplicating the loop body n times –Each copy is guarded using an if statement that checks the loop condition At the end of the n repetitions an unwinding assertion is added which is the negation of the loop condition –Hence if the loop iterates more than n times in some execution, the unwinding assertion will be violated and we know that we need to increase the bound in order to guarantee correctness A similar strategy is used for recursive function calls –The recursion is unwound up to a certain bound and then an assertion is generated stating that the recursion does not go any deeper

117. A Simple Loop Example x=0; while (x < 2) { y=y+x; x++; } x=0; if (x < 2) { y=y+x; x++; } if (x < 2) { y=y+x; x++; } if (x < 2) { y=y+x; x++; } assert (! (x < 2)) Original codeUnwinding the loop 3 times Unwinding assertion:

118. From Code to SAT After eliminating loops and recursion, CBMC converts the input program to the static single assignment (SSA) form –In SSA each variable appears at the left hand side of an assignment only once –This is a standard program transformation that is performed by creating new variables In the resulting program each variable is assigned a value only once and all the branches are forward branches (there is no backward edge in the control flow graph) CBMC generates a Boolean logic formula from the program using bit vectors to represent variables

119. Another Simple Example x=x+y; if (x!=1) x=2; else x++; assert(x<=3); x 1 =x 0 +y 0 ; if (x 1 !=1) x 2 =2; else x 3 =x 1 +1; x 4 =(x 1 !=1)?x 2 :x 3 ; assert(x 4 <=3); C  x 1 =x 0 +y 0  x 2 =2  x 3 =x 1 +1  (x 1 !=1  x 4 =x 2  x 1 =1  x 4 =x 3 ) P  x 4 <= 3 Check if C   P is satisfiable, if it is then the assertion is violated C   P is converted to boolean logic using a bit vector representation for the integer variables y 0,x 0,x 1,x 2,x 3,x 4 Original codeConvert to static single assignment Generate constraints

120. Bounded Verification Approaches What we have discussed above is bounded verification by bounding the number of steps of the execution. For this approach to work the variable domains also need to be bounded, otherwise we cannot convert the problems to boolean SAT Bounding the execution steps and bounding the data domain are two orthogonal approaches. –When people say bounded verification it may refer to either of these –When people say bounded model checking it typically refers to bounding the execution steps

121. Symbolic Software Model Checking with Predicate Abstraction and Counter-Example Guided Abstraction Refinement

122. Model Checking Programs Using Abstraction Program model checking tools generally rely on automated abstraction techniques to reduce the state space of the system such as: –Abstract interpretation –Predicate abstraction If the abstraction is conservative then, if there is no error in the abstracted program we can conclude that there is no error in the original program In general the problem is to construct a finite state model from the program such that the errors or absence of errors can be demonstrated on the finite state model –Model extraction problem

123. Model Checking Programs via Abstraction Bandera –A tool for extracting finite state models from programs –Uses various abstract domains to map the state space of the program to a finite set of states via abstraction SLAM project at Microsoft Research –Symbolic model checking for C programs –Can handle unbounded recursion but does not handle concurrency –Uses predicate abstraction, counter-example guided abstraction refinement and BDDs

124. Abstraction (A simplified view) Abstraction is an effective tool in verification Given a transition system, we want to generate an abstract transition system which is easier to analyze However, we want to make sure that –If a property holds in the abstract transition system, it also holds in the original (concrete) transition system

125. Abstraction (A simplified view) How do we generate an abstract transition system? Merge states in the concrete transition system (based on some criteria) –This reduces the number of states, so it should be easier to do verification Do not eliminate transitions –This will make sure that the paths in the abstract transition system subsume the paths in the concrete transition system

126. Abstraction (A simplified view) For every path in the concrete transition system, there is an equivalent path in the abstract transition system –If no path in the abstract transition system violate a property, then no path in the concrete system can violate the property Using this reasoning we can verify ACTL, LTL and ACTL* properties in the abstract transition system –If the property holds on the abstract transition system, we are sure that the property holds in the concrete transition system –If the property does not hold in the abstract transition system, then we are not sure if the property holds or not in the concrete transition system

127. Abstraction (A simplified view) If the property does not hold in the abstract transition system, what can we do? We can refine the abstract transition system (split some states that we merged) We have to make sure that the refined transition system is still an abstraction of the concrete transition system Then, we can recheck the property again on the refined transition system –If the property does not hold again, we can refine again

128. Predicate Abstraction An automated abstraction technique which can be used to reduce the state space of a program The basic idea in predicate abstraction is to remove some variables from the program by just keeping information about a set of predicates about them For example a predicate such as x = y maybe the only information necessary about variables x and y to determine the behavior of the program –In that case we can just store a boolean variable which corresponds to the predicate x = y and remove variables x and y from the program –Predicate abstraction is a technique for doing such abstractions automatically

129. Predicate Abstraction Given a program and a set of predicates, predicate abstraction abstracts the program so that only the information about the given predicates are preserved The abstracted program adds nondeterminism since in some cases it may not be possible to figure out what the next value of a predicate will be based on the predicates in the given set One needs an automated theorem prover to compute the abstraction

130. Predicate Abstraction, A Very Simple Example Assume that we have two integer variables x,y We want to abstract the program using a single predicate “x=y” We will divide the states of the program to two: 1.The states where “x=y” is true 2.The states where “x=y” is false, i.e., “x  y” We will then merge all the states in the same set –This is an abstraction –Basically, we forget everything except the value of the predicate “x=y”

131. Predicate Abstraction, A Very Simple Example We will represent the predicate “x=y” as the boolean variable B in the abstract program –“B=true” will mean “x=y” and –“B=false” will mean “x  y” Assume that we want to abstract the following program which contains only one statement: y := y+1

132. Predicate Abstraction, Step 1 Calculate preconditions based on the predicate y := y + 1 {x = y} y := y + 1 {x  y} {x  y + 1} {x = y + 1} precondition for B being false after executing the statement y:=y+1 precondition for B being true after executing the statement y:=y+1 Using our temporal logic notation we can say something like: {x=y+1}  AX{x=y} Again, using our temporal logic notation: {x≠y+1}  AX{x≠y}

133. Predicate Abstraction, Step 2 Use decision procedures to determine if the predicates used for abstraction imply any of the preconditions x = y  x = y + 1 ? No x  y  x = y + 1 ? No x = y  x  y + 1 ? Yes x  y  x  y + 1 ? No

134. Predicate Abstraction, Step 3 Generate abstract code IF B THEN B := false ELSE B := true | false y := y + 1 Predicate abstraction wrt the predicate “x=y” y := y + 1 {x = y} y := y + 1 {x  y} {x  y + 1} {x = y + 1} 1) Compute preconditions x = y  x = y + 1 ? No x  y  x = y + 1 ? No x = y  x  y + 1 ? Yes x  y  x  y + 1 ? No 2) Check implications 3) Generate abstract code

135. Model Checking Push-down Automata A class of infinite state systems for which model checking is decidable Push-down automata: Finite state control + one stack LTL model checking for push-down automata is decidable This may sound like a theoretical result but it is the basis of the approach used in SLAM toolkit for model checking C programs –A program with finite data domains which uses recursion can be modeled as a pushdown automaton –A Boolean program generated by predicate abstraction can be represented as a pushdown automaton

136. Predicate Abstraction + Model Checking Push Down Automata Predicate abstraction combined with results on model checking pushdown automata led to some promising tools –SLAM project at Microsoft Research for verification of C programs –This tool is being used to verify device drivers at Microsoft The main idea: –Use predicate abstraction to obtain finite state abstractions of a program –A program with finite data domains and recursion can be modeled as a pushdown automaton –Use results on model checking push-down automata to verify the abstracted (recursive) program

137. SLAM Toolkit SLAM toolkit was developed to find errors in windows device drivers –Examples in my slides are from the following paper: “The SLAM Toolkit”, Thomas Ball and Sriram K. Rajamani, CAV 2001 Windows device drivers are required to interact with the windows kernel according to certain interface rules SLAM toolkit has an interface specification language called SLIC (Specification Language for Interface Checking) which is used for writing these interface rules The SLAM toolkit instruments the driver code with assertions based on these interface rules

138. A SLIC Specification for a Lock This specification states that KeAcquireSpinLock has to be called before KeReleaseSpinLock is called, and KeAcquireSpinLock cannot be called back to back before a KeReleaseSpinLock is called, and vice versa state { enum { Unlocked=0, Locked=1 } state = Unlocked; } KeAcquireSpinLock.return { if (state == Locked) abort; else state = Locked; } KeReleaseSpinLock.return { if (state == Unlocked) abort; else state = Unlocked; } SLIC specification:

139. A SLIC Specification for a Lock state { enum { Unlocked=0, Locked=1 } state = Unlocked; } KeAcquireSpinLock.return { if (state == Locked) abort; else state = Locked; } KeReleaseSpinLock.return { if (state == Unlocked) abort; else state = Unlocked; } enum { Unlocked=0, Locked=1 } state = Unlocked; void slic_abort() { SLIC_ERROR: ; } void KeAcquireSpinLock_return() { if (state == Locked) slic_abort(); else state = Locked; } void KeReleaseSpinLock_return { if (state == Unlocked) slic_abort(); else state = Unlocked; } SLIC specification: Generated C Code:

140. An Example void example() { do { KeAcquireSpinLock(); nPacketsOld = nPackets; req = devExt->WLHV; if(req && req->status){ devExt->WLHV = req->Next; KeReleaseSpinLock(); irp = req->irp; if(req->status > 0){ irp->IoS.Status = SUCCESS; irp->IoS.Info = req->Status; } else { irp->IoS.Status = FAIL; irp->IoS.Info = req->Status; } SmartDevFreeBlock(req); IoCompleteRequest(irp); nPackets++; } } while(nPackets!=nPacketsOld); KeReleaseSpinLock(); } void example() { do { KeAcquireSpinLock(); A: KeAcquireSpinLock_return(); nPacketsOld = nPackets; req = devExt->WLHV; if(req && req->status){ devExt->WLHV = req->Next; KeReleaseSpinLock(); B: KeReleaseSpinLock_return(); irp = req->irp; if(req->status > 0){ irp->IoS.Status = SUCCESS; irp->IoS.Info = req->Status; } else { irp->IoS.Status = FAIL; irp->IoS.Info = req->Status; } SmartDevFreeBlock(req); IoCompleteRequest(irp); nPackets++; } } while(nPackets!=nPacketsOld); KeReleaseSpinLock(); C: KeReleaseSpinLock_return(); } Instrumented code

141. Boolean Programs After instrumenting the code, the SLAM toolkit converts the instrumented C program to a Boolean program using predicate abstraction The Boolean program consists of only Boolean variables –The Boolean variables in the Boolean program are the predicates that are used during predicate abstraction The Boolean program can have unbounded recursion

142. Boolean Programs decl {state==Locked}, {state==Unlocked} := F,T; void slic_abort() begin SLIC_ERROR: skip; end void KeAcquireSpinLock_return() begin if ({state==Locked}) slic_abort(); else {state==Locked}, {state==Unlocked} := T,F; end void KeReleaseSpinLock_return() begin if ({state == Unlocked}) slic_abort(); else {state==Locked}, {state==Unlocked} := F,T; end enum { Unlocked=0, Locked=1 } state = Unlocked; void slic_abort() { SLIC_ERROR: ; } void KeAcquireSpinLock_return() { if (state == Locked) slic_abort(); else state = Locked; } void KeReleaseSpinLock_return { if (state == Unlocked) slic_abort(); else state = Unlocked; } C Code: Boolean Program:

143. void example() { do { KeAcquireSpinLock(); A: KeAcquireSpinLock_return(); nPacketsOld = nPackets; req = devExt->WLHV; if(req && req->status){ devExt->WLHV = req->Next; KeReleaseSpinLock(); B: KeReleaseSpinLock_return(); irp = req->irp; if(req->status > 0){ irp->IoS.Status = SUCCESS; irp->IoS.Info = req->Status; } else { irp->IoS.Status = FAIL; irp->IoS.Info = req->Status; } SmartDevFreeBlock(req); IoCompleteRequest(irp); nPackets++; } } while(nPackets!=nPacketsOld); KeReleaseSpinLock(); C: KeReleaseSpinLock_return(); } void example() begin do skip; A: KeAcquireSpinLock_return(); skip; if (*) then skip; B: KeReleaseSpinLock_return(); skip; if (*) then skip; else skip; fi skip; fi while (*); skip; C: KeReleaseSpinLock_return(); end C Code: Boolean Program:

144. Abstraction Preserves Correctness The Boolean program that is generated with predicate abstraction is non-deterministic. –Non-determinism is used to handle the cases where the predicates used during predicate abstraction are not sufficient enough to determine which branch will be taken If we find no error in the generated abstract Boolean program then we are sure that there are no errors in the original program –The abstract Boolean program allows more behaviors than the original program due to non-determinism. –Hence, if the abstract Boolean program is correct then the original program is also correct.

145. Counter-Example Guided Abstraction Refinement However, if we find an error in the abstract Boolean program this does not mean that the original program is incorrect. –The erroneous behavior in the abstract Boolean program could be an infeasible execution path that is caused by the non-determinism introduced during abstraction. Counter-example guided abstraction refinement is a technique used to iteratively refine the abstract program in order to remove the spurious counter-example traces

146. Counter-Example Guided Abstraction Refinement The basic idea in counter-example guided abstraction refinement is the following: First look for an error in the abstract program (if there are no errors, we can terminate since we know that the original program is correct) If there is an error in the abstract program, generate a counter-example path on the abstract program Check if the generated counter-example path is feasible using a theorem prover. If the generated path is infeasible add the predicate from the branch condition where an infeasible choice is made to the predicate set and generate a new abstract program.

147. Counter-Example Guided Abstraction Refinement void example() begin do skip; A: KeAcquireSpinLock_return(); skip; if (*) then skip; B: KeReleaseSpinLock_return(); skip; if (*) then skip; else skip; fi skip; fi while (*); skip; C: KeReleaseSpinLock_return(); end Boolean Program: Refined Boolean Program: void example() begin do skip; A: KeAcquireSpinLock_return(); b := T; if (*) then skip; B: KeReleaseSpinLock_return(); skip; if (*) then skip; else skip; fi b := b ? F : *; fi while (!b); skip; C: KeReleaseSpinLock_return(); end (using the predicate (nPackets = npacketsOld)) the boolean variable b represents the predicate (nPackets = npacketsOld)

148. Counter-Example Guided Abstraction Refinement Using counter-example guided abstraction refinement we are iteratively creating more an more refined abstractions This iterative abstraction refinement loop is not guaranteed to converge for infinite domains –This is not surprising since automated verification for infinite domains is undecidable in general The challenge in this approach is automatically choosing the right set of predicates for abstraction refinement –This is similar to finding a loop invariant that is strong enough to prove the property of interest