1. Applications of Craig Interpolation to Model Checking K. L. McMillan Cadence Berkeley Labs

2. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Proof ! Debugging ! Dont have a cow, man…

3. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Interpolation If A B = false, there exists an interpolant A' for (A,B) such that: A A' A' B = false A' refers only to common variables of A,B Example: –A = p q, B = q r, A' = q Interpolants from proofs –given a resolution refutation of A B, A' can be derived in linear time. (Craig,57) (Pudlak,Krajicek,97)

4. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Applications to Model Checking Think of interpolation as a weak form of quantifier elimination (i.e., image comp.). –Let W = sup(A) \ sup(B) –Then W. A is the strongest interpolant for (A,B) –An interpolant is thus a weak approximation of the projection W. A, strong enough to refute B. –Image computation is the most costly aspect of symbolic model checking and predicate abstraction. Interpolation allows us to weaken the image relative to a property we want to prove.

5. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Applications Propositional case –Finite-state model checking using a SAT solver –Very robust method for hardware verification First-order case –Infinite-state model checking using a FO prover. –Verify, for example, parameterized protocols Predicate abstraction –Discover useful predicates for predicate abstraction –Computation of the abstract transition relation

6. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. DPLL SAT solvers Input: propositional formula in clause form Output: –A model of the formula, or –A refutation using resolution steps. Highly efficient –Can handle millions of clauses. –Very effective at reducing proofs to relevant clauses.

7. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Resolution and interpolation Resolution rule: Given a resolution refutation for clause sets (A,B), we can derive an interpolant for (A,B) in linear time. (Pudlak,Krajicek,97) Interpolant is a Boolean circuit whose structure mirrors that of the proof. (A p) ( p B) (A B)

8. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Interpolation rules Interpolant is a circuit that follows structure of the proof. A = (p)( p q)B = ( q r)( r) (p)( p q) (q)(q)( q r) (r)(r)( r) q =q

9. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Interpolation-based MC Classic symbolic model checking iterates the strongest post-condition operator to generate the strongest inductive invariant. –Requires quantifier elimination We can exploit interpolation to compute an weakened post-condition operator that is strong enough to prove a given property. –Allows SAT-only symbolic model checking –Procedure is complete for finite-state systems

10. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Modeling System modeled by a transition constraint a b cp g Notation: Q means "add n primes to the symbols in Q" g = a b p = g c c' = p Model: C = { g = a b, p = g c, c' = p }

11. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Bounded model checking Unfold the model k times: U = C C... C a b cp g a b cp g a b cp g... I F Use SAT solver to check satisfiability of I U F If unsatisfiable: property has no Cex of length k can produce a refutation proof P Biere,et al. TACAS99

12. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Reachability Is there a path (of any length) from I to F satisfying transition constraint C? Reachability fixed point: R 0 = I R i+1 = R i Img(R i,C) R = R i Image operator: Img(P,C) = V'. V. (P(V) C(V,V)) F is reachable iff R F false

13. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Reachability IF R1R1 R2R2... R = I Img(I,C) = R 1 Img(R 1,C) Strongest invariant may be very expensive

14. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Overapproximation An overapproximate image op. is Img' s.t. for all P, Img(P,C) implies Img'(P,C) Overapprimate reachability: R' 0 = I R' i+1 = R' i Img'(R' i,C) R' = R' i Img' is adequate (w.r.t.) F, when –if P cannot reach F, Img(P,C) cannot reach F If Img' is adequate, then –F is reachable iff R' F false

15. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Adequate image PF Img(P,C) Reached from PCan reach F Img(P,C) But how do you get an adequate Img'?

16. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. k-adequate image operator Img' is k-adequate (w.r.t.) F, when –if P cannot reach F, Img(P,C) cannot reach F within k steps Note, if k > diameter, then k-adequate is equivalent to adequate.

17. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Interpolation-based image Idea -- use unfolding to enforce k-adequacy A = P C B = C C C F P F CCCCCCC AB t=0 t=k Let Img'(P) 0 = A', where A' is an interpolant for (A,B)... Img' is k-adequate!

18. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Huh? A A' –Img(P,C) Img'(P,C) A' B = false – Img'(P,C) cannot reach F in k steps Hence Img' is k-adequate overapprox. P F CCCCCCC AB t=0 t=k A' Note: if A,B are consistent, then let Img(P,C) = T.

19. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Intuition A' tells is everything the prover deduced about the image of P in proving it can't reach F in k steps. Hence, A' is in some sense an abstraction of the image relative to the property. P F CCCCCCC AB t=0 t=k A'

20. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Reachability algorithm let k = 0 repeat if I can reach F within k steps, answer reachable R = I while Img'(R,C) F = false R' = Img'(R,C) R if R' = R answer unreachable R = R' end while increase k end repeat

21. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Termination Since k increases at every iteration, eventually k > d, the diameter, in which case Img' is adequate, and hence we terminate. Notes: –don't need to know when k > d in order to terminate –often termination occurs with k << d –depth bound for earlier method (Sheeran et al '00) is "longest simple path", which can be exponentially longer than diameter

22. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Performance v. Localization time, interpolation method time, proof-based abstraction Source: Nina Amla

23. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. k-bound comparison proof-based abstraction, last k interpolation last k

24. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Interpolation v K-induction Run time, k-induction Run time, interpolation

25. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. From finite- to infinite-state Up to now… –Model checking finite state systems –Interpolants for propositional formulas Now we extend to… –Model checking infinite-state systems –Interpolants for first-order formulas Craig 1957 gives a straightforward method that eliminates individual variables by adding quantifiers......but our goal is to eliminate quantifiers!

26. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Quantifier-free interpolants In LIUF, we can obtain quantifier-free interpolants from proofs. Linear inequalities (over rationals) –handle index and pointer arithmetic Uninterpreted functions –model memory and register file contents Here, we will just give the "intuition". For precise rules, see references in the paper.

27. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Linear inequalities Naturally generated by Simplex procedure, Fourier-Motzkin. Interpolants can be computed in linear time (0 x) (0 y) (0 c 1 x + c 2 y) 0 c 1,c 2 Pudlak,1997

28. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Interpolation example A = (0 x-y) (0 y-z-1)B = (0 z-x) (0 y-z-1)(0 z-x) (0 y-x-1)(0 x-y) (0 -1) (0 y-z-1)(0 0) (0 y-z-1)(0 x-y) (0 x-z-1) Just sum the inequalities from A, and you get an interpolant.

29. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Adding EUF Rules for equality and uninterpreted functions –Transitivity –Equality contradiction –Congruence a = b b = c a = c a = b a b a=b f(a)=f(b)

30. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Interpolants and transitivity Prove c = m by transitivity chain Interpolant is [c = e, h = k] –summarizes "subchains" on A side –solves all A variables in terms of common vars A B cd e f g h i j k m

31. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Transitivity rule Apply transitivity to eliminate center variable from interpolant. A B cd e f g h i j k m c = i i = m c = m [c = e, i = h] [i = k] [c = e, h = k]

32. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Equality contradiction rule Use solns for c,i to rewrite disequality. Obtain disequality over common vars. A B cd e f g h i c = i [c = e, i = h] (c i) in A [e h]

33. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Congruence rule Use congruence to derive a soln for f(c). Doesn't work for arity > 1 –can solve this problem, but it's complicated! A B cd e q g h i j k m c = m f(c) = f(m) [c = e, h = k] f(c) f(e) [f(c) = f(e), h = k]

34. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Putting it together Also have rules for: –combining equalities and inequalities. –connecting to Boolean reasoning. This allows us to derive quantifier-free interpolants from proofs in LIUF. Can also extend to... –restricted integer arithmetic –restricted use of select/update Note: quantifiers in A,B still yield quantifiers in the interpolant.

35. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Infinite-state model checking Same as finite-state case, but express transition constraints in LIUF. –now, no termination guarantee Some simple examples... –Safety of N-process "bakery" algorithm –Safety of simplified N-process cache protocol –Fisher's timed mutual exclusion But note, for systems with function symbols in the state, we do sometimes need quantifiers in the invariant...

36. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Quantified interpolants Parameterized systems require universal quantifiers in the transition constraints. –example: array update Idea (Craig, 1957): –Instantiating a quantifier in A yields a quantifier in interpolant A' a[i] = x j. a'(j) = if i=j then x else a(j)

37. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. 2) interpolate... a(z) = 0 a(z) = 1 a'(z) = if z=x then 1 else a(z) a'(z) = if z=y then 1 else a(z) 1) instantiate... Quantifier example That is, instantiation of an A-side quantifier with a B-side variable leads to a quantifier in the interpolant. j. a'(j) = 0 a[x] = 1; a[y] = 1; a(z) = 2 A B 3) quantify... z. a(z) = 0 a(z) = 1

38. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Example: N-process "bakery" N processes p(i) of the following form: j. ticket(i) > ticket(j) j. ticket(j) > ticket(i) state(i) = NC NC C Property: i j. (C(i) C(j)) Invariant derived (roughly): i j. (ticket(i) ticket(j) C(j)) (C(i) C(j))

39. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Predicate Abstraction Given a set of state predicates P, compute the strongest invariant of a program expressible as a Boolean combination of the predicates in P. Approach: –Abstract state to predicate valuation –Compute reachable states of abstraction Used successfully in software model checking –SLAM, BLAST, etc… (Graf and Saidi)

40. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Example A simple transition relation: x = x + 1, y = y Predicates: x=y, x 0 x = x + 1 y = y v x=y, v x 0 ? x=y x 0 x y x 0 Two key problems in predicate abstraction –Choosing the predicates –Computing the abstract transition relation We can apply interpolation to both yes v x=y, v x 0 x=y x 0 no!

41. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Predicate selection in B LAST Use atomic predicates occurring in interpolants Can always express invariant strong enough to exclude this trace. x 1 = ctr 0 ctr 1 = ctr 0 +1 y 1 = ctr 1 x 1 = m 0 y 1 m 0 +1 A B [y 1 =x 1 +1] x:=ctr; ctr := ctr+1; y := ctr; assume x=m; assume y m+1; Program trace we want to refute x 1 = ctr 0 ctr 1 = ctr 0 +1 y 1 = ctr 1 x 1 = m 0 y 1 m 0 +1 Translate into transition constraints Compute interpolants [HJMM,POPL04]

42. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Experiments ProgramLOC* Previous Time New Time Predicates Total Average kbfiltr 12k1m12s3m48s726.5 floppy 17k7m10s25m20s2407.7 diskperf 14k5m36s13m32s14010 cdaudio 18k20m18s23m51s2567.8 parport 61kDNF74m58s7538.1 parclass 138kDNF77m40s3827.2 Windows DDK IRP 22 state * Pre-processed Source: R. Jhala

43. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Transition relation refinement Abstract transition relation: –Let W = sup(C), and let P be the predicates –Then the abstract transition relation is: –Best known approaches to computing C' uses an exponential number of decision procedure calls. –We can use interpolants to approximate this image computation...

44. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Transition interpolants An abstract bounded model checking instance A = C' B = I C' C' C' C' F I F C' AB t=0 t=k The ith transition invariant is T i = A', where A' is an interpolant for (A,B) derived from refutation P. B * Q means hide the quantified variables in Q by renaming

45. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Using transition interpolants From the interpolant properties, we know: –Each T i is only over the state variables v p –The conjunction of the T i 's rules out all bad paths of length k. Abstraction refinement loop: –Start with "true" as abstract transition relation –While abstraction has a bad path (of length k): Do BMC at depth k, abd strengthen the abstract transition relation using the resulting T i 's Eventually, the property is proved, or BMC finds a "real" abstract counterexample.

46. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Comparison to CEX-based method Properties of arrays -- no benchmarks can be handled by Cartesian abstraction

47. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Comparison by # of refinements Interpolation method is choosing more relevant refinements.

48. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Summary Interpolants can be derived from proofs in certain first-order theories, including LIUF. Provides a weak replacement for quantifier elimination in various applications. –Finite-state model-checking with a SAT solver –Infinite-state model checking with decision proc. –Predicate selection –Transition relation approximation Exploit the prover's ability to focus proofs on relevant facts, to construct approximations tailored to properties.

49. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Future work Potential to improve the efficiency of hardware model checking by not modeling at the bit level. Potential to move from control-oriented to data-oriented properties in software model checking. Modify prover to control the atomic predicates in the interpolants. –Relatively compete method of predicate selection

50. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Infinite-state verification Predicate abstraction approach (Graf,Saïdi,97) –Choose a set of predicates to represent state I.e., for bakery: ticket 1 > ticket 0 and ticket 0 > ticket 1 –Transform C into a predicate-state transducer –Interpolants are now strictly Boolean Convergence guaranteed, but may have false negatives Advantages of interpolation approach: –Avoid conversion to a Boolean formula –Avoid building BDDs! –Strong ability to ignore irrelevant predicates

51. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Software model checking Predicate abstraction –requires a judicious choice of predicates –the ability to filter out irrelevant predicates could allow a much larger set to be used Cartesian abstraction –loses correlation between predicates –sometimes too coarse (false negatives) –interpolation-based method will use correlation only if needed for proof -- no need for Cartesian abstraction

52. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Conclusion SAT solvers have the ability: –to generate refutations for bounded reachability –to filter out irrelevant facts. These abilities can be exploited to generate an abstract image operator, using Craig interpolation. This yields a reachability procedure that –operates directly on infinite-state systems –is robust w.r.t. irrelevant facts For these reasons, may be useful for software model checking.

53. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. SAT solvers Find a satisfying assignment to a set of clauses, or prove unsatisfiability. Very effective at finding temporal counterexamples in "bounded model checking. Also some application to proving temporal properties (Sheeran et al '00) but limited capacity.

54. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Refutations Conflict clause generation in SAT solvers is really sequence of resolution steps Given an unsatisfiable problem, solver can produce a refutation –proof of empty clause by resolution steps This talk: –two methods to exploit refutations in unbounded model checking –these have characteristics that may be useful for software model checking

55. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Proof-based abstraction BMC at depth k Cex? done No Cex? Use refutation to choose abstraction MC abstraction done True? False? Increase k

56. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Abstraction (cont) Property: G (c X c) a b cp g Model: C = { g = a b, p = g c, c' = p } ' free variable C' property, C C' C property

57. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Advantage C' may refer to fewer state variables than C –reduction in the state explosion problem But how do we choose the set of constraints in C'?

58. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Proof-based abstraction Let C' be set of constraints used in proof P: C' = { c C | some c i occurs in P } C' admits no counterexample of length k –let U' = C' 0 C' 1... C' k-1 –P is a refutation of I 0 U' F k Model check property on C' –property true for C' implies true for C –else Cex of length k' > k (why?) restart for k = k'

59. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Algorithm BMC C at depth k Cex? done No Cex? Refutation P induces abstraction C' Model check C' done True? Cex of depth k'? let k = k' Notice: MC counterexample is thrown away!

60. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Termination Depth k increases at each iteration Eventually k > d, diameter of C' If k > d, no counterexample is possible In practice, termination uses occurs when k d/2 Usually, diameter C' << diameter of C

61. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Abstraction results solid = original, gray = manual, open = proof-based abstraction

62. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Inference SAT solver seems to be very effective at narrowing down the proof to relevant facts. In most cases, it did better than manual abstraction.

63. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Run times

64. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Run time breakdown solid = BMC time, open = MC time

65. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. IBM GP benchmarks Thanks to Jason Baumgartner

66. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. A (fuzzy) hypothesis Parameterized models allowing no abstraction SAT-based BMC "succeeds" when number of relevant variables is small, and fails otherwise. "success" is BMC for k = diameter of relevant logic ModelMax state vars German protocol42 "swap"21

67. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Industrial benchmarks

68. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Implications Most of the time if bounded model checking succeeds, unbounded model checking also succeeds No need to settle for time bounded result Bounded model checking may be applicable only to localizable properties

69. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. An interpolating decision proc. Convert to CNF form Boolean reasoning done by SAT solver –Generates blocking clauses Clauses necessary to refute satisfying Boolean assignments –Produces refutations using resolution Ground decision procedure –Linear inequalities, equality, uninterpreted functions –Discharges blocking clauses

70. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Cutting planes proofs Also supports equality Interpolants can be computed in linear time Can be extended to handle congruence w.r.t. uninterpreted functions. (0 a) (0 b) (0 c 1 a + c 2 b) 0 c 1,c 2

71. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. Interpolation example A = (0 a-b) (0 b-c-1)B = (0 c-a) (0 b-c-1)(0 c-a) (0 b-a-1)(0 a-b) (0 -1) (0 b-c-1)(0 0) (0 b-c-1)(0 a-b) (0 a-c-1)

72. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification. An interpolating decision proc SAT solver Ground Decision Procedure Blocking clauses proofs (A,B) in CNF Interpolation proof A SAT!