1 Completeness and Complexity of Bounded Model Checking Ed Clarke Daniel Kroening Joel Ouaknine Carnegie Mellon University, Pittsburgh, USA Ofer Strichman Technion, Haifa, Israel

2 Overview  Bounded Model Checking of LTL: the (traditional) syntactic translation scheme  The semantic translation scheme  The Completeness Threshold problem  A solution to the Completeness Threshold problem  The complexity of Bounded Model Checking (2exp)  The complexity gap and how it can be closed

3 Bounded Model Checking ( Biere, Cimatti, Clarke, Zhu, 1999 )  Model checking: is M a model of  (M ²  )?  Bounded Model Checking (BMC): is there a counterexample to M ²  up to a given depth k ?  BMC is widely accepted as a complementary to Model- Checking.

4 Bounded Model Checking ( Biere, Cimatti, Clarke, Zhu, 1999 )  BMC can be performed with SAT (no need to detect fixpoints).  SAT formulation of BMC: Keep k copies of each variable Check if [ M ] k Æ [ :  ] k is satisfiable, where: [ M ] k represents all traces of M up to length k [ :  ] k represents all traces of length up to k that satisfy :  [ :  ] k = (… formulation in next few slides)

5 Generating [  ] k is based on expansion formulas for LTL (Manna & Pnueli): BMC (syntactic) translation ( Biere, Cimatti, Clarke, Zhu, 1999 )

6 The no-loop case (finite traces) Expansion rule BMC translation Base case: k

7 BMC (syntactic) translation ( Biere, Cimatti, Clarke, Zhu, 1999 ) The loop case (infinite traces) Expansion rule BMC translation Base case: l s( i ) = i + 1 if i < k, and l otherwise k =

8 LTL model checking (Vardi-Wolper)  Given M, , construct a Buchi automaton B   LTL model checking: is  : M £ B   empty?  Emptiness checking: is there a path to a loop with an accepting state ? s0s0

9  “Unroll”  k times  Find a witness to Gtrue with the fairness constraint s0s0 A semantic BMC translation (Based on Vardi-Wolper) (Was mentioned by [De-Moura, Rushby, Sorea, 2002] in the context of infinite systems)

10 Advantages of the semantic translation Syntactic TranslationSemantic Translation Size of formula O( k ¢ | M | + k 2 ¢ |  |)O ( k ¢ | M | + k ¢ |  |) Optimizations w.r.t. LTL formulas NoneEfficient Buchi construction from LTL Computing CTOnly for Gp and FpFull LTL

11 Advantages of the semantic translation Syntactic TranslationSemantic Translation Size of formula O( k ¢ | M | + k 2 ¢ |  |)O ( k ¢ | M | + k ¢ |  |) Optimizations w.r.t. LTL formulas NoneEfficient Buchi construction from LTL Computing CTOnly for Gp and FpFull LTL

12 The no-loop case (finite traces) For i  k: For i > k: BMC syntactic translation (Biere, Cimatti, Clarke, Zhu, 1999)

13 Bounded Model Checking k = 0 BMC(M, ,k) yes k++ k ¸ ?k ¸ ? no

14 How big should k be?  For every model M and LTL property  there exists k s.t.  We call the minimal such k the Completeness Threshold ( CT )  Clearly if M ²  then CT = 0  Conclusion: computing CT is at least as hard as model checking

15 The Completeness Threshold  Computing CT is as hard as model checking  The value of CT depends on the model M, the property  and the translation scheme.  Strategy: find over-approximations to CT based on graph theoretic properties of M

16  Diameter d(M) = longest shortest path between any two reachable states.  Recurrence Diameter rd(M) = longest loop-free path between any two reachable states. d(M) = 2 rd(M) = 3  Initialized Diameter d I (M)  Initialized Recurrence Diameter rd I (M) Basic notions…

17 The Completeness Threshold  Theorem: for Gp properties CT = d I (M) ( Biere, Cimatti, Clarke, Zhu, 1999 ) s0s0 pp Arbitrary path  Theorem: for Fp properties CT= rd I (M)+1 (Kroening, Strichman, 2003) s0s0 pp pp pp pp pp  Theorem: for an LTL property  CT = ?

18 Advantages of the semantic translation Syntactic TranslationSemantic Translation Size of formula O( k ¢ | M | + k 2 ¢ |  |)O ( k ¢ | M | + k ¢ |  |) Optimizations w.r.t. LTL formulas NoneEfficient Buchi construction from LTL Computing CTOnly for Gp and FpFull LTL

19 Completeness threshold for LTL  It cannot be longer than rd I (  )+1  It cannot be longer than d I (  ) + d(  )  Result: min(rd I (  )+1, d I (  ) + d(  )) s0s0

20 CT: examples d I (  ) + d(  ) = 6 rd I (  ) + 1= 4 d I (  ) + d(  ) = 2 rd I (  ) + 1= 4 s0s0 s0s0

21 Completeness Threshold for CTL  CTL is modular. It can be analyzed one temporal operator at a time. s0s0 p p EGEFp CT(EG) CT(EF)

22 Completeness Threshold for CTL A tight (?) bound on CT:

23 Computing CT (diameter)  Computing d(  ) symbolically with QBF: find minimal k s.t. for all i,j, if j is reachable from i, it is reachable in k or less steps. k-long path s 0 -- s k+1  Complexity: 2-exp k+1-long path s 0 -- s k+1

24 Computing CT (diameter)  Computing d(  ) explicitly: Generate the graph  Apply Floyd-Warshall (O|  | 3 ) to find shortest paths Find longest among all shortest paths  O(|  | 3 )  exp 3 in the size of the representation of   Why is there a complexity gap (2-exp Vs. exp 3 )? QBF tries in the worst case all paths between every two states. Unlike Floyd-Warshall, QBF does not use transitivity information like:

25 Computing CT (recurrence diameter)  Finding the longest loop-free path in a graph is NP- complete in the size of the graph.  The graph can be exponential in the number of variables.  Conclusion: in practice computing the recurrence diameter is 2-exp in the no. of variables.  Computing rd(y) symbolically with SAT. Find largest k that satisfies: With Sorting Networks: O(n log n)

26 Complexity of BMC CT · (min(rd I (  )+1, d I (  ) + d(  )))  The value of CT can be exponential in the # of state variables.  BMC SAT formula grows linearly with k Conclusion: standard SAT based BMC is worst-case 2-exp

27 The complexity GAP  SAT based BMC is 2-exp in the # state variables.  LTL model checking is 1-exp in the # state variables.  So why use BMC ? Finding bugs when k is small In many cases rd(y) and d(y) are not exponential and are even rather small. SAT, in practice, is very efficient.

28 Closing the complexity gap  Why is there a complexity gap ?  LTL-MC with 2-dfs : dfs1 dfs2  Every state is visited not more than twice

29 Closing the complexity gap  2-dfs Each state is visited not more than twice  SAT Each state can potentially be visited an exponential no. of times, because all paths are explored.

30 Closing the complexity gap (for G p)  Force a static order, following a forward traversal  Each time a state i is fully evaluated (assigned): Prevent the search from revisiting it through deeper paths (by adding conflict clauses) When backtracking from state i, prevent the search from revisiting it in step i If : p i holds stop and return “Counterexample found”

31 Work in progress  Challenges: Formally prove that the restricted version is 1-exp. Remove requirement of static order, and stay 1-exp. Extend to full LTL How to combine logic minimization and template clauses Implementation & experiments

32 Closing the complexity gap  Restricted SAT-BMC for LTL (/symbolic 2-dfs) Force a static order, following a forward traversal Each time a state i is fully evaluated (assigned): Prevent the search from revisiting it through deeper paths, e.g. If (x i Æ : y i ) is a visited state, then for i < j · CT add the following state clause: ( : x j Ç y j ). We denote this clause by Sc i j When backtracking, from state i, prevent the search from revisiting it in step i (add ( : x i Ç y i )). Let last-accepting[i] = index of the last accepting state · i If a conflict arises in step j due to a state-clause SC i j s.t. i · last-accepting[j-1] and SC i i is satisfied, Return (“counterexample found”)

33 Closing the complexity gap  Is ‘1-exp SAT’ better or worse than BMC ?  Bad news: We gave up the main power of SAT: dynamic splitting heuristics. We may generate an exponential no. of added constraints  Good news Single exp. instead of double exp. No need to compute CT. (Instead of pre-computing CT we can maintain a list of states and add their negation ‘when needed’).

34 Closing the complexity gap  Is restricted SAT better or worse than explicit LTL-MC ?  Not clear ! Unlike dfs, SAT has heuristics for progressing. SAT has pruning ability of sets of states

35 Comparing the algorithms… 2-dfs LTL MCRestricted-SAT BMC SAT - BMC TimeEXPEXP 2 2-EXP Memory*EXPEXP 2 EXP GuidanceNoneRestrictedFull PruningStatesSets of states * Assuming the SAT solver restricts the size of its added clauses

36 LTL-MC vs. restricted SAT BMC 2-dfs LTL MCRestricted-SAT BMC SAT - BMC TimeEXPEXP 2 2-EXP MemoryEXP P Shortest CE ‘from below’ Yes Requires CTNo Yes (2-EXP) GuidanceNoneRestrictedFull PruningStatesSets of states

37 LTL-MC vs. restricted SAT BMC 2-dfs LTL MCRestricted-SAT BMC SAT - BMC TimeEXPEXP 2 2-EXP MemoryEXP P Shortest CE ‘from below’ Yes Requires CTNo Yes (2-EXP) GuidanceNoneRestrictedFull PruningStatesSets of states

38 LTL-MC vs. restricted SAT BMC 2-dfs LTL MCRestricted-SAT BMC SAT - BMC TimeEXPEXP 2 2-EXP MemoryEXP P Shortest CE ‘from below’ Yes Requires CTNo Yes (2-EXP) GuidanceNoneRestrictedFull PruningStatesSets of states

39 lk The loop case (infinite traces) i+1 i < k li = k succ(i) = BMC syntactic translation (Biere, Cimatti, Clarke, Zhu, 1999)

40 A semantic translation (Based on the Vardi-Wolper algorithm)  Buchi automata B: h S,S 0, ,F,L i  Let inf(W) be the set of states visited infinite no. of times by a run W  B accepts W iff there exists f 2 F s.t. inf(W) Å f  ;