1. Concrete Model Checking with Abstract Matching and Refinement Corina Păsăreanu QSS, NASA Ames Research Center Radek Pelánek Masaryk University, Brno, Czech Republic Willem Visser RIACS/USRA, NASA Ames Research Center

2. Introduction Abstraction in software verification –Used to reduce data domains of a program –Described as abstract interpretation –Classic approach: over-approximation SLAM, Blast, Magic; see also Bandera, Feaver –Preserves true results –Abstract counter-examples may be infeasible –Counter-example based iterative refinement Our approach: Under-approximation based abstraction with refinement –Goal: error detection –Explores only feasible system behaviors –Preserves errors of safety properties –Iterative refinement based on checking “exactness” of abstraction

3. Predicate Abstraction Maps a (possibly infinite state) concrete transition system into a finite state system –Via a set of predicates: Preds = {p 1, p 2 … p n } Abstraction function  : ConcreteStates → BitVectors  (s) = b 1 b 2 …b n, b i =1  s╞ p i Traditional approaches: May abstract transitions: –Over-approximate concrete transitions –a 1 → may a 2 :  s 1 s.t.  (s 1 )= a 1 and  s 2 s.t.  (s 2 )=a 2, s.t. s 1 → s 2 Must abstract transitions: –Under-approximate concrete transitions –a 1 → must a 2 :  s 1 s.t.  (s 1 )= a 1,  s 2 s.t.  (s 2 )=a 2 and s 1 → s 2 Compute may/must transitions automatically: –Use a theorem prover/decision procedure: require 2 n × n × 2 calls

4. Our Approach Concrete search with abstract matching: Traverse the concrete system For each explored concrete state –Store abstract version of the state –Use predicate abstraction Abstract state used to determine if the search should continue or backtrack Does not build abstract transitions –It executes the concrete transitions directly Decision procedure invoked during refinement: –At most 2 calls for each explored transition

5. Example Concrete system A,0 D,1D,0 E,1E,2 B,1C,0

6. Example Concrete system May abstraction p = (x < 2) Over-approximation A,p D,p E,pE,!p B,pC,p A,0 D,1D,0 E,1E,2 B,1C,0

7. Example Concrete system May abstraction p = (x < 2) Over-approximation A,p D,p E,pE,!p B,pC,p A,0 D,1D,0 E,1E,2 B,1C,0 abstraction 

8. Example Concrete system May abstraction p = (x < 2) Over-approximation A,p D,p E,pE,!p B,pC,p A,0 D,1D,0 E,1E,2 B,1C,0 Must abstraction p = (x < 2) Under-approximation A,p D,p B,pC,p

9. p Example Concrete system May abstraction p = (x < 2) Over-approximation Must abstraction p = (x < 2) Under-approximation !pp p A,0 B,1D,1E,2 A,p D,p B,pC,p A,p D,p E,pE,!p B,pC,p A,0 D,1D,0 E,1E,2 B,1C,0 Concrete search w/ abstract matching p = (x < 2)

10. p Example Concrete system May abstraction p = (x < 2) Over-approximation Must abstraction p = (x < 2) Under-approximation !pp p A,0 B,1D,1E,2 A,p D,p B,pC,p A,p D,p E,pE,!p B,pC,p A,0 D,1D,0 E,1E,2 B,1C,0 Concrete search w/ abstract matching p = (x < 2) p C,0 D,0

11. Example Concrete system May abstraction p = (x < 2) Over-approximation Must abstraction p = (x < 2) Under-approximation !p p pp p A,0 B,1 C,0 D,1 D,0 E,2 A,p D,p B,pC,p A,p D,p E,pE,!p B,pC,p A,0 D,1D,0 E,1E,2 B,1C,0 Concrete search w/ abstract matching p = (x < 2) abstraction not “exact”

12. Example Concrete system May abstraction p = (x < 2) Over-approximation Must abstraction p = (x < 2) Under-approximation !p p pp p A,0 D,1 D,0 E,2 B,1 C,0 A,p D,p B,pC,p A,p D,p E,pE,!p B,pC,p A,0 D,1D,0 E,1E,2 B,1C,0 p,!q !p,!q p,q A,0 D,1 D,0 E,2 B,1 C,0 E,2 p,!q Concrete search w/ abstract matching p = (x < 2) Concrete search w/ abstract matching p = (x < 2) q = (x < 1) abstraction not “exact” refine

13. (Concrete) Search Algorithm PROCEDURE dfs() BEGIN add(s 0, VisitedStates); push(s 0, Stack); WHILE ! empty(Stack) DO s = pop(Stack); FOR all transitions t enabled in s DO s' = successor(s, t); IF s' NOT IN VisitedStates THEN add(s', VisitedStates); push(s', Stack); FI; OD; END;

14. Search w/ Abstract Matching PROCEDURE  Search ( Preds ) BEGIN add(α Preds (s 0 ), VisitedStates); push(s 0, Stack); WHILE ! empty(Stack) DO s = pop(Stack); FOR all transitions t enabled in s DO s' = successor(s, t); IF α Preds (s') NOT IN VisitedStates THEN add(α Preds (s'), VisitedStates); push(s', Stack); FI; OD; END;

15. wp(x>0,x=x-1) x>0 A,2 B,1 x=x-1 wp(x>0,x=x+1) Abstraction Refinement Check if abstraction is exact with respect to each transition t: s  s’ Check if the induced abstract transition is a must transition w/ a decision procedure If not, add new predicates Use weakest precondition calculations  (s)  wp(  (s’),t) Abstraction is exact x>0 A,2 B,3 x=x+1 Abstraction is refined Add new predicate (x-1>0) from failed check and repeat  Search  x+1>0 True  x-1>0 False

16. Iterative Refinement Check if bad state  err is reachable BEGIN Preds =  ; WHILE true DO  Search(Preds); /* during  Search perform: IF  err is reachable THEN output counterexample FI; check if abstraction is exact for each transition NewPreds = newly generated predicates from failed checks */ IF NewPreds =  THEN output unreachable FI; Preds = Preds ∪ NewPreds; OD; END;

17. Correctness and Termination In general –The iterative algorithm might not terminate If it terminates –It finds an error or –It computes a finite bisimilar structure Refinement is non-monotonic If a finite (reachable) bisimulation quotient exists then –It will eventually compute a finite bisimilar structure –May still fail to terminate

18. Pathological Example A: x = 0; y = 0; B: while (y >= 0) C: y = x + y; D: A,p,q B,p,q C,p,q p = (y ≥ 0) q = (x+y ≥ 0) A,p,q,r B,p,q,r C,p,q,r p = (y ≥ 0) q = (x+y ≥ 0) r = (2x+y ≥ 0) … The algorithm may not terminate even for systems with finite (reachable) states Example (check “D is unreachable”): Heuristic to enforce termination p = (y ≥ 0) A,p B,p C,p y = x + y

19. Implementation Implementation for simple guarded command language –PERL, OCAML –Uses SIMPLIFY as a decision procedure Working on integrating the technique in Java PathFinder (JPF) model checker –Supports symbolic execution - using the Omega Library –Use forward (instead of backward) refinement –Handle dynamic data structures

20. Applications Property verification for the Bakery mutual exclusion protocol –Search order matters 5 iterations for breadth first search order 4 iterations for depth first search order Error detection in RAX (Remote Agent Executive) –Component extracted from an embedded spacecraft-control application –Deadlocked in space –Error found faster than over-approximation based analysis Test input generation for Java container classes using JPF –Abstract matching but no refinement –Explore method call sequences –Match states between calls to avoid generation of redundant tests –Better testing coverage as compared to other methods

21. Related Work Refinement of under-approximations –For SAT based bounded model checking – Grumberg et al. [POPL’05] May and must abstractions –Branching time properties – Godefroid et al [Concur’01] –“Hyper” must transitions for monotonicity – Shoham and Grumberg [TACAS’04] –Dams and Namjoshi, de Alfaro et al [LICS’04], Ball et al [CAV’05] –Our previous work – choice free search [TACAS’01] Model driven software verification –Use abstraction mappings during concrete model checking – Holzmann and Joshi [SPIN’04] Over-approximation based predicate abstraction

22. Conclusions Model checking algorithm –Under-approximation refinement –Integrates abstract analysis with concrete program execution –Uses decision procedure to detect incompleteness of abstraction and to refine the abstraction Comparison with standard over-approximation abstraction –Finds errors faster (potentially) –More efficient (in the number of theorem prover calls) –Complementary, should be combined Prototype implementation –Used for the analysis of concurrent programs Future work –Complex properties (e.g. liveness), backward vs. forward refinement, –Evaluation, better criteria for termination

23. Applications Property verification for the Bakery mutual exclusion protocol –Search order matters 5 iterations for breadth first search order 4 iterations for depth first search order IterationConcrete states Abstract states New predicates 11711{x≤y} 21812{x+1≤y, x≤y+1, y≥0} 32619{x+2≤y, y≥1, x≤1} 44432{y≤1, x≤0, y≥2} 54836 

24. Applications Error detection in RAX (Remote Agent Executive) –Component extracted from an embedded spacecraft- control application –Deadlocked in space –Error found in one iteration (BFS) –Error found faster than over-approximation based analysis

25. Applications RAX: also tried to compute bisimilar structure Algorithm does not terminate –Keeps refining unreachable state space Added two new predicates: e1 ≥ 0, e2 ≥ 0 –Algorithm terminates after 3 iterations IterationConcrete states Abstract states New predicates 15635{c1=e1, c2=e2} 26844{e1=0, e2=0} 310065{e1=-1, e2=-1} 410065{e1=-2, e2=-2} 510065…

26. Test input generation for Java container classes using JPF –Abstract matching but no refinement –Explore method call sequences –Match states between calls to avoid generation of redundant tests Better testing coverage as compared to other methods Coverage highly dependent on abstraction Automatic refinement essential Applications

27. Test input generation for Java containers –Abstract matching but no refinement –Match states between API calls –Better testing coverage as compared to other methods –Coverage highly dependent on abstraction –Automatic refinement essential Applications SUT (Java container) Env (M, N, A) M = seq. length of API calls [0,N) = range of input values A = specifies the abstraction API put remove … Model checking Explore call sequences Measure coverage Generate test cases Test case: put(1); put(0); remove(0);

28. Non-monotonic Refinement Refinement Matched State space explored

29. Non-monotonic Refinement Refinement

30. Explanation Bisimulation: symmetric relation ~ –s ~ s’ iff for every s → s 1 there exists s’→ s 1 ’ s.t. s 1 ~ s 1 ’ Two transition systems are bisimilar if –Their initial states are bisimilar ~ induces a quotient transition system –States are equivalence classes –A → B if there exist s in A and s’ in B s.t. s → s’

31. Predicate Abstraction Maps a (possibly infinite state) concrete transition system into a finite state system –Via a set of predicates: Preds = {p 1, p 2 … p n } Abstraction function:  : ConcreteStates → BitVectors  (s) = b 1 b 2 …b n, b i =1  s╞ p i May abstract transitions: Must abstract transitions: Over-approximate concrete Under-approximate concrete transitions Compute may/must transitions automatically: –Use a theorem prover/decision procedure: require 2 n × n × 2 calls x>0 x≤0 12 01 x=x-1 … … x>0 12 23 x=x+1 … …

32. Over-approximation Based Predicate Abstraction Our approach Show property holds Over-approximation based Counter-example driven refinement Expensive computation to calculate abstraction Find defects Under-approximation based Abstraction driven refinement Simple computation to calculate abstraction mapping Comparison Concrete System Refinement State space: Concrete System Refinement State space:

33. To say: Reachability properties We don’t abstract the program counter