1. Efficient Modular Glass Box Software Model Checking Michael Roberson Chandrasekhar Boyapati The University of Michigan

2. Software Model Checking Exhaustively test programs Exhaustively test programs ̶ On all possible inputs ̶ On all possible schedules ̶ Up to finite bounds

3. Binary Tree State Space Initial State State Space Explosion

4. State Space Reduction Many software model checkers Many software model checkers ̶ Verisoft, JPF, CMC, SLAM, Blast, Magic, … Many state space reduction techniques Many state space reduction techniques ̶ Partial order reduction ̶ Predicate abstraction ̶ Effective for control-oriented properties Our work is on Glass Box Software Model Checking Our work is on Glass Box Software Model Checking ̶ Effective for data-oriented properties ̶ Significantly more efficient than previous model checkers

5. Modular Glass Box Checking Check modules against abstractions Check modules against abstractions Check program replacing modules with abstractions Check program replacing modules with abstractions Modular glass box model checking is important Modular glass box model checking is important ̶ Further improve scalability of glass box checking Modular glass box model checking is nontrivial Modular glass box model checking is nontrivial

6. Modular Checking Initial State of Module Initial State of Abstraction Check outputs at each step Modular checking in traditional model checkers

7. Modular Glass Box Checking We can't reach this transition! We cannot use reachability through transitions We cannot use reachability through transitions Programmers must provide a class invariant Programmers must provide a class invariant State space includes all states that satisfy the invariant State space includes all states that satisfy the invariant Programmers must provide an abstraction function Programmers must provide an abstraction function We use it to generate the abstraction of each state We use it to generate the abstraction of each state

8. Equal Modular Glass Box Checking Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality c1c2 a1a2 a2' Operation Abstraction a1_output c1_output

9. Outline Motivation Motivation Example Example Approach Approach Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

10. Integer Counter class IntegerCounter { Map map = new SearchTree(); int max_frequency = 0; int most_frequent = 0; void count(int i) { Integer frequency = (Integer)map.get(i); if (frequency == null) frequency = new Integer(0); map.insert(i, new Integer(frequency+1)); if (frequency >= max_frequency) { max_frequency = frequency; most_frequent = i; } int get_most_frequent() { return most_frequent; } int get_max_frequency() { return max_frequency; } new AbstractMap(); Count an integer Return most frequent integer Return frequency of most frequent integer Frequencies are stored in a Map Modular Approach: Replace Module with Abstraction

11. Module vs Abstraction SearchTree SearchTree ̶ Implements Map get, insert, delete get, insert, delete ̶ Balanced binary tree ̶ Efficient execution ̶ Larger state space AbstractMap Implements Map get, insert, delete Linked list Simple execution Smaller state space 5 26 14 12456 vs

12. Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality Equal c2 a1a2 a2' Operation Abstraction 5 26 14 c1

13. Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality 5 26 14 Equal c2 a1a2 a2' Operation Abstraction 12456

14. Equal a2 insert(3,x) Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality 5 26 14 insert(3,x) 5 26 14 3 123456 c2 a2' Operation Abstraction 12456

15. Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality Equal insert(3,x) 5 26 14 5 26 14 3 123456 a2' Abstraction 12456 a1_output c1_output

16. Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality Equal insert(3,x) 5 26 14 5 26 14 3 123456 a2' Abstraction 12456 123456

17. Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality Equal insert(3,x) 5 26 14 5 26 14 3 123456 Abstraction 12456 123456

18. Glass Box Pruning 5 26 14 insert(3,x) 5 26 14 3 5 2 14 5 2 14 3 5 26 4 3 7 5 26 4 7 An insert operation only touches one path An insert operation only touches one path Insert behaves similarly on many states Insert behaves similarly on many states

19. insert(3,x) 5 26 4 3 7 5 26 4 7 5 2 14 5 2 14 3 Glass Box Pruning 5 26 14 insert(3,x) 5 26 14 3 PRUNED We don't need to check more than one of these We don't need to check more than one of these We can prune all others from the state space We can prune all others from the state space

20. insert(3,x) 5 26 4 3 7 5 26 4 7 5 2 14 5 2 14 3 Glass Box Pruning 5 26 14 insert(3,x) 5 26 14 3 PRUNED We check each tree path, not each tree We check each tree path, not each tree This reduces the state space dramatically This reduces the state space dramatically

21. Checking IntegerCounter 5 26 14 IntegerCounter' 12456 IntegerCounter with SearchTreeIntegerCounter with AbstractMap Smaller state space Better state space reduction Faster analysis vs 2 41 65 4 51 62 4 25 16 5 26 14 5 26 14 4 26 15 12456 IntegerCounter

22. Outline Motivation Motivation Example Example Approach Approach ̶ Program specification ̶ Search algorithm ̶ State space representation ̶ State space reduction Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

23. Module Implementation class SearchTree implements Map { class Node { int key; Object value; Node left; Node right; } Node root; Object get(int key) { /*... */ } void insert(int key, Object value) { /*... */ } void remove(int key) { /*... */ } Map interface methods c1c2 Operation

24. Equal a2 a2' Abstraction class AbstractMap implements Map { class Node { Object key; Object value; Node next; } Node head; Object get(int key) { /*... */ } void insert(int key, Object value) { /*... */ } void remove(int key) { /*... */ } @Declarative boolean equalTo(AbstractMap m) { /*... */ } Map interface methods Equality test Declarative methods Declarative methods ̶ Subset of Java ̶ Free of side effects ̶ Used for specification ̶ Aid our analyses a1a2 Operation

25. Module Specification class SearchTree implements Map { class Node { int key; Object value; Node left; Node right; } Node root; /*... Map operations... */ @Declarative boolean repOk() { /*... */ } AbstractMap abstraction() { /*... */ } Module Invariant Abstraction function c1 a1 Abstraction

26. Approach Program specification Program specification Search algorithm Search algorithm State space representation State space representation State space reduction State space reduction

27. Search Algorithm 5 26 14 Choose an unchecked valid state Choose an unchecked valid state insert(3,x) @Declarative boolean repOk() { /*... */ }

28. Search Algorithm 5 26 14 Generate its abstraction Generate its abstraction insert(3,x) 12456 AbstractMap abstraction() { /*... */ }

29. Search Algorithm 5 26 14 Run the operation on both states Run the operation on both states insert(3,x) 5 26 14 3 123456 void insert(int key, Object value) { /*... */ } 12456

30. Search Algorithm Generate the post-state abstraction Generate the post-state abstraction insert(3,x) AbstractMap abstraction() { /*... */ } 5 26 14 5 26 14 3 123456 12456 123456

31. Search Algorithm Check invariant and abstraction equality Check invariant and abstraction equality insert(3,x) @Declarative boolean repOk() { /*... */ } @Declarative boolean equalTo(AbstractMap m) { /*... */ } 5 26 14 5 26 14 3 123456 12456 123456

32. insert(3,x) 5 2 14 5 2 14 3 5 26 4 3 7 5 26 4 7 State Space Reduction 5 26 14 insert(3,x) 5 26 14 3 PRUNED Identify and prune similar states Identify and prune similar states

33. Search Algorithm Let S be the states that satisfy repOk() While S is not empty Choose a state s in S. Check s. Let P be the set of states similar to s S = S - P Need efficient representation and operations for these sets!

34. Approach Program specification Program specification Search algorithm Search algorithm State space representation State space representation State space reduction State space reduction

35. Representation Represent a set as a boolean formula Represent a set as a boolean formula ̶ Encode each field as bits (b0, b1, …) ̶ Constrain the bits using boolean operations n1.left = null || n1.key > n2.key  b4  (b1   b7)  ((b1   b7)  b0   b6) key = {b0,b1} value = {b2,b3} left = {b4} right = {b5} n1 key = {b6,b7} value = {b8,b9} left = {} right = {} n2...

36. Representation Initialize to set of states that satisfy invariant Initialize to set of states that satisfy invariant ̶ Construct a formula describing invariant boolean formula @Declarative boolean repOk() { /*... */ }

37. Representation Initialize to set of states that satisfy invariant Initialize to set of states that satisfy invariant ̶ Construct a formula describing invariant Declarative methods Declarative methods ̶ No assignment, object creation, or loops ̶ Declarative methods allow efficient translation ̶ Declarative methods produce compact formulas @Declarative boolean repOk() { /*... */ }

38. Search Algorithm Use a SAT solver Add ¬P to the SAT solver Let S be the states that satisfy repOk() While S is not empty Choose a state s in S. Check s. Let P be the set of states similar to s S = S - P

39. Approach Program specification Program specification Search algorithm Search algorithm State space representation State space representation State space reduction State space reduction ̶ Dynamic analysis ̶ Static analysis

40. Dynamic Analysis Discover and prune states that are similar Discover and prune states that are similar Symbolic execution Symbolic execution ̶ Generates a path constraint, P ̶ P holds for states that traverse the same code path ̶ P is the set of similar states to be pruned 5 26 14 insert(3,x)

41. Dynamic Analysis Discover and prune states that are similar Discover and prune states that are similar Symbolic execution Symbolic execution ̶ Generates a path constraint, P ̶ P holds for states that traverse the same code path ̶ P is the set of similar states to be pruned n1 n2n3 n4n5 op(key,value) op = insert && root = n1 && key < n1.key && n1.left = n2 && key > n2.key && n2.right = n5 && key < n5.key && n5.left = null Operation is insert Node exists, is greater/less than key Final node does not exist (yet)

42. Static Analysis Dynamic analysis finds P, the similar states Dynamic analysis finds P, the similar states Pruning these states is not always correct! Pruning these states is not always correct! class WhyStaticAnalysis { boolean a, b; void operation() { if (a) a = false; else a = true; } @Declarative boolean repOk() { return a || b; } a = true b = true a = false b = true a = true b = false a = false b = false a = true P := We use a static analysis to ensure correctness We use a static analysis to ensure correctness ̶ All pruned transitions checked together ̶ Any error within finite bounds is caught 

43. Static Analysis class WhyStaticAnalysis { boolean a, b; void operation() { if (a) a = false; else a = true; } @Declarative boolean repOk() { return a || b; } a = true b = true a = false b = true a = true b a = false b a = true P := Prestate of a pruned transition Poststate of a pruned transition repOk= (a || b) repOk Pre = (a || b) a=true = true repOk Post = (a || b) a=false = b repOk Pre  repOk Post = b Not valid when b = false! Invariant Prestate Invariant Poststate Invariant Correct Transition

44. Static Analysis For every valid prestate in P, the following hold For every valid prestate in P, the following hold ̶ The invariant is maintained in the poststate ̶ Equality of abstractions repOk pre  repOk post && abs_post.equalTo(abs_post') Use a SAT solver to check Use a SAT solver to check ̶ If it holds then pruning is sound ̶ If not, we have a counterexample Equal prepost abs pre abs post abs post' Operation Abstraction boolean formula

45. Outline Motivation Motivation Example Example Approach Approach Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

46. Checking Modules vs Abstractions TreeMap TreeMap ̶ Implemented with a red-black tree HashMap HashMap ̶ Implemented with a hash table AbstractMap AbstractMap ̶ Implemented with a linked list of (key, value) pairs

47. Checking Modules vs Abstractions TreeSet TreeSet ̶ Implemented with a TreeMap HashSet HashSet ̶ Implemented with a HashMap AbstractSet AbstractSet ̶ Implemented with a linked list of set items

48. Maps vs AbstractMap Benchmark Max Number of Nodes JPFKoratBlast Glass Box Checker TreeMap vs AbstractMap 1 2 3 4 … 8 9 … 15 31 63 1.218 5.556 memory out 0.608 0.613 0.676 0.732 … 2202.31 timeout aborted0.188 0.244 0.392 0.485 … 1.124 1.491 … 4.571 40.405 787.411 HashMap vs AbstractMap 1 2 3 4 … 10 11 12 … 16 32 64 0.674 6.514 memory out 0.465 0.497 0.539 0.810 … 150.932 1203.986 timeout aborted0.176 0.227 0.256 0.305 … 0.780 0.953 1.162 … 2.879 75.139 2004.723 We check over 2 35 trees in under 15 minutes

49. Sets vs AbstractSet Benchmark Max Number of Nodes JPFKoratBlast Glass Box Checker TreeSet vs AbstractSet 1 2 3 4 … 13 14 15 … 31 63 0.537 0.950 26.816 memory out 0.638 0.648 0.693 1.020 … 615.524 1706.41 timeout aborted0.195 0.246 0.393 0.489 … 3.065 3.399 4.481 … 39.789 796.955 HashSet vs AbstractSet 1 2 3 4 … 13 14 15 16 … 32 64 0.728 0.781 6.574 memory out 0.520 0.511 0.716 0.570 … 238.420 593.045 952.317 timeout aborted0.171 0.221 0.248 0.299 … 1.344 1.608 2.029 2.816 … 68.011 2543.034

50. Checking Clients BenchmarkMax Size Original Program Maps Replaced with AbstractMap IntegerCounter 1 2 3 … 7 15 31 63 127 255 0.198 0.279 0.469 … 1.182 5.591 41.5 632.488 timeout 0.113 0.154 0.164 … 0.267 0.539 1.867 10.794 93.276 946.091 DualCache 1 2 3 … 7 15 31 63 127 255 0.203 0.283 0.503 … 1.207 5.765 53.434 723.267 timeout 0.222 0.323 0.327 … 0.529 1.015 4.057 26.192 215.521 2180.506

51. Checking Clients Benchmark Max Number of Nodes Original Program Maps Replaced with AbstractMap TreeSet 1 2 3 … 7 15 31 63 127 255 511 0.195 0.246 0.393... 0.961 4.481 39.789 796.955 timeout 0.122 0.153 0.167 … 0.251 0.429 0.991 3.388 16.690 184.827 425.328 HashSet 1 2 3 4 5 … 16 32 64 128 256 512 0.171 0.221 0.248 0.299 0.363 … 2.816 68.011 2543.034 timeout 0.106 0.141 0.153 0.169 0.193 … 0.451 0.989 3.464 17.071 91.629 754.426

52. Related Work State space reduction techniques State space reduction techniques ̶ Abstraction & refinement [SLAM; Blast; Magic] ̶ Partial order reduction [Godefroid97; Flanagan05] ̶ Heap canonicalization [Musuvathi05; Iosif02] ̶ Symmetry reduction [Ip93]

53. Related Work Software model checkers Software model checkers ̶ Verisoft [Godefroid97] ̶ Java Pathfinder [Visser00] ̶ CMC [Musuvathi02] ̶ Bandera [Corbett00] ̶ Bogor [Dwyer05] ̶ SLAM [Ball01] ̶ Blast [Henzinger02] ̶ Magic [Chaki03] ̶ Jalloy [Vaziri03] ̶ Miniatur [Dolby07]

54. Conclusions Significant improvement over traditional model checkers for checking complex data dependent properties Significant improvement over traditional model checkers for checking complex data dependent properties A promising approach to checking much larger programs and broader classes of program properties than is currently possible A promising approach to checking much larger programs and broader classes of program properties than is currently possible Modular Glass Box Model Checking Offers: