Efficient Modular Glass Box Software Model Checking Michael Roberson Chandrasekhar Boyapati The University of Michigan

Software Model Checking Exhaustively test programs Exhaustively test programs ̶ On all possible inputs ̶ On all possible schedules ̶ Up to finite bounds

Binary Tree State Space Initial State State Space Explosion

State Space Reduction Many software model checkers Many software model checkers ̶ Verisoft, JPF, CMC, SLAM, Blast, Magic, … Many state space reduction techniques Many state space reduction techniques ̶ Partial order reduction ̶ Predicate abstraction ̶ Effective for control-oriented properties Our work is on Glass Box Software Model Checking Our work is on Glass Box Software Model Checking ̶ Effective for data-oriented properties ̶ Significantly more efficient than previous model checkers

Modular Glass Box Checking Check modules against abstractions Check modules against abstractions Check program replacing modules with abstractions Check program replacing modules with abstractions Modular glass box model checking is important Modular glass box model checking is important ̶ Further improve scalability of glass box checking Modular glass box model checking is nontrivial Modular glass box model checking is nontrivial

Modular Checking Initial State of Module Initial State of Abstraction Check outputs at each step Modular checking in traditional model checkers

Modular Glass Box Checking We can't reach this transition! We cannot use reachability through transitions We cannot use reachability through transitions Programmers must provide a class invariant Programmers must provide a class invariant State space includes all states that satisfy the invariant State space includes all states that satisfy the invariant Programmers must provide an abstraction function Programmers must provide an abstraction function We use it to generate the abstraction of each state We use it to generate the abstraction of each state

Equal Modular Glass Box Checking Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality c1c2 a1a2 a2' Operation Abstraction a1_output c1_output

Outline Motivation Motivation Example Example Approach Approach Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

Integer Counter class IntegerCounter { Map map = new SearchTree(); int max_frequency = 0; int most_frequent = 0; void count(int i) { Integer frequency = (Integer)map.get(i); if (frequency == null) frequency = new Integer(0); map.insert(i, new Integer(frequency+1)); if (frequency >= max_frequency) { max_frequency = frequency; most_frequent = i; } int get_most_frequent() { return most_frequent; } int get_max_frequency() { return max_frequency; } new AbstractMap(); Count an integer Return most frequent integer Return frequency of most frequent integer Frequencies are stored in a Map Modular Approach: Replace Module with Abstraction

Module vs Abstraction SearchTree SearchTree ̶ Implements Map get, insert, delete get, insert, delete ̶ Balanced binary tree ̶ Efficient execution ̶ Larger state space AbstractMap Implements Map get, insert, delete Linked list Simple execution Smaller state space vs

Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality Equal c2 a1a2 a2' Operation Abstraction c1

Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality Equal c2 a1a2 a2' Operation Abstraction 12456

Equal a2 insert(3,x) Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality insert(3,x) c2 a2' Operation Abstraction 12456

Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality Equal insert(3,x) a2' Abstraction a1_output c1_output

Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality Equal insert(3,x) a2' Abstraction

Checking SearchTree Choose a state Choose a state Generate abstraction Generate abstraction Run one operation Run one operation Check outputs Check outputs Generate abstraction on post-state Generate abstraction on post-state Check for equality Check for equality Equal insert(3,x) Abstraction

Glass Box Pruning insert(3,x) An insert operation only touches one path An insert operation only touches one path Insert behaves similarly on many states Insert behaves similarly on many states

insert(3,x) Glass Box Pruning insert(3,x) PRUNED We don't need to check more than one of these We don't need to check more than one of these We can prune all others from the state space We can prune all others from the state space

insert(3,x) Glass Box Pruning insert(3,x) PRUNED We check each tree path, not each tree We check each tree path, not each tree This reduces the state space dramatically This reduces the state space dramatically

Checking IntegerCounter IntegerCounter' IntegerCounter with SearchTreeIntegerCounter with AbstractMap Smaller state space Better state space reduction Faster analysis vs IntegerCounter

Outline Motivation Motivation Example Example Approach Approach ̶ Program specification ̶ Search algorithm ̶ State space representation ̶ State space reduction Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

Module Implementation class SearchTree implements Map { class Node { int key; Object value; Node left; Node right; } Node root; Object get(int key) { /*... */ } void insert(int key, Object value) { /*... */ } void remove(int key) { /*... */ } Map interface methods c1c2 Operation

Equal a2 a2' Abstraction class AbstractMap implements Map { class Node { Object key; Object value; Node next; } Node head; Object get(int key) { /*... */ } void insert(int key, Object value) { /*... */ } void remove(int key) { /*... */ boolean equalTo(AbstractMap m) { /*... */ } Map interface methods Equality test Declarative methods Declarative methods ̶ Subset of Java ̶ Free of side effects ̶ Used for specification ̶ Aid our analyses a1a2 Operation

Module Specification class SearchTree implements Map { class Node { int key; Object value; Node left; Node right; } Node root; /*... Map operations... boolean repOk() { /*... */ } AbstractMap abstraction() { /*... */ } Module Invariant Abstraction function c1 a1 Abstraction

Approach Program specification Program specification Search algorithm Search algorithm State space representation State space representation State space reduction State space reduction

Search Algorithm Choose an unchecked valid state Choose an unchecked valid state boolean repOk() { /*... */ }

Search Algorithm Generate its abstraction Generate its abstraction insert(3,x) AbstractMap abstraction() { /*... */ }

Search Algorithm Run the operation on both states Run the operation on both states insert(3,x) void insert(int key, Object value) { /*... */ } 12456

Search Algorithm Generate the post-state abstraction Generate the post-state abstraction insert(3,x) AbstractMap abstraction() { /*... */ }

Search Algorithm Check invariant and abstraction equality Check invariant and abstraction equality boolean repOk() { /*... */ boolean equalTo(AbstractMap m) { /*... */ }

insert(3,x) State Space Reduction insert(3,x) PRUNED Identify and prune similar states Identify and prune similar states

Search Algorithm Let S be the states that satisfy repOk() While S is not empty Choose a state s in S. Check s. Let P be the set of states similar to s S = S - P Need efficient representation and operations for these sets!

Approach Program specification Program specification Search algorithm Search algorithm State space representation State space representation State space reduction State space reduction

Representation Represent a set as a boolean formula Represent a set as a boolean formula ̶ Encode each field as bits (b0, b1, …) ̶ Constrain the bits using boolean operations n1.left = null || n1.key > n2.key  b4  (b1   b7)  ((b1   b7)  b0   b6) key = {b0,b1} value = {b2,b3} left = {b4} right = {b5} n1 key = {b6,b7} value = {b8,b9} left = {} right = {} n2...

Representation Initialize to set of states that satisfy invariant Initialize to set of states that satisfy invariant ̶ Construct a formula describing invariant boolean boolean repOk() { /*... */ }

Representation Initialize to set of states that satisfy invariant Initialize to set of states that satisfy invariant ̶ Construct a formula describing invariant Declarative methods Declarative methods ̶ No assignment, object creation, or loops ̶ Declarative methods allow efficient translation ̶ Declarative methods produce compact boolean repOk() { /*... */ }

Search Algorithm Use a SAT solver Add ¬P to the SAT solver Let S be the states that satisfy repOk() While S is not empty Choose a state s in S. Check s. Let P be the set of states similar to s S = S - P

Approach Program specification Program specification Search algorithm Search algorithm State space representation State space representation State space reduction State space reduction ̶ Dynamic analysis ̶ Static analysis

Dynamic Analysis Discover and prune states that are similar Discover and prune states that are similar Symbolic execution Symbolic execution ̶ Generates a path constraint, P ̶ P holds for states that traverse the same code path ̶ P is the set of similar states to be pruned insert(3,x)

Dynamic Analysis Discover and prune states that are similar Discover and prune states that are similar Symbolic execution Symbolic execution ̶ Generates a path constraint, P ̶ P holds for states that traverse the same code path ̶ P is the set of similar states to be pruned n1 n2n3 n4n5 op(key,value) op = insert && root = n1 && key < n1.key && n1.left = n2 && key > n2.key && n2.right = n5 && key < n5.key && n5.left = null Operation is insert Node exists, is greater/less than key Final node does not exist (yet)

Static Analysis Dynamic analysis finds P, the similar states Dynamic analysis finds P, the similar states Pruning these states is not always correct! Pruning these states is not always correct! class WhyStaticAnalysis { boolean a, b; void operation() { if (a) a = false; else a = true; boolean repOk() { return a || b; } a = true b = true a = false b = true a = true b = false a = false b = false a = true P := We use a static analysis to ensure correctness We use a static analysis to ensure correctness ̶ All pruned transitions checked together ̶ Any error within finite bounds is caught 

Static Analysis class WhyStaticAnalysis { boolean a, b; void operation() { if (a) a = false; else a = true; boolean repOk() { return a || b; } a = true b = true a = false b = true a = true b a = false b a = true P := Prestate of a pruned transition Poststate of a pruned transition repOk= (a || b) repOk Pre = (a || b) a=true = true repOk Post = (a || b) a=false = b repOk Pre  repOk Post = b Not valid when b = false! Invariant Prestate Invariant Poststate Invariant Correct Transition

Static Analysis For every valid prestate in P, the following hold For every valid prestate in P, the following hold ̶ The invariant is maintained in the poststate ̶ Equality of abstractions repOk pre  repOk post && abs_post.equalTo(abs_post') Use a SAT solver to check Use a SAT solver to check ̶ If it holds then pruning is sound ̶ If not, we have a counterexample Equal prepost abs pre abs post abs post' Operation Abstraction boolean formula

Outline Motivation Motivation Example Example Approach Approach Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

Checking Modules vs Abstractions TreeMap TreeMap ̶ Implemented with a red-black tree HashMap HashMap ̶ Implemented with a hash table AbstractMap AbstractMap ̶ Implemented with a linked list of (key, value) pairs

Checking Modules vs Abstractions TreeSet TreeSet ̶ Implemented with a TreeMap HashSet HashSet ̶ Implemented with a HashMap AbstractSet AbstractSet ̶ Implemented with a linked list of set items

Maps vs AbstractMap Benchmark Max Number of Nodes JPFKoratBlast Glass Box Checker TreeMap vs AbstractMap … 8 9 … memory out … timeout aborted … … HashMap vs AbstractMap … … memory out … timeout aborted … … We check over 2 35 trees in under 15 minutes

Sets vs AbstractSet Benchmark Max Number of Nodes JPFKoratBlast Glass Box Checker TreeSet vs AbstractSet … … memory out … timeout aborted … … HashSet vs AbstractSet … … memory out … timeout aborted … …

Checking Clients BenchmarkMax Size Original Program Maps Replaced with AbstractMap IntegerCounter … … timeout … DualCache … … timeout …

Checking Clients Benchmark Max Number of Nodes Original Program Maps Replaced with AbstractMap TreeSet … timeout … HashSet … … timeout …

Related Work State space reduction techniques State space reduction techniques ̶ Abstraction & refinement [SLAM; Blast; Magic] ̶ Partial order reduction [Godefroid97; Flanagan05] ̶ Heap canonicalization [Musuvathi05; Iosif02] ̶ Symmetry reduction [Ip93]

Related Work Software model checkers Software model checkers ̶ Verisoft [Godefroid97] ̶ Java Pathfinder [Visser00] ̶ CMC [Musuvathi02] ̶ Bandera [Corbett00] ̶ Bogor [Dwyer05] ̶ SLAM [Ball01] ̶ Blast [Henzinger02] ̶ Magic [Chaki03] ̶ Jalloy [Vaziri03] ̶ Miniatur [Dolby07]

Conclusions Significant improvement over traditional model checkers for checking complex data dependent properties Significant improvement over traditional model checkers for checking complex data dependent properties A promising approach to checking much larger programs and broader classes of program properties than is currently possible A promising approach to checking much larger programs and broader classes of program properties than is currently possible Modular Glass Box Model Checking Offers: