Automatic Verification of Pointer Programs using Grammar-based Shape Analysis Hongseok Yang Seoul National University (Joint Work with Oukseh Lee and Kwangkeun Yi)

Automatic Verification of Pointer Programs Inference of program invariants:  crucial for automatic verification.  Difficulty: unboundedly many new heap cells. h:=nil; while (*) { x:=new(nil,nil); if (h=nil) { h:=x; } else { x->next:=h; h->prev:=x; h:=x; } } h nil h h h h Need to “summarize” heap cells.

Goal: Precise “High-level” Invariants h nil h h p dlist(p) dlist(p) ::= nil | pdlist(c) nil c Existing technology: Shape analysis[SaReWi96,99]. Idea: Use a grammar to find a good abstraction of each heap object (i.e., cells and their pointers).

Demo Binomial heap construction: (all pointers to nil are omitted.) Our immediate goal was to handle the binomial heap construction algorithm.

Structure of Our Analysis Abstract Execution Normalize D#D# Nk#Nk# while(B){ C1; C2; C3; C4; } D#D# Nk#Nk# Embed

Abstract Domain D # : Grammar D # = P f (Graph) x Grammar + {T} A grammar R is a set of following rules:   (x) = nil | O | … where V 1, V 2 2 {nil, self, x,  (_),  (self),  (nil),  (x) } Examples:  tree(x) = nil | O  dList(x) = nil | O V1V1 V2V2 tree(_) x dList(self)

Abstract Domain D # : Shape Graph D # = P f (Graph) x Grammar + {T} Shape graph: Each “node” is concrete (“a”), annotated with nil (“d”), or annotated with a nonterminal (“c” and “b”). An element (S,G) in D # is called abstract state. x c:tree(_) a y b:dList(a) d:nil Stack Heap

Normalized Abstract Domain N k # Idealized version of normalization: 1. Group nodes according to heap objects; 2. Compute the best grammar that describes each group; 3. Ensure that each shape graph doesn’t use more than k nodes. Example: N k # ( µ D # ) consists of normalized abstract states.  The actual definition of N k # is not algorithmic. x c:nil a d:nile:nil b x x a:nil a:tree(_) tree(_) tree(_) = nil | O normalize

Definition of Analysis Analysis of programs without loops:  Forward analysis « C ¬ : D # ! D #  Case pruning « B ¬ : D # ! D # « while B C ¬ A = Fix A v F = t n F n (normalize(A)).  F : N k # ! N k #  F = A’. normalize(A’ t « B ¬ ( « C ¬ A’)))

Doubly Linked List Construction h := nil; while (*) { var x; x := new; if (h = nil) { h := x; } else { x->next := h; h->prev := x; h := x }

Inferred Loop Invariant Inferred abstract state (i.e., shape-graph set and grammar):  (x)= nil | O  (x)= nil | O prev next nil  (self) x h a:  (_)

3 rd Iteration Step Abstract state A 2 after the 2 rd iteration: Inferred invariant A: A = normalize(A 2 t « LoopBody ¬ (A 2 ))  (x)= nil | O  (x)= nil | O prev next nil  (self) nil x h a:  (_)  (x)= nil | O  (x)= nil | O prev next nil  (self) x h a:  (_)

Computation of A 2 t« LoopBody ¬ (A 2 )  (x)= nil | O  (x)= nil | O prev next nil  (self)nil x h a:  (_) h x:=new x f:nilg:nil e prevnext x f:nilg:nil e prevnext h a:nil if(h=nil)… x f:nil e prevnext a prev next c:  (a) h g:nilb:nil next prev False Branch 1.Unroll . 2.Prune cases. 3.“Execute”. 4.Join results. 5.Collect “garbage”. True Branch

Normalization 1: Identify Heap Objects  (x)= nil | O  (x)= nil | O prev next nil  (self) nil x h a:  (_) h b:nilc:nil a prevnext h b:nil a prevnext c prev next d:  (c)  (x)=O prev next nil Identify data structures, and express them by nonterminals.

Normalization 1: Identify Heap Objects  (x)= nil | O  (x)= nil | O prev next nil  (self) nil x h a:  (_) hh b:nil a prevnext c prev next d:  (c)  (x)=O prev next nil Identify data structures, and express them by nonterminals.  (x)=O prev next x  (self) a:  (_)

Normalization 1: Identify Heap Objects  (x)= nil | O  (x)= nil | O prev next nil  (self) nil x h a:  (_) hh b:nil a prevnext  (x)=O prev next nil Identify data structures, and express them by nonterminals.  (x)=O prev next x  (self) a:  (_)  (x)=O prev next nil  (self) c:  (a)

Normalization 1: Identify Heap Objects  (x)= nil | O  (x)= nil | O prev next nil  (self) nil x h a:  (_) hh  (x)=O prev next nil Identify data structures, and express them by nonterminals.  (x)=O prev next x  (self) a:  (_)  (x)=O prev next nil  (self) a:  (_)

Normalization 2: Unify Similar Shape Graphs  (x)= nil | O  (x)= nil | O prev next nil  (self) nil x h a:  (_) hh  (x)=O prev next nil Roughly, two shape graphs are similar iff they coincide except the use of nonterminals.  (x)=O prev next x  (self) a:  (_)  (x)=O prev next nil  (self) a:  (_)  (x)= nil | O h a:  (_) prev next nil  (self) prev next nil | O prev next nil  (self) | O

Normalization 3: Collect Garbage  (x)= nil | O  (x)= nil | O prev next nil  (self) nil x  (x)=O prev next nil Eliminate the definitions of unused nonterminals from the grammar.  (x)=O prev next x  (self)  (x)=O prev next nil  (self)  (x)= nil | O h a:  (_) prev next nil  (self) prev next nil | O prev next nil  (self) | O ,  are not used

Normalization 4: Simplify the Grammar  (x)= nil | O prev next nil x Regard  (x) and nil as the same. Combine “same” cases and “same” definitions.  (x)=O prev next x  (self)  (x)= nil | O h a:  (_) prev next nil  (self) prev next nil | O prev next nil  (self) | O “Same” Cases | nil

Normalization 4: Simplify the Grammar  (x)= nil | O prev next nil x Regard  (x) and nil as the same. Combine “same” cases and “same” definitions.  (x)=O prev next x  (self)  (x)= nil | O h a:  (_) prev next nil  (self) | O prev next nil  (self) | nil “Same” Definitions  (self) | O prev next x  (self)

Normalization 4: Simplify the Grammar  (x)= nil | O prev next nil x Regard  (x) and nil as the same. Combine “same” cases and “same” definitions.  (x)= nil | O h a:  (_) prev next nil  (self) | O prev next x  (self) “Same” Cases

Normalization 4: Simplify the Grammar  (x)= nil Regard  (x) and nil as the same. Combine “same” cases and “same” definitions.  (x)= nil | O h a:  (_) prev next nil  (self) | O prev next x  (self)

Summary 1. “Execute” the loop body abstractly: « LoopBody ¬ A 2 2. Join the old and new values: A 2 t « LoopBody ¬ A 2 3. Normalize the obtained abstract state: 1. For each shape graph, identify heap objects and express them using nonterminals. 2. Unify similar shape graphs. 3. Remove the definitions of unused nonterminals. 4. Simplify the grammar.

Correctness The meaning of each abstract state (G,R) is given by an assertion “trans(G,R)” in sep. logic. Correctness theorem: If « C ¬ (G,R) = (G’,R’), then {trans(G,R)}C{trans(G’,R’)} is derivable in sep. logic. Termination: Since the domain N k # is finite, the analysis terminates.

Conclusion Presented an analysis that infers the loop invariant of complex pointer programs. The key idea is to use a grammar to describe the structure of a heap object (i.e., data structure). Future work: 1. Develop a systematic reusable framework. 2. Handle data structures with more extensive sharing.  dags and trees with linked leaf nodes, etc. 3. Prove a property that relates the input and ouput states.  SW recovers link fields to their original values.

Inferred Loop Invariant Inferred shape-graph set and grammar: Representation by an assertion:  (x)= nil | O  (x)= nil | O prev next nil  (self) x h a:  (_) letrec  (a,x) = (emp Æ a=nil) Ç 9 b.(a  nil,b) *  (b,a)  (a,x) = (emp Æ a=nil) Ç 9 b.(a  x,b) *  (b,a)  in  9 a. h=a Æ 8 x.  (a,x)

Abstract Domain D # D # = P f (Graph) x Grammar + {T} Shape graph: Each “node” can be concrete (“a”), annotated with nil (“d”), or annotated with a nonterminal (“c” and “b”). Semantics by separation-logic assertions: 9 abcd.(x=a Æ y=b) Æ (( 8 y.tree(c,y))*(a  c,d)*(c=nil Æ emp)*dList(b,a)) Formal definition:  Graph = (Var ! fin SymL) x (SymL ! fin Val)  Val = {nil,,  (a),  () | a,b 2 SymL,  2 NonTerm } x c:tree(_) a y b:dList(a) d:nil Stack Heap

Grammar A grammar R is a set of following rules:   (x) = nil | O | … where V 1, V 2 2 {nil, self, x,  (_),  (self),  (nil) } Examples:  tree(x) = nil | OdList(x) = x | O Semantics by separation-logic assertions: tree(c,x) = (c=nil Æ emp) Ç 9 lr.(c  l,r)*( 8 y.tree(l,y))*( 8 y.tree(r,y)) dList(c,p) = (c=nil Æ emp) Ç 9 n.(c  p,n)*dList(n,c) Formal definition:  Grammar = NonTerm ! fin P nf ({nil} + Case x Case)  Case = {nil, self, arg,  (_),  (arg),  (self) |  2 NonTerm } V1V1 V2V2 tree(_) x dList(self)

Normalized Abstract Domain N # N # consists of abstract states (G,R) in D # s.t. 1. all “data structures” are expressed by nonterminals: 1. All “similar” shape graphs and rules are merged. x c:nil a b:  (_) x c:nil a d:nile:nil b yx c:nil a b:  (_) y  x) = nil | O  x) = nil | O  (_) nil