1 Dependable Software via Automated Verification Huu Hai Nguyen(NUS) Cristina David(NUS) Shengchao Qin(Durham, UK) Wei-Ngan Chin(NUS)

2 Goal Verify shapely and mutable data structures with invariants involving size properties

3 Challenges Strong updates in the presence of aliasing Entailment checking with recursive predicates Emphasis : practical verification

4 Separation Logic Foundations O’Hearn and Pym, “The Logic of Bunched Implications”, Bulletin of Symbolic Logic 1999 Reynolds, “Separation Logic: A Logic for Shared Mutable Data Structures”, LICS 2002 Extension to Hoare logic to reason about shared mutable data structures.  : spatial conjunction --  : spatial implication

5 Examples (Reynolds LICS 2002) 3 y 3 x xy x  3,y  y  3,x 3 x y x  3,y  y  3,x 3 x x  3,y y

6 Outline Background Our Approach How to Verify? Entailment Approximation Expressiveness Multiple Pre/Post Set of States Coercion Rules

7 Our Work Size properties Data structure with derived size properties and invariant. Length, height-balanced, sortedness etc. User-defined inductive predicates predicates are custom designed Sound entailment checking (but incomplete)

8 Overview code verifier entailment prover Pre/Post Shape Predicate Coercion Rule Code

9 Example Predicates sortl  n,min,max   self::node  min,null   min=max Æ n=1  self::node  min,q   q::sortl  n-1,k,max   min  k inv n  1 Æ min · max Non-empty sorted list ll  n   self=null  n=0  self::node  _,r   r::ll  n-1  inv n  0 Singly-linked list derived attribute (c.f. model field) user-supplied invariant self null

10 Example Predicates lseg  n,p   self=p  n=0  self::node  _,r   r::lseg  n-1,p  inv n  0 List Segment Non-empty Circular list clist  n   self::node  _,r   r::lseg  n-1,self  inv n  1 self p r

11 Example Predicates AVL Tree (near-balanced tree) avl  h   self = null Æ h = 0 Ç self::node h, h, p, q i  p::avl h h1 i  q::avl h h2 i Æ -1 · h1-h2 · 1 Æ h = max(h1,h2) + 1 inv h ¸ 0

12 What Can Be Written? Heap part Describes shapes using separation logic Pure part Size properties (arithmetic constraints) Pointer constraint self points to the “root” of data structure A “root” pointer is a pointer from which every node is reachable.

13 Ensuring Automation & Termination Well-formed A predicate captures a set of nodes reachable from self. Applied to predicate defns + specifications Well-founded self bound to a data node, not predicate At most one data node for each conjunct in a predicate Applied to predicate defns

14 Outline Background Our Approach How to Verify? Entailment Approximation Expressiveness Multiple Pre/Post Set of States Coercion Rules

15 Verification Methods and loops are annotated with pre- and post-conditions. Entailment checks Precondition at call site Postcondition at end of method

16 Verification Rules

17 Core Language

18 Insert into a Sorted List node insert(node x, node vn) requires x::sortl  n, sm,lg   vn::node  v,_  ensures res::sortl  n+1,min(v,sm),max(v,lg)  { if (vn.val≤x.val) { vn.next = x; return vn; } else if (x.next=null) then { x.next = vn; vn.next = null; return x; } else { x.next = insert(x.next, vn); return x; } }

19 { x’::sortl  vn’::node } vn.next = x; { (x’::node  n=1 Æ mi=mx  x’::node  q::sortl  mi  k  n  2)  vn’::node  v  mi } if (vn.val <= x.val) { return vn; { res::sortl } } { (x’::node  n=1  mi=mx  x’::node  q::sortl  mi  k  n  2)  vn’::node  v  mi } { (x’::node  n=1  mi=mx  x’::node  q::sortl  mi  k  n  2)  vn’::node  v  mi Æ res=vn’ }

20 Outline Background Our Approach How to Verify? Entailment Approximation Expressiveness Multiple Pre/Post Set of States Coercion Rules

21 Entailment  1 `  2   3  1 “subsumes” all heap nodes “present” in  2. Remaining nodes are kept in  3. ALGO : Cover all nodes in  2 that are subsumed by  1, then convert to an arithmetic implication check. A smart “frame inferring” prover.

22 Key Steps of Entailment Matching Aliased data nodes/predicates are matched and their components/arguments unified Unfolding Replaces a predicate on LHS by its definition to match a node on the RHS Folding Triggered by a predicate on the RHS and a data node on the LHS. Recursive invocation.

23 Entailment Algorithm

24 Matching x::ll h n iÆ n>1 ` 9 m ¢ x::ll h m iÆ m>0 matching n>1 ` n>0 x::ll h n iÆ n>1 ` x::ll h m iÆ m>0 matching n=m Æ n>1 ` n>0 free var defn moved to current state in LHS

25 Preserving Free Variables Specification void insert(node x, int v) requires x::ll h n i Æ n>0 ensures x::ll h n+1 i Call x = new node(0, null); insert(x, 1); Precondition check x’::node h 0,null i ` x’::ll h n i Æ n>0 n=1 ` n>0 Program state after adding postcondition x’::ll h n+1 i Æ n=1 free variable linking pre and post conditions

26 Unfolding x::ll h n iÆ n>1 ` 9 r, m ¢ x::node h _,r i  r::ll h m iÆ m>0 Unfolding 9 q ¢ x::node h _, q i  q::ll h n-1 i Æ n>1 ` 9 r, m ¢ x::node h _, r i  r::ll h m i Æ m>0 Matching q1::ll h n-1 iÆ n>1 ` 9 m ¢ q1::ll Æ m>0

27 y=null ` y::ll h m i Æ m=0 Recursive entailment y=null ` y=null Æ m=0 Ç y::node h _,r i  r::ll h m-1 i Back y=null Æ m=0 ` m=0 Folding: Base Case

28 x::node h _, r i Æ r=null ` x::ll h n i Æ n>0 Recursive entailment x::node h _, r i Æ r=null ` 9 q,k ¢ x::node h _,q i  q::ll h k i Æ k=n-1 Ç x=null Æ n=0 r=null ` 9 k ¢ r::ll h k i Æ k=n-1 Back r=null Æ n-1=0 ` n>0 Folding: Recursive Case

29 Outline Background Our Approach Why Verification? Entailment Approximation Expressiveness Multiple Pre/Post Set of States Coercion Rules

30 Approximation Each predicate is approximated by a pure constraint. When the consequent’s heap is empty, the antecedent is approximated by a pure constraint. Entailment is reduced to entailment of pure constraints.

31 Approximation: Examples XPure(x::node h i  y::node h i ) = ex i. x=i Æ i>0  ex j. x=j Æ j>0 =  i,j. (x=i Æ i>0 Æ y=j Æ j>0 Æ i  j) XPure 0 (x::ll h n i ) = n ¸ 0 (too weak) XPure 1 (x::ll h n i ) =  i. (x=0 Æ n=0 Ç x=i Æ i>0 Æ n>0)

32 Translating to Pure Form

33 Outline Background Our Approach How to Verify? Entailment Approximation Expressiveness Multiple Pre/Post Set of States Coercion Rules

34 Multiple Pre/Post Each method may have multiple pre/post conditions. node append (node x, node y) requires x::ll  n   y::ll  m  ensures res::ll  n+m  Æ requires x::sortl  n,a1,a2   y::sortl  m,b1,b2  & a2 · b1 ensures res::sortl  n+m,a1,b2  { if x=null then { return y} else { x.next = append(x.next,y); return x; } }

35 Multiple Pre/Post More pre/post for append possible! node append (node x, node y) requires x::ll  n  Æ x=y Æ n>0 ensures res::clist  n  Æ requires x::sortl  n,a1,a2  Æ y=null ensures res::sortl  n,a1,a2  …  requires x::ll  n  Æ x  y ensures res::lseg  n,y 

36 Set of States  1 `  2  {  3,..,  n } {  3,..,  n } denotes non-deterministic outcome from proof search. { } means entailment has failed. Modified Floyd-Hoare style forward reasoning :  {  } code {  1,..,  n }

37 Related Predicates Some predicates may be related. Every sorted list is also an unsorted list. ll  n   self=null  n=0  self::node  _,r   r::ll  n-1  inv n  0 sortl  n,min,max   self::node  min,null   min=max Æ n=1  self::node  min,q   q::sortl  n-1,k,max   min  k inv n  1 Æ min · max sortl  n,min,max  ) self::ll  n  This coercion is used in bubble sorting.

38 Coercion Rules Allowed c 1  v 1 *   ) 9 v * ¢ self::c 2  v 2 *    Æ  Implication c 1  v 1 *    ( 9 v * ¢ self::c 2  v 2 *    Æ  Reverse-Implication c 1  v 1 *   , 9 v * ¢ self::c 2  v 2 *    Æ  Equivalence

39 Equivalence Coercion Definition of list segment is inadequate Termination ensured by well-formed and well-founded property Important to have equivalence coercion : lseg  n,p   self=p  n=0  self::node  _,r   r::lseg  n-1,p  inv n  0 basic definition extra property

40 Proof Search via Equiv. Coercion x::lseg  n,p  … ` x::lseg  m,r  … match … ` [n/m,p/r] … unfold lseg  n,p  x::lseg  i,q   q::lseg  j,p  Æ n=i+j … ` x::lseg  m,r  … fold lseg  m,r  x::lseg  n,p  … ` x::lseg  i,q   q::lseg  j,r  Æ m=i+j

41 Experiments

42 Conclusion User-defined data structures with size-based invariants Sound and terminating entailment checking Expressive verification with proof search and coercion rules. Future work: inference + debugging + extensions.

43 Termination Each step reduces consequent or antecedent. No infinite unfolding self must point to a data node Consequent is reduced with after each unfold No infinite folding Antecedent is reduced with each folding.

44 Handling Multiple Pre/Post

45 Verification of Shared Mutable Data Structures Hoare and He, “A Trace Model for Pointers and Objects”, ECOOP 1999 The complexity of pointer swing “perhaps a crippling disadvantage” Bornat, “Proving Pointer Programs in Hoare Logic”, MPC 2000 Aliasing Inductive formulae Complexity of proof

46 Existential Variables

47 Example: how to compare these two? x::ll |- x::ll & m>0 Precondition m must be a fresh variable x::ll |- ex m. x::ll & m>0

48 Matching

49 Folding Summarize a heap configuration to a predicate May result in multiple instances of the predicate All resulted possibilities are explored

50 Example x::ll & n>1 |- ex r. x::node & r!=null

51 Example (animated steps) y=null |- y::ll & n>0 x::node * r::sortl<> & … |- x::sortl

52 Soundness What are the issues with soundness: Free variables are fresh

53 Predicate Definition Need to make clear the distinction between self and other arguments This helps explain why “free” arguments can be treated that way

54 Related Works Berdine, Calcagno, O’Hearn, “Symbolic Execution with Separation Logic”, APLAS 2005 Fixed set of predicates without size properties Complete semantics for these predicates Sims, “Extending Separation Logic with Fixpoints and Postponed Substitution”, AMAST 2004 Formulated using least/greatest fixpoint operators Unclear how to carry out entailment/analysis

55 Set of States

56 Why Verification? Annotation versus Analysis Scope for user-guidance Improved expressivity. Validator for analysis. Towards Grand Challenge of verified software. Utopia : Marriage of annotation and analysis

57 AVL tree avl  h   self = null Æ h = 0 Ç self::node h, h, p, q i  p::avl h h1 i  q::avl h h2 i Æ -1 · h1-h2 · 1 Æ h = max(h1,h2) + 1 inv h ¸ 0 avl  h,b   self = null Æ h = 0 Æ b=0 Ç self::node h, h, p, q i  p::avl h h1,b1 i  q::avl h h2,b2 i Æ -1 · h1-h2 · 1 Æ h = max(h1,h2) + 1 Æ b = h1 - h2 inv h ¸ 0 Æ -1 · b · 1 derived attribute captures more information which may be needed for some verification tasks

58 AVL Insert node insert(node t, int x) requires t::avl h th, b i ensures res::avl h resh, resb i Æ (t  null Æ (th=resh Ç resh=th+1 Æ resb  0) Ç t=null Æ resh=1 Æ resb=0); { if (t==null) {…} else if (x < t.ele) { t.left = insert(t.left, x); if (height(t.left) - height(t.right) == 2) { if (height(t.left.left) > height(t.left.right)) t = rotate_left_child(t); else t = double_left_child(t); } … } t’::node h _,th,l,r i  l::avl h lh,lb i  r::avl h rh,rb i t’::node h _,h,l1,r i  l1::avl h lh1,lb1 i  r::avl h rh,rb i Æ lh1 – rh = 2 Æ (l  null Æ (lh1 = lh Ç lh1 = lh + 1 Æ lb1  0) Æ -1 · lh – rh · 1 Ç lh1=1 Æ l = null)