1. Refinements to techniques for verifying shape analysis invariants in Coq Kenneth Roe GBO Presentation 9/30/2013 The Johns Hopkins University

2. Summary Formal Methods for Imperative Languages Such as C – Many bugs caused by corruption of data structures – Use formal methods to document data structure invariants and then verify correct program execution – Framework being developed in the Coq interactive theorem prover

3. Research contribution Extension of Coq based program verification to larger programs with more complex data structures. – Existing systems only work on small examples

4. A tree traversal example in C Struct list { t = NULL; struct list *fld_n; } else { struct tree *fld_t; list *tmp = i->n; }; t = i->fld_t; free(l); Struct tree { i = tmp; struct tree *fld_l, *fld_r; } int value; } else if (t->r==NULL) { }; t = t->fld_l; } else if (t->l==NULL) { struct list *p; t = t->fld_r; void build_pre_order(struct tree *r) { } else { struct list *i = NULL, *n, *x; n = i; struct tree *t = r; i = malloc( p = NULL; sizeof(struct list)); while (t) { i->fld_n = n; n = p; x = t->fld_r; p = malloc(sizeof(struct list)); i->fld_t = x; p->fld_t = t;t = t->fld_l; p->fld_n = n; } if (t->fld_l==NULL && t->fld_r==NULL) { } if (i==NULL) {}

5. What this program does 1 2 34 5 6 Nil t p i r 10 12 14 16 18 20

6. What this program does 1 2 34 5 6 10 Nil t p i r 10 12 14 16 18 20

7. What this program does 1 2 34 5 6 10 Nil 18 Nil t p i r 10 12 14 16 18 20

8. What this program does 1 2 34 5 6 12 10 Nil 18 Nil t p i r 10 12 14 16 18 20

9. What this program does 1 2 34 5 6 12 10 Nil 16 18 Nil t p i r 10 12 14 16 18 20

10. What this program does 1 2 34 5 6 14 12 10 Nil 16 18 Nil t p i r 10 12 14 16 18 20

11. What this program does 1 2 34 5 6 14 12 10 Nil 18 Nil t p i r 10 12 14 16 18 20

12. What this program does 1 2 34 5 6 16 14 12 10 Nil 18 Nil t p i r 10 12 14 16 18 20

13. What this program does 1 2 34 5 6 16 14 12 10 Nil t p i r 10 12 14 16 18 20

14. What this program does 1 2 34 5 6 18 16 14 12 10 Nil t p i r 10 12 14 16 18 20

15. What this program does 1 2 34 5 6 18 16 14 12 10 Nil t p i r 10 12 14 16 18 20

16. What this program does 1 2 34 5 6 20 18 16 14 12 10 Nil t p i r 10 12 14 16 18 20

17. What this program does 1 2 34 5 6 20 18 16 14 12 10 Nil t p i r 10 12 14 16 18 20

18. Invariants to be formally proven The program maintains two well formed linked lists, the heads of which are pointed to by i and p. – By well formed we mean that memory on the heap is properly allocated for the lists and there are no loops in the data structures. The program maintains a well formed tree pointed to by r. t always points to an element in the tree rooted at r. The two lists and the tree do not share any nodes. Other than the memory used for the two lists and the tree, no other heap memory is allocated. The fld_t field of every element in both list structures points to an element in the tree.

19. Coq Goal { ? } WHILE not (T == 0) DO N := P; NEW P, 2;... { ? }

20. Program state Environment R=10 I=30 P=40 T=10 Heap e = { R → 10, I → 20, P → 30, T → 10 } h = {10 → 12, 11 → 18, 12 → 14, 13 → 16, 14 → 0, 15 → 0, 16 → 0, 17 → 0, 18 → 20, 19 → 0, 20 → 0, 21 → 0, …}

21. Program state Environment R=10 I=30 P=40 T=10 Heap e = { R → 10, I → 20, P → 30, T → 10 } h = {10 → 12, 11 → 18, 12 → 14, 13 → 16, 14 → 0, 15 → 0, 16 → 0, 17 → 0, 18 → 20, 19 → 0, 20 → 0, 21 → 0} ∃ v 0. (e,h) ⊨ TREE(R,v 0,2,[0,1]) X v 0 =[10, [12,[14,[0],[0]], [16,[0],[0]]], [18, [20,[0],[0]], [0]]] struct tree { struct tree *left; struct tree * right; }

22. Separation logic (e,h) ⊨ ∃ v 0 v 1 v 2 TREE(R,v 0,2,[0,1]) * TREE(I,v 1,2[0]) * TREE(P,v 2,2,[0]) h={10 → 12,11 → 18,12 → 14,13 → 16,14 → 0,15 → 0,16 → 0,17 → 0,18 → 20, 19→ 0,20 → 0,21 → 0,30 → 32,31 → 10,32 → 0,33 → 12,40 → 42,41 → 14, 42 → 44,43 → 12,44 → 0,45 → 10}

23. Separation logic (e, h) ⊨ s1 ∗ s2 if and only if ∃ h′, h′′. (e,h′) ⊨ s1 ⋀ (e,h′′) ⊨ s2 ⋀ dom(h1)∩dom(h2)= ∅ ⋀ h=h′ ∪ h′′

24. Data structure relationships (e,h) ⊨ ∃ v 0 v 1 v 2 TREE(R,v 0,2,[0,1]) * TREE(I,v 1,2,[0]) * TREE(P,v 2,2,[0])

25. Data structure relationships (e,h) ⊨ ∃ v 0 v 1 v 2 TREE(R,v 0,2,[0,1]) * TREE(I,v 1,2[0]) * TREE(P,v 2,2,[0]) * ∀ v 3 ∈ TreeRecords(v 1 ). [nth(find(v 1,v 3 ),2) inTree v 0 ]

26. Data structure relationships (e,h) ⊨ ∃ v 0 v 1 v 2 TREE(R,v 0,2,[0,1]) * TREE(I,v 1,2[0]) * TREE(P,v 2,2,[0]) * ∀ v 3 ∈ TreeRecords(v 1 ). [nth(find(v 1,v 3 ),2) inTree v 0 ] * ∀ v 3 ∈ TreeRecords(v 2 ). [nth(find(v 2,v 3 ),2) inTree v 0 ]

27. Data structure relationships (e,h) ⊨ ∃ v 0 v 1 v 2 TREE(R,v 0,2,[0,1]) * TREE(I,v 1,2[0]) * TREE(P,v 2,2,[0]) * ∀ v 3 ∈ TreeRecords(v 1 ). [nth(find(v 1,v 3 ),2) inTree v 0 ] * ∀ v 3 ∈ TreeRecords(v 2 ). [nth(find(v 2,v 3 ),2) inTree v 0 ] * [T = 0 ∨ T inTree v 0 ] T →

28. Deep model Process – Create data structure for predicates – Write semantic interpretation function – Write customized tactics Advantage: greater flexibility in designing tactics – Tactics can be any function that transforms the data structure – Tactic is proven correct once and used for all verifications

29. Summary of tactics Forward propagation Fold/unfold Merge – Works by pairing off identical pieces Simplify State implications – Also works by pairing off

30. Results (so far) Tree traversal – Code size: ~30 lines – Invariant size: ~10 lines – Proof check time: ~5 minutes – Main proof size: ~220 lines – Status: top level complete, lemmas need to be proven DPLL (A decision procedure for sentential satisfiability) – Code size: ~200 lines – Invariant size: ~52 lines – Status: Proof incomplete

31. Research contributions Extension of Coq separation logic reasoning to larger programs with more complex data structures. Creation of a library of useful predicates, functions and tactics – Deep model allows greater control over the design of tactics Key challenge: Performance tuning – Tradeoffs between performance and automation Development of a powerful simplification tactic – Simplification tactic executed after every major proof step – Based on term rewriting (with contextual rewriting) concepts – Automates reasoning about associativity, communtivity and other simple property classes Design decisions in creating canonical form addressed

32. Related work

33. Proposed work for PhD Finish DPLL verification Prove all underlying theorems Create improved presentation framework for the environment

34. Deep model Consider proving: a+c = a+b+c−b

35. Deep Model type expr = | Const int | Var id | Plus expr×expr | Minus expr×expr | Times expr×expr Fixpoint eval (env : id → nat) (e : expr) := match e with |Const c = ⇒ c |Var v = ⇒ env v |Plus e1 e2 = ⇒ (eval env e1) + (eval env e2) |Minus e1 e2 = ⇒ (eval env e1) − (eval env e2) |Times e1 e2 = ⇒ (eval env e1) ∗ (eval env e2) Definition simplify e := … Theorem ∀ env. ∀ e. eval env (simplify e) = eval env e

36. Deep Model Prove the following: Plus a b = simplify (Plus a (Plus b (Plus (Minus c b)))

37. Deep Model Parameterized predicate data types type expr = | Const int | Var id | Fun id × list expr

38. Verification of initialization { ∃ v0.TREE(R, v0, 2, [0,1])} T := R; I := 0; P := 0; { ∃ v0. ∃ v1. ∃ v2. TREE(R, v0, 2, [0,1]) ∗ TREE(I, v1, 2, [0]) ∗ TREE(P, v2, 2, [0]) ∗ ∀ v3 ∈ TreeRecords(v1).[nth(find(v1,v3),2) inTree v0] ∗ ∀ v3 ∈ TreeRecords(v2).[nth(find(v2,v3),2) inTree v0] ∗ [T = 0 ∨ T inTree v0]}

39. Verification of initialization { ∃ v0.TREE(R, v0, 2, [0,1])} T := R; I := 0; P := 0; {?1234}

40. Verification of initialization { ∃ v0.TREE(R, v0, 2, [0,1])} T := R; I := 0; P := 0; {?1234}

41. Verification of initialization { ∃ v0.TREE(R, v0, 2, [0,1]) * [T=R]} I := 0; P := 0; {?1234}

42. Verification of initialization { ∃ v0.TREE(R, v0, 2, [0,1]) * [T=R]} I := 0; P := 0; {?1234}

43. Verification of initialization { ∃ v0.TREE(R, v0, 2, [0,1]) * [T=R] * [I = 0]} P := 0; {?1234}

44. Verification of initialization { ∃ v0.TREE(R, v0, 2, [0,1]) * [T=R] * [I = 0]} P := 0; {?1234}

45. Verification of initialization ∃ v0.TREE(R, v0, 2, [0,1]) * [T=R] * [I = 0] * [P = 0] -> ?1234

46. Verification of initialization ∃ v0.TREE(R, v0, 2, [0,1]) * [T=R] * [I = 0] * [P = 0] -> ?1234 ?1234 = ∃ v0.TREE(R, v0, 2, [0,1]) * [T=R] * [I = 0] * [P = 0]

47. Verification of initialization ?1234 → ∃ v0. ∃ v1. ∃ v2. TREE(R, v0, 2, [0,1]) ∗ TREE(I, v1, 2, [0]) ∗ TREE(P, v2, 2, [0]) ∗ ∀ v3 ∈ TreeRecords(v1). [nth(find(v1,v3),2) inTree v0] ∗ ∀ v3 ∈ TreeRecords(v2). [nth(find(v2,v3),2) inTree v0] ∗ [T = 0 ∨ T inTree v0]

48. Verification of initialization ∃ v0.TREE(R, v0, 2, [0,1]) * [T=R] * [I = 0] * [P = 0] → ∃ v0. ∃ v1. ∃ v2. TREE(R, v0, 2, [0,1]) ∗ TREE(I, v1, 2, [0]) ∗ TREE(P, v2, 2, [0]) ∗ ∀ v3 ∈ TreeRecords(v1). [nth(find(v1,v3),2) inTree v0] ∗ ∀ v3 ∈ TreeRecords(v2). [nth(find(v2,v3),2) inTree v0] ∗ [T = 0 ∨ T inTree v0]

49. Unfold example { ∃ v0 ∃ v1 ∃ v2[Tmp l = 0] ∗ [l /= 0] ∗ [tmp r = 0] ∗ [Tmp r = 0 ∨ Tmp r ∈ TreeRecords(v0)] ∗ [nth(nth(find(v0,T)),2),0) = (Tmp r)] ∗ [nth(nth(find(v0, T )), 1), 0) = 0] ∗ [T ∈ TreeRecords(v0)] ∗ P + 0 → N ∗ P + 1 → T ∗ [T /= 0] ∗ TREE(R, v0, 2, [0,1]) ∗ TREE(I, v1, 2, [0]) * TREE(N, v2, 2, [0]) ∗ ∀ v3 ∈ TreeRecords(v1). [nth(find(v1,v3),2) inTree v0] ∗ ∀ v3 ∈ TreeRecords(v2). [nth(find(v2,v3),2) inTree v0]} ∗ T := ∗ (I+1); … {?1234}

50. Unfold example ∃ v0 ∃ v1 ∃ v2 ∃ v3 ∃ v4 I + 1 → v1 ∗ I → nth(v0, 0) ∗ TREE(nth(v0, 0), nth([I, v0, v1], 1), 2, [0]) [Tmp r = 0 ∨ Tmp r ∈ TreeRecords(v0)] ∗ [nth(nth(find(v2,T)),2),0) = (Tmp r)] ∗ [nth(nth(find(v2, T )), 1), 0) = 0] ∗ [T ∈ TreeRecords(v2)] ∗ P+0→N ∗ P+1→ T ∗ [T /= 0] ∗ TREE(I, v1, 2, [0]) * [I + 1 → v1 ∗ I → nth(v0, 0)] ∗ TREE(nth(v0, 0), nth([I, v0, v1], 1), 2, [0]) ∗ TREE(R, v2, 2, [0,1]) ∗ Empty * TREE(N,v4,2,[0]) * ∀ v5 ∈ TreeRecords([I,v0,v1]). [nth(find([I,v0,v1)]),v5),2) inTree v2] ∗ ∀ v5 ∈ TreeRecords(v4). [nth(find(v4,v5),2) inTree v2]} T := ∗ (I+1);... {?1234}

51. Unfold example ∃ v0 ∃ v1 ∃ v2 ∃ v3 ∃ v4 ∃ v5 [T = v2] ∗ I + 1 → v2 ∗ I → nth(v1, 0) ∗ TREE(nth(v1, 0), nth([I, v1, v2], 1), 2, [0]) ∗ [Tmp r = 0 ∨ Tmp r ∈ TreeRecords(v1)] ∗ [nth(nth(find(v3,T)),2),0) = (Tmp r)] ∗ [nth(nth(find(v3,T )),1), 0) = 0] ∗ [T ∈ TreeRecords(v3)] ∗ P + 0 → N ∗ P + 1 → T ∗ [T ̸= 0] ∗ TREE(R, v3, 2, [0,1]) ∗ Empty ∗ TREE(N, v5, 2, [0]) ∗ ∀ v6 ∈ TreeRecords([I,v1,v2]). [nth(find([I,v1,v2],v6),2) inTree v3] ∗ ∀ v6 ∈ TreeRecords(v5). [nth(find(v5,v6),2) inTree v3]} {?1234}

52. DPLL Efficient SAT solving algorithm for CNF expressions such as: (A v ~B v C) ^ (A v ~C v D)

53. Data structure #define VAR_COUNT 4 char assignments[VAR_COUNT]; struct clause { struct clause *next; char positive_lit[VAR_COUNT]; char negative_lit[VAR_COUNT]; char watch_var[VAR_COUNT]; struct clause *watch_next[VAR_COUNT]; struct clause *watch_prev[VAR_COUNT]; } *clauses; struct clause *watches[VAR_COUNT]; struct assignments_to_do { struct assignments_to_do *next, *prev; int var; char value; int unit_prop; } *assignments_to_do_head, *assignments_to_do_tail; struct assignment_stack { struct assignment_stack *next; int var; char value; char unit_prop; } *stack;

54. DPLL Data structure diagram

55. DPLL invariant The first part of the invariant are spacial constructs asserting the two arrays and three dynamic data structures in the heap. ARRAY(root, count, functional_representation) is a spatial predicate for arrays. The functional representation is a list of the elements. AbsExistsT v0. AbsExistsT v1. AbsExists v2. AbsExistsT v3. AbsExistsT v4. TREE(clauses,v0,sizeof_clause,[next_offset])) * TREE(assignments_to_do_head,v1,sizeof_assignment_stack,[next_offset]) * TREE(stack,v2,sizeof_assignment_stack,[next_offset]) * ARRAY(assignments,var_count,v3) * ARRAY(watches,var_count,v4) * Next, we add on two assertions that guarantee that both the assignment_stack v2 and assignment array v3 are consistent. We use (a,b)--->c as an abbreviation for nth(find(a,b),c). (AbsAll v5 in TreeRecords(v2). ([nth(v3,(v2,v5)-->stack_var_offset)==(v2,v5)-->stack_val_offset])) * (AbsAll v5 in range(0,var_count-1). ([nth(v3,v5)==0] *\/* AbsExists v6 in (TreeRecords(v2)). ([((v2,v6)-->stack_var_offset==v5 /\ (v2,v6)-->stack_val_offset==nth(v3,v5))]) )) * …