ECI 2007: Specification and Verification of Object- Oriented Programs Lecture 1

The Verifying Compiler “ A verifying compiler uses automated.. reasoning to check the correctness of the program that it compiles. Correctness is specified by types, assertions,.. and other redundant annotations that accompany the program.” [Hoare, 2004]

Theorem Proving and Software Meets spec/Found Bug Theorem in a logic Program Specification Semantics VC generation Validity Provability (theorem proving) Soundness: –If the theorem is valid then the program meets specification –If the theorem is provable then it is valid

Spec# Approach for a Verifying Compiler As source language we use C# As specifications we use method contracts, invariants, and also class, field and type annotations As program logic we use Dijkstra’s weakest preconditions For automatic verification we use type checking, verification condition generation (VCG) and automatic theorem proving (ATP)

Demo (Spec#)

Spec# Tool Architecture Spec# (annotated C#) Boogie PL Spec# Compiler VC Generator Formulas Automatic Theorem Prover Z3 SMT solver

Lectures 1.Logic of Object-oriented Programs 2.Invariants and Ownership 3.Verification Condition Generation 4.Proving Verification Conditions From Boogie PL To Formulas From Spec# To BoogiePL SMT solvers

Programs ! Theorems = Axiomatic Semantics Consists of: –A language for making assertions about programs –Rules for establishing when assertions hold Typical assertions: –During the execution, only non-null pointers are dereferenced –This program terminates with x = 0 Partial vs. total correctness assertions –Safety vs. liveness properties –Usually focus on safety (partial correctness)

Hoare Triples Partial correctness: { A } s { B} –When you start s in any state that satisfies A –If the execution of s terminates –It does so in a state that satisfies B Total correctness: [ A ] s [ B ] –When you start s in any state that satisfies A –The execution of s terminates and –It does so in a state that satisfies B Defined inductively on the structure of statements

Hoare Rules {A} s 1 {C} {C} s 2 {B} { A } s 1 ; s 2 {B} {A Æ E} s 1 {B} {A Æ : E} s 2 {B} {A} if E then s 1 else s 2 {B} {I Æ E} s {I} I Æ : E ) B { I} while E do s {B}

Hoare Rules: Assignment Example: { A } x := x + 2 {x >= 5 }. What is A? General rule: Surprising how simple the rule is ! The key is to compute “backwards” the precondition from the postcondition Forward rule is more complicated: { B[E/x] } x := E {B} { A } x := E { 9 x’. A[x’/x] Æ x = E[x’/x] }

Weakest preconditions Dijkstra’s idea: To verify that { A } s { B} a) Let Pre(s, B) = {A’ | { A’} s { B} } b) (Pre(s,B),  ) is a lattice - false  Pre(s,B) - if  Pre(s,B) and  Pre(s,B), then  Pre(s,B) - if  Pre(s,B) and  Pre(s,B), then  Pre(s,B) c) WP(s, B) = lub  (Pre(s, B)) d) Compute WP(s, B) and prove A  WP(s, B)

Predicate lattice falsetrue ) strong weak Pre(s, B) weakest precondition: WP(s, B) A

Weakest preconditions WP(x := E, B) = B[E/x] WP(s 1 ; s 2, B) = WP(s 1, WP(s 2, B)) WP(if E then s 1 else s 2, B) = E ) WP(s 1, B) Æ : E ) WP(s 2, B) WP(assert E, B) = E  B

Example returns c requires true ensures c = a  b bool or(bool a, bool b) { if (a) c := true else c := b } Conjecture to be proved: true  (a  true = a  b)  (  a  b = a  b) WP(S, c = a  b) = (a  true = a  b)  (  a  b = a  b) S

Weakest preconditions (Contd.) What about loops? Define a family of weakest preconditions –WP k (while E do S, B) = weakest precondition from which if the loop terminates in k or fewer iterations, then it terminates in B WP 0 = : E ) B WP i+1 = WP i  (E ) WP(s, WP i )) WP(while E do S, B) = Æ k ¸ 0 WP k = glb  {WP k | k ¸ 0} –Hard to compute –Can we find something easier yet sufficient ?

Not quite weakest preconditions Recall what we are trying to do: falsetrue ) strong weak Pre(s, B) weakest precondition: WP(s, B) A verification condition: VC(s, B) We shall construct a verification condition: VC(s, B) –The loops are annotated with loop invariants ! –VC is guaranteed stronger than WP –But hopefully still weaker than A: A ) VC(s, B) ) WP(s, B)

Desugaring loops WP(havoc x, B) =  x. B WP(assume E, B) = E  B while I,T E do s  assert I; havoc T; assume I; if (E) { s; assert I; assume false } –I is the loop invariant (provided by the programmer) –T is the set of loop targets (can be approximated by scanning the loop body)

returns c requires b >= 0 ensures c = a + b int add(int a, int b) { int t; t := b; c := a; invariant t  0  c = a+b-t while (t > 0) { c := c + 1; t := t – 1; } Example 1 Conjecture to be proved: b  0  VC(A, c = a + b) L A returns c requires b >= 0 ensures c = a + b int add(int a, int b) { int t; t := b; c := a; assert t  0  c = a+b-t; havoc c,t; assume t  0  c = a+b-t; if (t > 0) { c := c + 1; t := t – 1; assert t  0  c = a+b-t; assume false; } A’

VC(L, c = a + b)   c,t. (t  0  c = a + b – t   t > 0  t - 1  0  c + 1 = a + b – (t - 1)  t  0  c = a + b) L returns c requires b >= 0 ensures c = a + b int add(int a, int b) { int t; t := b; c := a; assert t  0  c = a+b-t; havoc c,t; assume t  0  c = a+b-t; if (t > 0) { c := c + 1; t := t – 1; assert t  0  c = a+b-t; assume false; } A’ VC(A’, c = a + b)  b  0  a = a + b – b   c,t. (t  0  c = a + b – t   t > 0  t - 1  0  c + 1 = a + b – (t - 1)  t  0  c = a + b) Conjecture to be proved: b  0  VC(A’, c = a + b)

returns c ensures c = a + b int add(int a, int b) { int t; t := b; c := a; invariant c = a+b-t while (t  0) { c := c + 1; t := t – 1; } Example 2 Conjecture to be proved: true  VC(A, c = a + b) L A returns c ensures c = a + b int add(int a, int b) { int t; t := b; c := a; assert c = a+b-t; havoc c,t; assume c = a+b-t; if (t  0) { c := c + 1; t := t – 1; assert c = a+b-t; assume false; } A’

Invariants Are Not Easy Consider the following code from QuickSort int partition(int *a, int L 0, int H 0, int pivot) { int L = L 0, H = H 0 ; while(L < H) { while(a[L] < pivot) L ++; while(a[H] > pivot) H --; if(L < H) { swap a[L] and a[H] } L++; } return L } Consider verifying only memory safety What is the loop invariant for the outer loop ?

Verification conditions (Contd.) What about procedure calls? –Annotate each procedure with a precondition pre, a modifies clause M, a postcondition post –f()  assert pre; havoc M; assume post;

Program verification Annotate each procedure with a precondition, a modifies clause, and a postcondition and each loop with an invariant and loop targets Verify each procedure separately requires pre modifies M ensures post f() { S; } Verify that the formula VC(assume pre; S; assert post, true) is valid and the targets of S are contained in M

Exponential Verification Conditions WP(x := E, B) = B[E/x] VC(if E then s 1 else s 2, B) = (E ) VC(s 1, B)) Æ ( : E ) VC(s 2, B)) The following two culprits may result in a VC that is exponential in the size of the program.

s 1  x := x + x;. x := x + x; n statements VC(s 1, x > 1) has 2 n occurrences of x. s 2  if (b 1 ) x := x+1; else x := x+2;. if (b n ) x := x+1; else x := x+2; n statements VC(s 2, x > 1) has at least 2 n-1 occurrences of .

Efficient VC Generation How can we get a formula that is linear in the size of the (loop-free and call-free) program?

Eliminate assignments Dynamic single assignment (DSA) form –There is at most one definition for each variable on each path –Replace defs/uses with new incarnations x := x+1 with x n+1 = x n + 1 –Replace havoc x with new incarnations x n+1 –At join points unify variable incarnations Eliminate assignments by replacing –x := E with assume x = E

Example assume x = 1; x := x + 1; if (x = 0) { x := x + 2; } else { x := x + 3; } assert x = 5 assume x 0 = 1; assume x 1 = x 0 + 1; if (x 1 = 0) { assume x 2 = x 1 + 2; assume x 4 = x 2 ; } else { assume x 3 = x 1 + 3; assume x 4 = x 3 ; } assert x 4 = 5;

Efficient VC Generation (I) WP(x := E, B) = B[E/x] compare with WP(assume x = E, B) = (x = E  B) Intuition: The decorated variables essentially give names to sub-expressions in the original VC. The unique definition of each variable now occurs as an assumption.

Efficient VC Generation (II) VC(if E then s 1 else s 2, B) =  E  VC(s 1, B)  : E  VC(s 2, B) VC(if E then s 1 else s 2, B) =  E  VC(s 1, true)  B  : E  VC(s 2, true)  B compare with  E  VC(s 1, true)  : E  VC(s 2, true)  B B = B occurs only once in the VC eliminating the exponential blowup.

Unstructured control flow Control flow is not always structured –exceptions, goto statements How do we generated verification conditions for programs with unstructured control flow?

Unstructured control flow Eliminate loops, procedure calls, assignments –Passive program with acyclic control flow Eliminate if-then-else statements using goto statements Finally, program is a collection of blocks Each block is –l: A 1,…,A m ; goto l 1,…,l n –l is block label –A is either assume E or assert E Distinguished “start” label

Example assume x 0 = 1; assume x 1 = x 0 + 1; if (x 1 = 0) { assume x 2 = x 1 + 2; assume x 4 = x 2 ; } else { assume x 3 = x 1 + 3; assume x 4 = x 3 ; } assert x 4 = 5; start: assume x 0 = 1; goto l 1 ; l 1 : assume x 1 = x 0 + 1; goto l 2, l 3 ; l 2 : assume x 1 = 0; assume x 2 = x 1 + 2; assume x 4 = x 2 ; goto l 4 ; l 3 : assume x 1  0; assume x 3 = x 1 + 3; assume x 4 = x 3 ; goto l 4 ; l 4 : assert x 4 = 5

VC Generation for Unstructured Control Flow For each block A = l: S goto l 1,..,l n introduce a boolean variable A ok –A ok holds iff all executions starting at A do not end in a failed assertion Introduce a Block Equation for each block A: BE A  A ok  VC(S,  B  Succ(A). B ok ) VC of entire program: (  A. BE A )  Start ok

Spec# Tool Architecture Spec# (annotated C#) Boogie PL Spec# Compiler VC Generator Formulas Automatic Theorem Prover Z3 SMT solver

Boogie PL: Parts Boogie PL source contains a first order theory to encode the background semantics of the source language and the program, described –constants, functions and axioms an imperative part used to encode the traces of the source program, described by –procedures, pre and postconditions, mutable variables, and unstructured code

Limits of Boogie PL Boogie PL does not contain structured types a heap expressions with side effects visibility subtyping dynamic dispatch

Declaration procedure Find(xs: [int] int, ct: int, x: int) returns (result: int); Implementation implementation Find(xs: [int] int, ct: int, x: int) returns (result: int) {…} Call call r := Find(bits, 100, 32) Boogie PL: Procedures

Boogie PL: Procedure Specifications proc Find(xs: [int] int, ct: int, x: int) returns (result: int); requires ct ≥ 0; ensures result ≥ 0  result < ct  xs[result] = x; ensures result < 0  !(  i:int :: 0 ≤ i  i < ct  xs[i] = x);

var xs: [int] int; var ct: int; proc Find(x: int) returns (result: int); requires ct ≥ 0; ensures result ≥ 0  result < ct  xs[result]=x; ensures result < 0  !(  i:int :: 0 ≤ i  i < ct  xs[i] = x); impl Find(x: int) returns (result: int) { start:ct := 0; result := -1; return; } A Bogus Implementation?

Postconditions must say which variables x might change –modifies x; variables not mentioned are not allowed to change often relate pre-state and post-state –ensures x == old(x)+1; More about Postconditions

proc Find(x: int) returns (result: int); … modifies ct; // would allow the previous implementation ensures ct == old(ct); // would disallow the change despite modifies clause