ESC Java

Static Analysis Spectrum Power Cost Type checking Data-flow analysis Model checking Program verification AutomatedManual ESC

Is This Program Correct? int square(int n) { int k = 0, r = 0, s = 1; while(k != n) { r = r + s; s = s + 2; k = k + 1; } return r; } Type checking not enough to check this –Neither is data-flow analysis, nor model checking

Program Verification Program verification is the most powerful static analysis method –Can reason about all properties of programs Cannot fully automate But … –Can automate certain parts (ESC/Java) –Teaches how to reason about programs in a systematic way

Specifying Programs Before we check a program we must specify what it does We need formal specifications –English comments are not enough We use logic notation –Theory of pre- and post-conditions

State Predicates A predicate is a boolean expression on the program state (e.g., variables, object fields) Examples: –x == 8 –x < y –true –false –( 8 i. 0 = 0)

Using Predicates to Specify Programs We focus first on how to specify a statement Hoare triple for statement S { P } S { Q } Says that if S is started in a state that satisfies P, and S terminates, then it terminates in Q –This is the liberal version, which doesn’t care about termination –Strict version: if S is started in a state that satisfies P then S terminates in Q preconditionpostcondition

Hoare Triples. Examples. { true } x = 12 { x == 12 } { y >= 0 } x = 12 { x == 12 } { true } x = 12 { x >= 0 } (Programs satisfy many possible specifications) { x < 10 } x = x + 1 { x < 11 } { n >= 0 } x = fact(n) { x == n ! } { true } a = 0; if(x != 0) { a = 2 * x; } { a == 2*x }

Computing Hoare Triples We compute the triples using rules –One rule for each statement kind –Rules for composed statements

Assignment Assignment is the simplest operation and the trickiest one to reason about ! { y >= 2 } x = 5 { ? } { x == y } x = x + 1 { ? } { ? } x = 5 { x == y } { ? } x = x + 1 { x == y } { ? } x = x + 1 { x 2 + y 2 == z 2 } { x 2 + y 2 == z 2 } x = x + 1 { ? }

Assignment Rule Rule for assignment { Q[x := E] } x = E { Q} Examples: –{ 12 == 12 } x = 12 { x == 12 } –{ 12 >= 0 } x = 12 { x >= 0 } –{ ? } x = x + 1 { x >= 0 } –{ x >= 1 } x = x + 1 { ? } Q with x replaced by E x == 12 with x replaced by 12

Relaxing Specifications Consider { x >= 1 } x = x + 1 { x >= 2 } –It is very tight specification. We can relax it Example: { x >= 5 } x = x + 1 { x >= 2 } (since x >= 5 ) x + 1 >= 2) x = E { P } if P ) Q[x:=E] { Q }

Assignments: forward and backward Two ways to look at the rules: –Backward: given post-condition, what is pre- condition? –Forward: given pre-condition, what is post-condition? x = E { ??? } { Q } x = E { P } { ??? }

Assignments: forward and backward Two ways to look at the rules: –Backward: given post-condition, what is pre- condition? –Forward: given pre-condition, what is post-condition? x = E { Q [ x := E] } { Q } x = E { P } { ??? }

Assignments: forward and backward Two ways to look at the rules: –Backward: given post-condition, what is pre- condition? –Forward: given pre-condition, what is post-condition? x = E { Q [ x := E] } { Q } x = E { P } { }

Example of running it forward { x == y } x = x + 1 { ? }

Example of running it forward { x == y } x = x + 1 { ? }

Forward or Backward Forward reasoning –Know the precondition –Want to know what postconditon the code establishes Backward reasoning –Know what we want to code to establish –Must find in what precondition this happens Backward is used most often –Start with what you want to verify –Instead of verifying everything the code does

Weakest precondition wp(S, Q) is the weakest P such that { P } S { Q } –Order on predicates: Strong ) Weak –wp returns the “best” possible predicate wp(x := E, Q) = Q[x := E] In general: S { P } if P ) wp(S,Q) { Q }

Weakest precondition This points to a verification algorithm: –Given function body annotated with pre-condition P and post-condition Q: Compute wp of Q with respect to functon body Ask a theorem prover to show that P implies the wp The wp function we will use is liberal (P does not guarantee termination) –If using both strict and liberal in the same context, the usual notation is wlp the liberal version and wp for the strict one

Strongest postcondition sp(S, P) is the strongest Q such that { P } S { Q } –Recall: Strong ) Weak –sp returns the “best” possible predicate sp(x := E, P) = … In general: S { P } { Q } if sp(S,P) ) Q

Strongest postcondition Strongest postcondition and weakest preconditions are symmetric This points to an equivalent verification algorithm: –Given function body annotated with pre-condition P and post-condition Q: Compute sp of P with respect to functon body Ask a theorem prover to show that the sp implies Q

Composing Specifications If { P } S 1 { R } and { R } S 2 { Q} then { P } S 1 ; S 2 { Q } Example: x = x - 1; y = y - 1 { x >= y }

Composing Specifications If { P } S 1 { R } and { R } S 2 { Q} then { P } S 1 ; S 2 { Q } Example: x = x - 1; y = y - 1 { x >= y }

In terms of wp and sp wp(S 1 ; S 2, Q) = wp(S 1,wp(S 2, Q)) sp(S 1 ; S 2, P) = sp(S 2,sp(S 1, P))

Conditionals Rule for the conditional (flow graph) Example: E { P } { P 1 } if P && E ) P 1 TF { P 2 } if P && ! E ) P 2 x == 0 { x >= 0 } TF { x == 0 } since x >= 0 && x == 0 ) x == 0 { x >= 1 } since x >= 0 && x != 0 ) x >= 1

Conditionals: Forward and Backward Recall: rule for the conditional Forward: given P, find P 1 and P 2 –pick P 1 to be P && E, and P 2 to be P && ! E Backward: given P 1 and P 2, find P –pick P to be (P 1 && E) || (P 2 && ! E) –Or pick P to be (E ) P 1 ) && (! E ) P 2 ) E { P } { P 1 } provided P && E ) P 1 T F { P 2 } provided P && ! E ) P 2

Joins Rule for the join: Forward: pick P to be P 1 || P 2 Backward: pick P 1, P 2 to be P { P 1 } { P 2 } { P } provided P 1 ) P and P 2 ) P

Review E { P } { P 1 } if P && E ) P 1 TF { P 2 } if P && ! E ) P 2 { P 1 } { P 2 } { P } if P 1 ) P and P 2 ) P x = E { P } { Q } if P ) Q[x:=E] Implication is always in the direction of the control flow

Review: forward E { P } { P && E } TF { P && ! E } { P 1 } { P 2 } { P 1 || P 2 } x = E { P } { \exists … }

Review: backward E {(E ) P 1 ) && (! E ) P 2 ) } { P 1 } TF { P 2 } { P } x = E {Q[x:=E] } { Q }

Example: Absolute value static int abs(int x) ensures \result >= 0 { if (x < 0) { x = -x; } if (c > 0) { c--; } return x; } x < 0 x = -x TF c > 0 c-- TF

Example: Absolute value x < 0 x = -x TF c > 0 c-- TF

Example: Absolute value x < 0 x = -x TF c > 0 c-- TF

In Simplify > (IMPLIES TRUE (AND (IMPLIES (< x 0) (AND (IMPLIES (> c 0) (>= (- 0 x) 0)) (IMPLIES ( = (- 0 x) 0)))) (IMPLIES (>= x 0) (AND (IMPLIES (> c 0) (>= x 0)) (IMPLIES ( = x 0)))))) 1: Valid. >

So far… Framework for checking pre and post conditions of computations without loops Suppose we want to check that some condition holds inside the computation, rather than at the end static int abs(int x) { if (x < 0) { x = -x; } if (c > 0) { c--; } return x; } Say we want to check that x > 0 here

Asserts { Q && E } assert(E) { Q } Backward: wp(assert(E), Q) = Q && E Forward: sp(assert(E), P) = ??? assert(E) Q Q && E assert(E) ??? P

Example: Absolute value with assert static int abs(int x) { if (x < 0) { x = -x; assert(x > 0); } if (c > 0) { c--; } return x; } x < 0 x = -x assert(x > 0) TF c > 0 c-- TF

Example: Absolute value with assert x < 0 x = -x assert(x > 0) TF c > 0 c-- TF

Example: Absolute value with assert x < 0 x = -x assert(x > 0) TF c > 0 c-- TF

Adding the postcondition back in x < 0 x = -x assert(x > 0) TF c > 0 c-- TF

Adding the postcondition back in x < 0 x = -x assert(x > 0) TF c > 0 c-- TF

Another Example: Double Locking “An attempt to re-acquire an acquired lock or release a released lock will cause a deadlock.” Calls to lock and unlock must alternate. lock unlock

Locking Rules We assume that the boolean predicate locked says if the lock is held or not { ! locked && P[locked := true] } lock { P } –lock behaves as assert(! locked); locked = true { locked && P[locked := false] } unlock { P } –unlock behaves as assert(locked); locked = false

Locking Example … lock … x==0 … unlock x==0 T T { ! L && P[L := true] } lock { P } { L && P[L := false] } unlock { P } { ! L }

Locking Example … lock … x==0 … unlock x==0 T T { ! L && P[L := true] } lock { P } { L && P[L := false] } unlock { P } { ! L }

Locking Example: forward direction … lock … x==0 … unlock { ! locked } { ! locked && x == 0 } { ! locked && x  0 } x==0 T T { ! locked && x == 0 } { locked && x == 0 } { locked = (x == 0) } { locked && x == 0 } { ! locked && (x == 0) } { ! locked && x  0 } { ! locked }

In Simplify > (DEFPRED (isLocked L) (EQ L locked)) (IMPLIES (NOT (isLocked L)) (AND (IMPLIES (EQ x 0) (AND (NOT (isLocked L)) (AND (IMPLIES (EQ x 0) (AND TRUE (NOT FALSE))) (IMPLIES (NEQ x 0) (NOT (AND TRUE (NOT FALSE))))))) (IMPLIES (NEQ x 0) (AND (IMPLIES (EQ x 0) (AND (isLocked L) (NOT FALSE))) (IMPLIES (NEQ x 0) (NOT (AND (isLocked L) (NOT FALSE)))))))) 1: Valid. >

Reasoning About Programs with Loops Loops can be handled using conditionals and joins Consider the while(E) S statement E { P } T F S { Q } { I } { I && E } if(1) P ) I (loop invariant holds initially) and(2) I && ! E ) Q (loop establishes the postcondition) and(3) { I && E } S { I } (loop invariant is preserved) Loop invariant

Loops in the backward direction Given Q, want to find weakest invariant I that will establish (2) and (3), then pick P to be I Finding weakest I is: –Undecidable in theory –Hard in practice E { P } T F S { Q } { I } { I && E } Loop invariant

Loops in the forward direction Given P, want to find strongest invariant I that will establish (1) and (3), then pick Q to be I && E Again, finding I is hard E { P } T F S { Q } { I } { I && E } Loop invariant

Loop Example Let’s verify { x ==8 && y ==16 } while(x > 0) { x --; y -= 2; } { y==0 } –Is this true ? –We must find an appropriate invariant I Try one that holds initially x == 8 && y == 16 Try one that holds at the end y == 0 x > 0 { x == 8 && y == 16 } T F x --; y -= 2 { y == 0 } { I } { I && x > 0 }

Loop Example (II) Guess the invariant y == 2*x Must check –Initial: x == 8 && y == 16 ) y == 2*x –Preservation: y == 2*x && x > 0 ) y – 2 = 2*(x – 1) –Final: y == 2*x && x <= 0 ) y == 0 x > 0 { x == 8 && y == 16 } T F x --; y -= 2 { y == 0 } { y == 2*x } { y == 2*x && x > 0 }

Loop Example (III) Guess the invariant y == 2*x && x >= 0 Must check –Initial: x == 8 && y == 16 ) y == 2*x && x >= 0 –Preservation: y == 2*x && x >= 0 && x > 0 ) y – 2 = 2*(x – 1) && x – 1 >= 0 –Final: y == 2*x && x >= 0 && x <= 0 ) y == 0 x > 0 { x == 8 && y == 16 } T F x --; y -= 2 { y == 0 } { y == 2*x && x >= 0} { y == 2*x && x >= 0 && x > 0 }

Functions Consider a binary search function bsearch int bsearch(int a[], int p) { { sorted(a) } … { r == -1 || (r >= 0 && r < a.length && a[r] == p) } return res; } The precondition and postconditon are the function specification –Also called a contract Postcondition Precondition

Function Calls Consider a call to function F(int in) –With return variable out –With precondition Pre, postcondition Post Rule for function call: y = F(E) { P } if P ) Pre[in:=E] } { Q } and Post[out := y, in := E] ) Q

Function Call Example Consider the call { sorted(array) } y = bsearch(array, 5) if( y != -1) { { array[ y ] == 5 } Show Post[r := y, a := array, p := 5] ) array[y] == 5 –Need to know y != -1 ! Show sorted[array] ) Pre[a := array]

Function Calls: backward y = F(E) { Q } Consider a call to function F(int in) –With return variable out –With precondition Pre, postcondition Post

Function Calls: backward y = F(E) { Q } Consider a call to function F(int in) –With return variable out –With precondition Pre, postcondition Post

Pointers and aliasing x = *y + 1 { ??? } { x == 5 }

Pointers and aliasing x = *y + 1 { *y == 4 } { x == 5 } Regular rule worked in this case!

Example where regular rule doesn’t work x = *y + 1

Example where regular rule doesn’t work x = *y + 1 { ??? } { x == *y + 1 }

Example where regular rule doesn’t work x = *y + 1 { y != &x Æ x == *y + 1 } { x == *y + 1 }

Pointer stores *x = y + 1 { ??? } { y == 5 }

Pointer stores *x = y + 1 { (x == &y ) y + 1 == 5) Æ (x != &y ) y == 5) } { y == 5 }

One solution Perform case analysis based on all the possible alias relationships between the LHS of the assignment and part of the postcondition Can use a static pointer analysis to prune some cases out However, exponentially many cases in the pointer analysis, which leads to large formulas. eg, how many cases here: *x = *y + a { *z == *v + b }

Another solution Up until now the program state has been implicit. Let’s make the program state explicit... A predicate is a function from program states to booleans. So for wp(S, Q), we have: –Q(  ) returns true if Q holds in  –wp(S, Q)(  ) returns true if wp(S, Q) holds in 

New formulation of wp Suppose step(S,  ) returns the program state resuling from executing S starting in program state . Then we can express wp as follows: wp(S, Q)(  ) =

New formulation of wp Suppose step(S,  ) returns the program state resuling from executing S starting in program state . Then we can express wp as follows: wp(S, Q)(  ) = Q(step(S,  ))

Example in Simplify syntax From previous slide: wp(S, Q)(  ) = Q(step(S,  )) *x = y + 1 { y == 5 } Q is: step(S,  ) is: wp(S, Q) is:

Example in Simplify syntax From previous slide: wp(S, Q)(  ) = Q(step(S,  )) *x = y + 1 { y == 5 } Q is: (EQ (select s y) 5) step(S,  ) is: (store s (select s x) (+ (select s y) 1)) wp(S, Q) is: (EQ (select (store s (select s x) (+ (select s y) 1)) y) 5)

ESC/Java summary Very general verification framework –Based on pre- and post-conditions Generate VC from code –Instead of modelling the semantics of the code inside the theorem prover Loops and procedures require user annotations –But can try to infer these