Proofs from Tests Nels E. Beckman Aditya V. Nori Sriram K. Rajamani Robert J. Simmons Carnegie Mellon UniversityMicrosoft Research India Carnegie Mellon University

The Problem Given – a sequential program P with inputs I (say, written in C) – an assertion “ assert(e) ” Questions – Bug finding: Does there exist an execution of the program P for some input I such that the assertion is violated? – Verification: Does the assertion hold for all possible inputs?

Possible solution: Testing The “old-fashioned” way Generate test cases and see if we can find an input that violates the assertion Possible approaches: – Random test case generation – Symbolic execution – “Concolic” execution (more recent, e.g. DART/CUTE)

What’s wrong with testing? If we view testing as a “black-box” activity, Dijkstra is right! After executing many tests, we still don’t know if there is another test that can violate the assertion

If we view testing as a “white-box” activity, and “observe” what happens inside the program (along with symbolic execution), we can do several interesting things: – We can generate test cases in a directed manner to find the bug – We can prove that the assertion holds for all inputs! Our hypothesis

Tests and Proofs

Tests and Proofs a=true, b=false, limit= × × × × × × × × × × × × × × × × ×

Tests and Proofs

3’ Tests and Proofs 1 3’’ 4’5’’ 6’’ 2’ 10’ 7’8’’ 9’’ ’’ 2’’ 5’ 4’’ 6’ 8’ 7’’ 9’

DASH: Proofs from Tests – Algorithm uses only test case generation operations – Maintains two data structures: A forest of reachable concrete states (tests) – Under-approximates executions of the program A region graph (an abstraction) – Over-approximates all executions of the program – Our goal: bug finding and proving If a test reaches an error, we have found bug If we refine the abstraction so that there is *no* path from the initial region to error region, we have a proof – Handles the richness of C New operator WP α uses only aliases α that are present along concrete tests that are executed Algorithm uses recursive invocations to handle inter-procedural analysis

Empirical Evaluation Current Status Yogi works on 904 (driver, property) pairs! 31 properties on which Yogi terminates and SLAM “times/spaces out”

Key Idea - I Frontier: Boundary between tested and untested regions × × × × × × × × × frontier

Key Idea 2 WP α : New refinement operation that does not depend on whole program alias information.

DASH Algorithm Main workhorse: test case generation Use counterexamples from current abstraction to “extend frontier” and generate tests When test case generation fails, use this information to “refine” abstraction at the frontier Use only aliases that happen on the tests! Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ

Example

Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ

τ=(0,1,2,3,4,7,8,9) Example y = 1 Symbolic execution + Theorem proving frontier Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ × × × × × × × × × × × × × × 10 ×

Symbolic execution + Theorem Proving τ=(0,1,2,3,4,7,8,9) yy0y0 lock.stateL xy0y0 (x =y) = (y 0 = y 0 ) = T (lock.state != L) = (L != L) = F symbolic memory constraints

Example Symbolic execution + Theorem proving frontier Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ × × × × × × × × × × × × × × 10 ×

Template-based refinement × × × × × × × × × × × × × × 10 × 8:¬ ρ 8:ρ ρ= (lock.state != L) ××

Template-based refinement 8:¬ ρ 8:ρ ρ= (lock.state != L) ×× :¬ρ 9 × × × × × × × × × × × × × × 10 × 8:ρ

Example τ=(0,1,2,3,4,7,,9) :¬ρ 9 × × × × × × × × × × × × × × 10 × 8:ρ Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ frontier

Proof! ⋀¬s 5⋀¬s 6⋀¬r 9 × × × × × × × × × × × 7⋀¬q × 8⋀¬p × 4⋀s 5⋀s 6⋀r 7⋀q 8⋀p × Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ 10

Template-based refinement frontier op IF(i>=j) ASSGN(i=i+j) CALL(foo(i,j)) op S k-1 SkSk × S k-2 T × witness

Template-based refinement S k-1 SkSk × S k-2 T × op S k-1 ∧¬ρ S k-1 ∧ρ SkSk × S k-2 T × op suitable predicate No theorem prover calls!

Candidates for suitable predicates S k-1 ∧¬ρ S k-1 ∧ρ SkSk × S k-2 T × op A.Strongest postcondition (SP) B.Weakest precondition (WP) Increased number of iterations, leading to non- termination in many cases Explodes in the presence of aliasing

What’s wrong with WP ? ASSGN(i=j) S k-1 *a<10 × S k-2 T ×

What’s wrong with WP ? S k-1 ∧¬ρ S k-1 ∧ρ *a<10 × S k-2 T × ASSGN(i=j) ρ = (a≠&i ∧ *a<10) ∨ (a=&i ∧ j<10) ρ = WP(*a<10, “i = j”)

What’s wrong with WP ? S k-1 ∧¬ρ S k-1 ∧ρ *a+*b<10 × S k-2 T × ASSGN(i=j) ρ = (a≠&i ∧ b≠&i ∧ *a+*b<10) ∨ (a=&i ∧ b≠&i ∧ j+*b<10) ∨ (a≠&i ∧ b=&i ∧ *a+j<10) ∨ (a=&j ∧ b=&i ∧ j+j<10)

What’s wrong with WP ? ¬((a≠&i ∧ b≠&i ∧ *a+*b<10) || (a=&i ∧ b≠&i ∧ j+*b<10) || (a≠&i ∧ b=&i ∧ *a+j<10) || (a=&j ∧ b=&i ∧ j+j<10)) *a+*b<10 × ASSGN(i=j) (a≠&i ∧ b≠&i ∧ *a+*b<10) || (a=&i ∧ b≠&i ∧ j+*b<10) || (a≠&i ∧ b=&i ∧ *a+j<10) || (a=&j ∧ b=&i ∧ j+j<10) In practice a global alias analysis required to prune the formula generated by WP

Deriving a suitable predicate *a+*b<10 ASSGN(i=j) a≠&i ∧ b≠&i ∧ *a+*b≥10a≠&i ∧ b≠&i ∧ *a+*b<10 a=&i ∧ b≠&i ∧ j+*b≥10 a≠&i ∧ b=&i ∧ *a+j≥10 a=&i ∧ b=&i ∧ j+j≥10 a=&i ∧ b≠&i ∧ j+*b<10 a≠&i ∧ b=&i ∧ *a+j<10 a=&i ∧ b=&i ∧ j+j<10 ×

Deriving a suitable predicate *a+*b<10 ASSGN(i=j) a≠&i ∧ b≠&i ∧ *a+*b≥10a≠&i ∧ b≠&i ∧ *a+*b<10 a=&i ∧ b≠&i ∧ j+*b≥10 a≠&i ∧ b=&i ∧ *a+j≥10 a=&i ∧ b=&i ∧ j+j≥10 a=&i ∧ b≠&i ∧ j+*b<10 a≠&i ∧ b=&i ∧ *a+j<10 a=&i ∧ b=&i ∧ j+j<10 ×

Refining with suitable predicate WP α *a+*b<10 ASSGN(i=j) a=&i ∧ b≠&i ∧ j+*b≥10 a ≠ &i ∨ b=&i ∨ j+*b<10 × - No global alias analysis required! - WP α stronger than WP and weaker than SP !

WP α :Template-based refinement Theorem: WP α (S k, op) is a suitable predicate for template-based refinement No theorem prover calls! S k-1 SkSk × S k-2 T × op S k-1 ∧¬ρ S k-1 ∧ρ SkSk × S k-2 T × op suitable predicate

Example

p = p1 p2 = malloc(); p2->lock = 0 p1 = malloc(); p1->lock = 0 Aliasing Example assume(p1->lock =1  p2->lock=1) p->lock = assume(!(p1->lock =1  p2->lock=1)) p = p2

Aliasing Example × × × × × × × frontier ρ = WP α = (p1->lock=1  p2->lock=1)

Aliasing Example : ¬ρ × × × × × × × 3: ρ frontier  = WP α = ¬((p≠p1  p≠p2)  ¬(p1->lock=1  p2->lock=1))

2 :  2:¬  Aliasing Example 0 1 3: ¬ρ × × × × × × × 3: ρ

2 :  2:¬  Aliasing Example - Proof 0 1: ¬μ 3: ¬ρ × × × × × × × 3: ρ 1: μ

Generalized Example

What about procedures? Key idea Perform a recursive Dash query on the called procedure and use the result to either generate a test or compute WP α S k-1 SkSk × S k-2 T × CALL(foo(i,j)) frontier

Interprocedural analysis S k-1 SkSk × S k-2 T × CALL(foo(i,j)) frontier

Interprocedural analysis S k-1 SkSk × S k-2 T × CALL(foo(i,j)) Dash[assume(φ 1 ), foo(i, j), assert(¬φ 2 )] - pass: perform refinement - fail: generate test

Soundness and Complexity Theorem. If Dash terminates on (P,φ), then either of the following is true: – If Dash returns (“pass”, Σ ≃ ), then Σ ≃ is a proof that P cannot reach ¬ φ – If Dash returns (“fail”, t ), then t certifies that P reaches ¬ φ Theorem. The complexity of Dash is precisely one theorem-prover call per iteration

Soundness and Complexity Theorem. If Dash terminates on (P,φ), then either of the following is true: – If Dash returns (“pass”, Σ ≃ ), then Σ ≃ is a proof that P cannot reach ¬ φ – If Dash returns (“fail”, t ), then t certifies that P reaches ¬ φ Theorem. Proofs at the same complexity as testing!

Empirical Evaluation Current Status Yogi works on 904 (driver, property) pairs! 31 properties on which Yogi terminates and SLAM “times/spaces out”

Acknowledgments Tom Ball Nikolaj Bjorner Leonardo de Moura Patrice Godefroid Akash Lal Jim Larus Rustan Leino Kanika Nema G. Ramalingam Sai Tetali Aditya Thakur

Rigorous Software Engineering Microsoft Research India