1. Proofs from Tests Nels E. Beckman Aditya V. Nori Sriram K. Rajamani Robert J. Simmons Carnegie Mellon UniversityMicrosoft Research India Carnegie Mellon University

2. The Problem Given – a sequential program P with inputs I (say, written in C) – an assertion “ assert(e) ” Questions – Bug finding: Does there exist an execution of the program P for some input I such that the assertion is violated? – Verification: Does the assertion hold for all possible inputs?

3. Possible solution: Testing The “old-fashioned” way Generate test cases and see if we can find an input that violates the assertion Possible approaches: – Random test case generation – Symbolic execution – “Concolic” execution (more recent, e.g. DART/CUTE)

4. What’s wrong with testing? If we view testing as a “black-box” activity, Dijkstra is right! After executing many tests, we still don’t know if there is another test that can violate the assertion

5. If we view testing as a “white-box” activity, and “observe” what happens inside the program (along with symbolic execution), we can do several interesting things: – We can generate test cases in a directed manner to find the bug – We can prove that the assertion holds for all inputs! Our hypothesis

6. Tests and Proofs 1 3 45 6 2 10 78 9 12 11

7. Tests and Proofs a=true, b=false, limit=2 1 3 45 6 2 10 78 9 × × × × × × × × × 12 11 × × × × × × × ×

8. Tests and Proofs 1 3 45 6 2 10 78 9 12 11

9. 3’ Tests and Proofs 1 3’’ 4’5’’ 6’’ 2’ 10’ 7’8’’ 9’’ 12 11 10’’ 2’’ 5’ 4’’ 6’ 8’ 7’’ 9’

10. DASH: Proofs from Tests – Algorithm uses only test case generation operations – Maintains two data structures: A forest of reachable concrete states (tests) – Under-approximates executions of the program A region graph (an abstraction) – Over-approximates all executions of the program – Our goal: bug finding and proving If a test reaches an error, we have found bug If we refine the abstraction so that there is *no* path from the initial region to error region, we have a proof – Handles the richness of C New operator WP α uses only aliases α that are present along concrete tests that are executed Algorithm uses recursive invocations to handle inter-procedural analysis

11. Empirical Evaluation Current Status Yogi works on 904 (driver, property) pairs! 31 properties on which Yogi terminates and SLAM “times/spaces out”

12. Key Idea - I Frontier: Boundary between tested and untested regions 0 1 2 3 4 7 8 9 × × × × × × × × × frontier

13. Key Idea 2 WP α : New refinement operation that does not depend on whole program alias information.

14. DASH Algorithm Main workhorse: test case generation Use counterexamples from current abstraction to “extend frontier” and generate tests When test case generation fails, use this information to “refine” abstraction at the frontier Use only aliases that happen on the tests! Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ

15. Example

16. Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ

17. τ=(0,1,2,3,4,7,8,9) Example y = 1 Symbolic execution + Theorem proving frontier Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ 0 1 2 3 4 5 6 7 8 9 × × × × × × × × × × × × × × 10 ×

18. Symbolic execution + Theorem Proving τ=(0,1,2,3,4,7,8,9) yy0y0 lock.stateL xy0y0 (x =y) = (y 0 = y 0 ) = T (lock.state != L) = (L != L) = F symbolic memory constraints

19. Example Symbolic execution + Theorem proving frontier Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ 0 1 2 3 4 5 6 7 8 9 × × × × × × × × × × × × × × 10 ×

20. Template-based refinement 0 1 2 3 4 5 6 7 8 9 × × × × × × × × × × × × × × 10 × 8:¬ ρ 8:ρ 9 8 9 ρ= (lock.state != L) ××

21. Template-based refinement 8:¬ ρ 8:ρ 9 8 9 ρ= (lock.state != L) ×× 0 1 2 3 4 5 6 7 8 :¬ρ 9 × × × × × × × × × × × × × × 10 × 8:ρ

22. Example τ=(0,1,2,3,4,7,,9) 0 1 2 3 4 5 6 7 8 :¬ρ 9 × × × × × × × × × × × × × × 10 × 8:ρ Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ frontier

23. Proof! 0 1 2 3 4⋀¬s 5⋀¬s 6⋀¬r 9 × × × × × × × × × × × 7⋀¬q × 8⋀¬p × 4⋀s 5⋀s 6⋀r 7⋀q 8⋀p × Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes no Proof! yes no Input: Program P Property ψ 10

24. Template-based refinement frontier op IF(i>=j) ASSGN(i=i+j) CALL(foo(i,j)) op S k-1 SkSk × S k-2 T × witness

25. Template-based refinement S k-1 SkSk × S k-2 T × op S k-1 ∧¬ρ S k-1 ∧ρ SkSk × S k-2 T × op suitable predicate No theorem prover calls!

26. Candidates for suitable predicates S k-1 ∧¬ρ S k-1 ∧ρ SkSk × S k-2 T × op A.Strongest postcondition (SP) B.Weakest precondition (WP) Increased number of iterations, leading to non- termination in many cases Explodes in the presence of aliasing

27. What’s wrong with WP ? ASSGN(i=j) S k-1 *a<10 × S k-2 T ×

28. What’s wrong with WP ? S k-1 ∧¬ρ S k-1 ∧ρ *a<10 × S k-2 T × ASSGN(i=j) ρ = (a≠&i ∧ *a<10) ∨ (a=&i ∧ j<10) ρ = WP(*a<10, “i = j”)

29. What’s wrong with WP ? S k-1 ∧¬ρ S k-1 ∧ρ *a+*b<10 × S k-2 T × ASSGN(i=j) ρ = (a≠&i ∧ b≠&i ∧ *a+*b<10) ∨ (a=&i ∧ b≠&i ∧ j+*b<10) ∨ (a≠&i ∧ b=&i ∧ *a+j<10) ∨ (a=&j ∧ b=&i ∧ j+j<10)

30. What’s wrong with WP ? ¬((a≠&i ∧ b≠&i ∧ *a+*b<10) || (a=&i ∧ b≠&i ∧ j+*b<10) || (a≠&i ∧ b=&i ∧ *a+j<10) || (a=&j ∧ b=&i ∧ j+j<10)) *a+*b<10 × ASSGN(i=j) (a≠&i ∧ b≠&i ∧ *a+*b<10) || (a=&i ∧ b≠&i ∧ j+*b<10) || (a≠&i ∧ b=&i ∧ *a+j<10) || (a=&j ∧ b=&i ∧ j+j<10) In practice a global alias analysis required to prune the formula generated by WP

31. Deriving a suitable predicate *a+*b<10 ASSGN(i=j) a≠&i ∧ b≠&i ∧ *a+*b≥10a≠&i ∧ b≠&i ∧ *a+*b<10 a=&i ∧ b≠&i ∧ j+*b≥10 a≠&i ∧ b=&i ∧ *a+j≥10 a=&i ∧ b=&i ∧ j+j≥10 a=&i ∧ b≠&i ∧ j+*b<10 a≠&i ∧ b=&i ∧ *a+j<10 a=&i ∧ b=&i ∧ j+j<10 ×

32. Deriving a suitable predicate *a+*b<10 ASSGN(i=j) a≠&i ∧ b≠&i ∧ *a+*b≥10a≠&i ∧ b≠&i ∧ *a+*b<10 a=&i ∧ b≠&i ∧ j+*b≥10 a≠&i ∧ b=&i ∧ *a+j≥10 a=&i ∧ b=&i ∧ j+j≥10 a=&i ∧ b≠&i ∧ j+*b<10 a≠&i ∧ b=&i ∧ *a+j<10 a=&i ∧ b=&i ∧ j+j<10 ×

33. Refining with suitable predicate WP α *a+*b<10 ASSGN(i=j) a=&i ∧ b≠&i ∧ j+*b≥10 a ≠ &i ∨ b=&i ∨ j+*b<10 × - No global alias analysis required! - WP α stronger than WP and weaker than SP !

34. WP α :Template-based refinement Theorem: WP α (S k, op) is a suitable predicate for template-based refinement No theorem prover calls! S k-1 SkSk × S k-2 T × op S k-1 ∧¬ρ S k-1 ∧ρ SkSk × S k-2 T × op suitable predicate

35. Example

36. p = p1 p2 = malloc(); p2->lock = 0 p1 = malloc(); p1->lock = 0 Aliasing Example assume(p1->lock =1  p2->lock=1) 0 1 2 3 4 p->lock = 1 6 5 7 assume(!(p1->lock =1  p2->lock=1)) p = p2

37. Aliasing Example 0 1 2 3 4 6 5 7 × × × × × × × frontier ρ = WP α = (p1->lock=1  p2->lock=1)

38. Aliasing Example 0 1 2 3: ¬ρ 4 6 5 7 × × × × × × × 3: ρ frontier  = WP α = ¬((p≠p1  p≠p2)  ¬(p1->lock=1  p2->lock=1))

39. 2 :  2:¬  Aliasing Example 0 1 3: ¬ρ 4 6 5 7 × × × × × × × 3: ρ

40. 2 :  2:¬  Aliasing Example - Proof 0 1: ¬μ 3: ¬ρ 4 6 5 7 × × × × × × × 3: ρ 1: μ

41. Generalized Example

42. What about procedures? Key idea Perform a recursive Dash query on the called procedure and use the result to either generate a test or compute WP α S k-1 SkSk × S k-2 T × CALL(foo(i,j)) frontier

43. Interprocedural analysis S k-1 SkSk × S k-2 T × CALL(foo(i,j)) frontier

44. Interprocedural analysis S k-1 SkSk × S k-2 T × CALL(foo(i,j)) Dash[assume(φ 1 ), foo(i, j), assert(¬φ 2 )] - pass: perform refinement - fail: generate test

45. Soundness and Complexity Theorem. If Dash terminates on (P,φ), then either of the following is true: – If Dash returns (“pass”, Σ ≃ ), then Σ ≃ is a proof that P cannot reach ¬ φ – If Dash returns (“fail”, t ), then t certifies that P reaches ¬ φ Theorem. The complexity of Dash is precisely one theorem-prover call per iteration

46. Soundness and Complexity Theorem. If Dash terminates on (P,φ), then either of the following is true: – If Dash returns (“pass”, Σ ≃ ), then Σ ≃ is a proof that P cannot reach ¬ φ – If Dash returns (“fail”, t ), then t certifies that P reaches ¬ φ Theorem. Proofs at the same complexity as testing!

47. Empirical Evaluation Current Status Yogi works on 904 (driver, property) pairs! 31 properties on which Yogi terminates and SLAM “times/spaces out”

48. Acknowledgments Tom Ball Nikolaj Bjorner Leonardo de Moura Patrice Godefroid Akash Lal Jim Larus Rustan Leino Kanika Nema G. Ramalingam Sai Tetali Aditya Thakur

49. Rigorous Software Engineering Microsoft Research India http://research.microsoft.com/research/rse