1. Synergy: A New Algorithm for Property Checking Bhargav S. Gulavani (IIT Bombay)‏ Yamini Kannan (Microsoft Research India)‏ Thomas A. Henzinger (EPFL)‏ Aditya V. Nori (Microsoft Research India)‏ Sriram K. Rajamani (Microsoft Research India)‏ Note: I modified the original ppt file for my presentation - Erkan

2. Problem statement Check if a program satisfies a given safety property: –API usage rules –Protocols on objects Interesting programs have infinite state spaces ranging over infinite domains –This problem in general is undecidable

3. Two approaches to property checking Testing: find inputs and executions that demonstrate effectively violations of a property -> Search for bugs Verification: find a proof that all executions of the program satisfy a property -> Proof for the absence of bugs

4. Tests: presence of bugs void foo(int a)‏ { 0: i = 0; 1: c = 0; 2: while (i < 1000) { 3: c = c + i; 4: i = i + 1; } 5: assume (a <= 0); 6: assert (false); } 0 6 5 3 2 4 1 × × × × × × × × × × × × × × × × (a=-5)‏

5. Proofs: absence of bugs void foo(int y1, int y2)‏ { 0: state = 1; 1: if (y1) { 2: x0 = x0 + 1; } else { 3: x0 = x0 – 1; } 4: if (y2) { 5: x1 = x1 + 1; } else { 6: x1 = x1 – 1; } 7: assert (state == 1); } O: state=1 1: state=1 2: state=13: state=1 4: state=1 5: state=16: state=1 7: state=1 Error exponential number of tests required linear proof exists!

6. Key insights Testing works when errors are easy to find and is inefficient for finding proofs Verification works when proofs are easy to find and is inefficient for finding errors

7. Questions Can we combine “systematically” testing with verification? How does one generate/direct test cases? –Can abstraction help? Given a spurious abstract error trace, how does one perform refinement? –Can testing help?

8. Solution: Synergy Combines under- and over-approximation reasoning (testing and verification) of programs. Unifies several disparate existing algorithms in the literature: – Counterexample driven refinement approaches for verification (SLAM, BLAST)‏ – Directed testing approaches (DART)‏ – Partition refinement algorithms (Lee-Yannakakis, Paige-Tarjan)‏

9. Synergy – sketch Can extend test beyond frontier? Refine proof Construct random tests Construct initial proof Input: Program P Property ψ Test succeeded? Bug! Proof succeeded? τ = error path in failed proof f = frontier of error path yes no yes no Proof! yes no

10. Example void foo(int y)‏ { 1: do { 2: lock.state = L; 3: x = y; 4: if (*) { 5: lock.state = U; 6: y = y + 1; } 7: } while (x != y); 8: if(lock.state != L)‏ 9: error(); } Does this program obey the locking rule?

11. Example void foo(int y)‏ { 1: do { 2: lock.state = L; 3: x = y; 4: if (*) { 5: lock.state = U; 6: y = y + 1; } 7: } while (x != y); 8: if(lock.state != L ); 9: error }

12. no Example void foo(int y)‏ { 1: do { 2: lock.state = L; 3: x = y; 4: if (*) { 5: lock.state = U; 6: y = y + 1; } 7: } while (x != y); 8: if(lock.state != L)‏ 9: error(); } Can extend test beyond frontier? Refine proof Construct random tests Construct initial proof Input: Program P Property ψ Test succeeded? Bug! Proof succeeded? τ = error path in failed proof f = frontier of error path yes no yes no Proof! yes

13. Example y = 1 Can extend test beyond frontier? Refine proof Construct random tests Construct initial proof Input: Program P Property ψ Test succeeded? Bug! Proof succeeded? τ = error path in failed proof f = frontier of error path yes no yes no Proof! yes no void foo(int y)‏ { 1: do { 2: lock.state = L; 3: x = y; 4: if (*) { 5: lock.state = U; 6: y = y + 1; } 7: } while (x != y); 8: if(lock.state != L)‏ 9: error(); } 0 1 2 3 4 5 6 7 8 9 × × × × × × × × × × × × × ×

14. Example Can extend test beyond frontier? Refine proof Construct random tests Construct initial proof Input: Program P Property ψ Test succeeded? Bug! Proof succeeded? τ = error path in failed proof f = frontier of error path yes no yes no Proof! yes no void foo(int y)‏ { 1: do { 2: lock.state = L; 3: x = y; 4: if (*) { 5: lock.state = U; 6: y = y + 1; } 7: } while (x != y); 8: if(lock.state != L)‏ 9: error(); } 0 1 2 3 4 5 6 7 8 9 × × × × × × × × × × × × × × y = 1 τ=(0,1,2,3,4,7,8,9)‏ frontier

15. Example Can extend test beyond frontier? Refine proof Construct random tests Construct initial proof Input: Program P Property ψ Test succeeded? Bug! Proof succeeded? τ = error path in failed proof f = frontier of error path yes no yes no Proof! yes no void foo(int y)‏ { 1: do { 2: lock.state = L; 3: x = y; 4: if (*) { 5: lock.state = U; 6: y = y + 1; } 7: } while (x != y); 8: if(lock.state != L)‏ 9: error(); } 0 1 2 3 4 5 6 7 8⋀¬p 9 × × × × × × × × × × × × × × 8⋀p × split into two regions wrt p=(lock.state != L)‏

16. Example Can extend test beyond frontier? Refine proof Construct random tests Construct initial proof Input: Program P Property ψ Test succeeded? Bug! Proof succeeded? τ = error path in failed proof f = frontier of error path yes no yes no Proof! yes no void foo(int y)‏ { 1: do { 2: lock.state = L; 3: x = y; 4: if (*) { 5: lock.state = U; 6: y = y + 1; } 7: } while (x != y); 8: if(lock.state != L)‏ 9: error(); } 0 1 2 3 4 5 6 7 8⋀¬p 9 × × × × × × × × × × × × × × 8⋀p × τ=(0,1,2,3,4,7,,9)‏ frontier

17. Correct, the program is void foo(int y)‏ { 1: do { 2: lock.state = L; 3: x = y; 4: if (*) { 5: lock.state = U; 6: y = y + 1; } 7: } while (x != y); 8: if(lock.state != L)‏ 9: error(); } 0 1 2 3 4⋀¬s 5⋀¬s 6⋀¬r 9 × × × × × × × × × × × 7⋀¬q × 8⋀¬p × 4⋀s 5⋀s 6⋀r 7⋀q 8⋀p ×

18. Example 0 6 5 3 2 4 1 Can extend test beyond frontier? Refine proof Construct random tests Construct initial proof Input: Program P Property ψ Test succeeded? Bug! Proof succeeded? τ = error path in failed proof f = frontier of error path yes no yes no Proof! yes no void foo(int a)‏ { 0: i = 0; 1: c = 0; 2: while (i < 1000)‏ { 3: c = c + i; 4: i = i + 1; } 5: if (a <= 0)‏ 6: error(); }

19. Example 0 6 5 3 2 4 1 Can extend test beyond frontier? Refine proof Construct random tests Construct initial proof Input: Program P Property ψ Test succeeded? Bug! Proof succeeded? τ = error path in failed proof f = frontier of error path yes no yes no Proof! yes no void foo(int a)‏ { 0: i = 0; 1: c = 0; 2: while (i < 1000)‏ { 3: c = c + i; 4: i = i + 1; } 5: if (a <= 0)‏ 6: error(); } × × × × × × × × × × × × a = 45

20. Example 0 6 5 3 2 4 1 Can extend test beyond frontier? Refine proof Construct random tests Construct initial proof Input: Program P Property ψ Test succeeded? Bug! Proof succeeded? τ = error path in failed proof f = frontier of error path yes no yes no Proof! yes no void foo(int a)‏ { 0: i = 0; 1: c = 0; 2: while (i < 1000)‏ { 3: c = c + i; 4: i = i + 1; } 5: if (a <= 0)‏ 6: error(); } × × × × × × × × × × × × τ=(0,1,2,(3,4,2) 1000,5,6)‏ frontier

21. Example 0 6 5 3 2 4 1 Can extend test beyond frontier? Refine proof Construct random tests Construct initial proof Input: Program P Property ψ Test succeeded? Bug! Proof succeeded? τ = error path in failed proof f = frontier of error path yes no yes no Proof! yes no void foo(int a)‏ { 0: i = 0; 1: c = 0; 2: while (i < 1000)‏ { 3: c = c + i; 4: i = i + 1; } 5: if (a <= 0)‏ 6: error(); } × × × × × × × × × × × × × a = -5

22. Experimental Evaluation