1. D IAGNOSING A BSTRACTION F AILURE IN S EPARATION L OGIC - BASED A NALYSES Arlen Cox Josh Berdine Samin Ishtiaq Christoph Wintersteiger

2. The Abstraction Refinement Dream Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

3. State of the Art Separation Logic Analysis Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure Previously Unexplored 1 2

4. Traditional Abstraction Refinement Not Our Contribution

5. Pick Abstract/Attempt Proof

6. Proof Fails

7. Diagnosing Abstraction Failure Weakest Precondition 1.An Abstract State 2.Concrete State Unreachable Reaches Error Contained in Abstract State

8. Partition the Abstract State

9. No WP() in Separation Logic Weakest Precondition

10. No WP() in Separation Logic int* p; … *p = 17; PSPACE-complete* due to aliasing * Calcagno, C., Yang, H., O’Hearn, P.W.: Computability and complexity results for a spatial assertion language for data structures. In: FSTTCS (2001)

11. Separation Logic-based Analyses  Restricted Logic Does not support separating implication ( ), general negation ( ), general conjunction ( ) Does not support separating implication ( ), general negation ( ), general conjunction ( )  Do not support backward reasoning No weakest precondition No weakest precondition  Contribution: A method to use forward analysis to diagnose failures  Contribution: A method for efficiently performing forward counterexample searches

12. …ll Example l = new ListNode(new Obj(), NULL); while(*) { l = new ListNode(new Obj(), l); l = new ListNode(new Obj(), l);} while(l != NULL) { n = l->next; n = l->next; free(l->data); free(l->data); free(l); free(l); l = n; l = n;} NULL

13. Background: Pick Abstraction Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

14. Pattern-Based Abstraction … l NULL

15. … l NULL

16. … l NULL

17. Background: Proof Attempt Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

18. Proof Search (SLL) l = new ListNode(new Obj(), NULL); while(*) { l = new ListNode(new Obj(), l); l = new ListNode(new Obj(), l);} while(l != NULL) { n = l->next; n = l->next; free(l->data); free(l->data); free(l); free(l); l = n; l = n;}

19. Proof Search (SLL) l = new ListNode(new Obj(), NULL); while(*) { l = new ListNode(new Obj(), l); l = new ListNode(new Obj(), l);} while(l != NULL) { n = l->next; n = l->next; free(l->data); free(l->data); free(l); free(l); l = n; l = n;}

20. Proof Search (SLL) l = new ListNode(new Obj(), NULL); while(*) { l = new ListNode(new Obj(), l); l = new ListNode(new Obj(), l);} while(l != NULL) { n = l->next; n = l->next; free(l->data); free(l->data); free(l); free(l); l = n; l = n;} l = new ListNode(new Obj(), l);

21. Proof Search (SLL) l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); assume(l != NULL) n = l->next; free(l->data); l = new ListNode(new Obj(), NULL); while(*) { l = new ListNode(new Obj(), l); l = new ListNode(new Obj(), l);} while(l != NULL) { n = l->next; n = l->next; free(l->data); free(l->data); free(l); free(l); l = n; l = n;}

22. Counterexamples Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

23. Traditional Bounded Model Checking l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); assume(l != NULL) n = l->next; free(l->data); free(l); l = n; assume(l == NULL) 1.Unroll Transition System 2.Check Property 3.Repeat -Can explode for deep properties -Doesn’t help proof process Not Our Contribution

24. Contribution: BMC Over Abstract Transition System l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); assume(l != NULL) n = l->next; free(l->data); 1.Unroll Abstract Transition System 2.Check Property 3.Repeat +Restricted search space +Finds counter- examples that caused this proof failure

25. Contribution: BMC Over Abstract Transition System

27. Must End in Error

28. Contribution: BMC Over Abstract Transition System

29. Unroll up to a bound

30. Contribution: BMC Over Abstract Transition System Stay in Error

31. Contribution: BMC Over Abstract Transition System Otherwise Transition According to Program

32. Contribution: BMC Over Abstract Transition System Send to SMT solver; quantifiers and all.

33. Contribution: BMC Over Abstract Transition System Send to SMT solver; quantifiers and all.

34. Encoding of Data Allocated Siz e Address p = malloc(size);

35. Data Allocated Siz e Address p = malloc(size); q = malloc(size); Encoding of

36. Data Allocated Siz e Address p = malloc(size); q = malloc(size); Encoding of

37. Data Allocated Siz e Address p = malloc(size); q = malloc(size); r = p + size; *r = 3; //(no error) Encoding of

38. Data Allocated Siz e Address p = malloc(size); q = malloc(size); r = p + size; *r = 3; //(error) Encoding of

39. Counterexample Search l = new ListNode(new Obj(), NULL); assume(l != NULL) n = l->next; free(l->data); l = new ListNode(new Obj(), l);

40. Counterexample Search l = new ListNode(new Obj(), NULL); assume(l != NULL) n = l->next; free(l->data); l = new ListNode(new Obj(), l); Just need structure. Don’t need separation logic formulas

41. Counterexample Search No Error

42. Counterexample Search l = new ListNode(new Obj(), NULL); No Error l NULL

43. Counterexample Search l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); Error Unreachable assume(l != NULL) n = l->next; free(l->data); No Error NULL l

44. Counterexample Search l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); Error Unreachable assume(l != NULL) n = l->next; free(l->data); No Error NULL l

45. Counterexample Search l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); Error Unreachable assume(l != NULL) n = l->next; free(l->data); No Error Error Unreachable assume(l != NULL) n = l->next; free(l->data); NULL l l = new ListNode(new Obj(), l);

46. Counterexample Search  Produces concrete counterexamples  Contribution: Only explores failed proof Finds counterexamples that would cause this particular proof failure Finds counterexamples that would cause this particular proof failure  Contribution: Relies on SMT solver for unrolling Property-guided, intelligent backtracking Property-guided, intelligent backtracking  Bit-precise memory model

47. Contribution: Diagnosing Failure Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

48. Diagnosing the Failure l = new ListNode(new Obj(), NULL); assume(l != NULL) n = l->next; free(l->data); Was the abstraction here responsible for failure? l = new ListNode(new Obj(), l);

49. Diagnosing the Failure assume(l != NULL) n = l->next; free(l->data); Delete Program Before Join Point l = new ListNode(new Obj(), l);

50. Diagnosing the Failure l = NULL assume(l != NULL) n = l->next; free(l->data); l = new ListNode(*, l); Synthesize Program Prefix that Creates Abstract State Precisely Error Found! l = new ListNode(new Obj(), l); Re-run Counterexample Search Non-deterministic data field

51. Diagnosing the Failure

55. for p in Join_Points(ATS) { ATS’ = Synthesize_Prefix(p, ATS) ATS’ = Synthesize_Prefix(p, ATS) CEx = Find_Counterexample(ATS’) CEx = Find_Counterexample(ATS’) if(exists CEx) { if(exists CEx) { ATS = Refine(ATS, p, CEx); ATS = Refine(ATS, p, CEx); }}

56. Picking New Abstraction Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

57. Picking New Abstraction  Partial order of abstractions  Pick next best abstraction

58. Proof Search with SLL_OBJ l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); assume(l != NULL) n = l->next; free(l->data); free(l); l = n; assume(l == NULL)

59. Conclusions Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

60. Conclusions Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure ✔ New BMC Approach Search abstract transition system instead of program Only finds causes for proof failure Use monolithic encoding Take advantage of intelligent backtracking

61. Conclusions Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure ✔ ✔ New Approach to Diagnosis Synthesize program prefix Use guided counterexample search to diagnose Find failing abstraction Find failing concrete value contained by abstraction

62. Conclusions Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure ✔ ✔ -

63. Questions?