D IAGNOSING A BSTRACTION F AILURE IN S EPARATION L OGIC - BASED A NALYSES Arlen Cox Josh Berdine Samin Ishtiaq Christoph Wintersteiger

The Abstraction Refinement Dream Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

State of the Art Separation Logic Analysis Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure Previously Unexplored 1 2

Traditional Abstraction Refinement Not Our Contribution

Pick Abstract/Attempt Proof

Proof Fails

Diagnosing Abstraction Failure Weakest Precondition 1.An Abstract State 2.Concrete State Unreachable Reaches Error Contained in Abstract State

Partition the Abstract State

No WP() in Separation Logic Weakest Precondition

No WP() in Separation Logic int* p; … *p = 17; PSPACE-complete* due to aliasing * Calcagno, C., Yang, H., O’Hearn, P.W.: Computability and complexity results for a spatial assertion language for data structures. In: FSTTCS (2001)

Separation Logic-based Analyses  Restricted Logic Does not support separating implication ( ), general negation ( ), general conjunction ( ) Does not support separating implication ( ), general negation ( ), general conjunction ( )  Do not support backward reasoning No weakest precondition No weakest precondition  Contribution: A method to use forward analysis to diagnose failures  Contribution: A method for efficiently performing forward counterexample searches

…ll Example l = new ListNode(new Obj(), NULL); while(*) { l = new ListNode(new Obj(), l); l = new ListNode(new Obj(), l);} while(l != NULL) { n = l->next; n = l->next; free(l->data); free(l->data); free(l); free(l); l = n; l = n;} NULL

Background: Pick Abstraction Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

Pattern-Based Abstraction … l NULL

… l NULL

… l NULL

Background: Proof Attempt Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

Proof Search (SLL) l = new ListNode(new Obj(), NULL); while(*) { l = new ListNode(new Obj(), l); l = new ListNode(new Obj(), l);} while(l != NULL) { n = l->next; n = l->next; free(l->data); free(l->data); free(l); free(l); l = n; l = n;}

Proof Search (SLL) l = new ListNode(new Obj(), NULL); while(*) { l = new ListNode(new Obj(), l); l = new ListNode(new Obj(), l);} while(l != NULL) { n = l->next; n = l->next; free(l->data); free(l->data); free(l); free(l); l = n; l = n;}

Proof Search (SLL) l = new ListNode(new Obj(), NULL); while(*) { l = new ListNode(new Obj(), l); l = new ListNode(new Obj(), l);} while(l != NULL) { n = l->next; n = l->next; free(l->data); free(l->data); free(l); free(l); l = n; l = n;} l = new ListNode(new Obj(), l);

Proof Search (SLL) l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); assume(l != NULL) n = l->next; free(l->data); l = new ListNode(new Obj(), NULL); while(*) { l = new ListNode(new Obj(), l); l = new ListNode(new Obj(), l);} while(l != NULL) { n = l->next; n = l->next; free(l->data); free(l->data); free(l); free(l); l = n; l = n;}

Counterexamples Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

Traditional Bounded Model Checking l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); assume(l != NULL) n = l->next; free(l->data); free(l); l = n; assume(l == NULL) 1.Unroll Transition System 2.Check Property 3.Repeat -Can explode for deep properties -Doesn’t help proof process Not Our Contribution

Contribution: BMC Over Abstract Transition System l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); assume(l != NULL) n = l->next; free(l->data); 1.Unroll Abstract Transition System 2.Check Property 3.Repeat +Restricted search space +Finds counter- examples that caused this proof failure

Contribution: BMC Over Abstract Transition System

Must End in Error

Contribution: BMC Over Abstract Transition System

Unroll up to a bound

Contribution: BMC Over Abstract Transition System Stay in Error

Contribution: BMC Over Abstract Transition System Otherwise Transition According to Program

Contribution: BMC Over Abstract Transition System Send to SMT solver; quantifiers and all.

Contribution: BMC Over Abstract Transition System Send to SMT solver; quantifiers and all.

Encoding of Data Allocated Siz e Address p = malloc(size);

Data Allocated Siz e Address p = malloc(size); q = malloc(size); Encoding of

Data Allocated Siz e Address p = malloc(size); q = malloc(size); Encoding of

Data Allocated Siz e Address p = malloc(size); q = malloc(size); r = p + size; *r = 3; //(no error) Encoding of

Data Allocated Siz e Address p = malloc(size); q = malloc(size); r = p + size; *r = 3; //(error) Encoding of

Counterexample Search l = new ListNode(new Obj(), NULL); assume(l != NULL) n = l->next; free(l->data); l = new ListNode(new Obj(), l);

Counterexample Search l = new ListNode(new Obj(), NULL); assume(l != NULL) n = l->next; free(l->data); l = new ListNode(new Obj(), l); Just need structure. Don’t need separation logic formulas

Counterexample Search No Error

Counterexample Search l = new ListNode(new Obj(), NULL); No Error l NULL

Counterexample Search l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); Error Unreachable assume(l != NULL) n = l->next; free(l->data); No Error NULL l

Counterexample Search l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); Error Unreachable assume(l != NULL) n = l->next; free(l->data); No Error NULL l

Counterexample Search l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); Error Unreachable assume(l != NULL) n = l->next; free(l->data); No Error Error Unreachable assume(l != NULL) n = l->next; free(l->data); NULL l l = new ListNode(new Obj(), l);

Counterexample Search  Produces concrete counterexamples  Contribution: Only explores failed proof Finds counterexamples that would cause this particular proof failure Finds counterexamples that would cause this particular proof failure  Contribution: Relies on SMT solver for unrolling Property-guided, intelligent backtracking Property-guided, intelligent backtracking  Bit-precise memory model

Contribution: Diagnosing Failure Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

Diagnosing the Failure l = new ListNode(new Obj(), NULL); assume(l != NULL) n = l->next; free(l->data); Was the abstraction here responsible for failure? l = new ListNode(new Obj(), l);

Diagnosing the Failure assume(l != NULL) n = l->next; free(l->data); Delete Program Before Join Point l = new ListNode(new Obj(), l);

Diagnosing the Failure l = NULL assume(l != NULL) n = l->next; free(l->data); l = new ListNode(*, l); Synthesize Program Prefix that Creates Abstract State Precisely Error Found! l = new ListNode(new Obj(), l); Re-run Counterexample Search Non-deterministic data field

Diagnosing the Failure

for p in Join_Points(ATS) { ATS’ = Synthesize_Prefix(p, ATS) ATS’ = Synthesize_Prefix(p, ATS) CEx = Find_Counterexample(ATS’) CEx = Find_Counterexample(ATS’) if(exists CEx) { if(exists CEx) { ATS = Refine(ATS, p, CEx); ATS = Refine(ATS, p, CEx); }}

Picking New Abstraction Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

Picking New Abstraction  Partial order of abstractions  Pick next best abstraction

Proof Search with SLL_OBJ l = new ListNode(new Obj(), NULL); l = new ListNode(new Obj(), l); assume(l != NULL) n = l->next; free(l->data); free(l); l = n; assume(l == NULL)

Conclusions Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure

Conclusions Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure ✔ New BMC Approach Search abstract transition system instead of program Only finds causes for proof failure Use monolithic encoding Take advantage of intelligent backtracking

Conclusions Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure ✔ ✔ New Approach to Diagnosis Synthesize program prefix Use guided counterexample search to diagnose Find failing abstraction Find failing concrete value contained by abstraction

Conclusions Start Verification Pick Abstraction Attempt Proof Pick New Abstraction Success Fix Bug Success Find Counterexample Failure Diagnose Failure Failure ✔ ✔ -

Questions?