Dagstuhl Seminar "Applied Deductive Verification" November Symbolically Computing Most-Precise Abstract Operations for Shape Analysis Greta Yorsh Joint work with Thomas Reps Mooly Sagiv

2 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Why use theorem prover?  Guarantee the most-precise result w.r.t. the abstraction  Modular reasoning assume guarantee reasoning scalability

3 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Outline  Background  The “assume” Operation  The assume Algorithm canonical abstraction  Main Results  Future Work ^

4 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Shape Analysis  Static program analysis  Determine “shape invariants” Verify programs (partially) Detect memory errors Prove properties about dynamically allocated data Detect logical errors Code optimizations  Abstract Interpretation [CC77] Galois Connection ( ,  )

5 Dagstuhl Seminar "Applied Deductive Verification" November 2003 a Concretization Function  Concrete Domain Abstract Domain (a)(a) 

6 Dagstuhl Seminar "Applied Deductive Verification" November 2003 C Concrete Domain Abstract Domain Abstraction Function  (C)(C) 

7 Dagstuhl Seminar "Applied Deductive Verification" November 2003  (  (C))  C Concrete Domain Abstract Domain Galois Connection ( ,  ) (C)(C) 

8 Dagstuhl Seminar "Applied Deductive Verification" November 2003  (a')   (  (C)) C Concrete Domain Abstract Domain Most Precise Abstract Value (C)(C)  a' 

9 Dagstuhl Seminar "Applied Deductive Verification" November 2003 New Approach  Use symbolic techniques in abstract interpretation For shape analysis For other abstract domains  What does it mean to employ decision procedure/theorem prover for shape analysis? symbolic concretization decision procedure for satisfiability ^  (a)

10 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Concrete DomainAbstract Domain Formulas a2a2 (a1)(a1) a1a1 store ⊧  (a 1 ) ^ store ⊭  (a 1 ) ^ Symbolic Concretization  (a) ^ ^  (a 1 ) (a2)(a2) S  (a) ⇔ S ⊧  (a) ^ ⊧

11 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Outline Background  The “assume” Operation  The assume Algorithm canonical abstraction  Main Results  Future Work ^ ✔

12 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Assume-Guarantee Reasoning T bar(); void foo() { T p;... p = bar();... } {pre bar, post bar } {pre foo, post foo } assume[pre foo ]; assert[pre bar ]; assume[post bar ]; assert[post foo ]; ^ Is  (a) ⇒  valid? assert[  ](a) assume[  ](a) ?

13 Dagstuhl Seminar "Applied Deductive Verification" November 2003 a X Concrete Domain Abstract Domain 〚〛〚〛 The “assume[  ](a)” Operation (a)(a)  Formulas

14 Dagstuhl Seminar "Applied Deductive Verification" November 2003 a 〚〛〚〛 X Concrete Domain Abstract Domain (a)(a) The “assume[  ](a)” Operation assume[  ]( a)  (X) 

15 Dagstuhl Seminar "Applied Deductive Verification" November 2003 a 〚〛〚〛 X Concrete Domain Abstract Domain (a)(a) The “assume[  ](a)” Operation assume[  ]( a)  ^   (X)

16 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Outline Shape Analysis The “assume” Operation  The assume Algorithm canonical abstraction  Main Results  Future Work ^ ✔ ✔

17 Dagstuhl Seminar "Applied Deductive Verification" November 2003 a X Concrete Domain Abstract Domain 〚〛〚〛 The assume[  ](a) Algorithm (a)(a) ^

18 Dagstuhl Seminar "Applied Deductive Verification" November 2003 a 〚〛〚〛 The assume[  ](a) Algorithm X Concrete Domain Abstract Domain (a)(a) ^

19 Dagstuhl Seminar "Applied Deductive Verification" November 2003 a 〚〛〚〛 The assume[  ](a) Algorithm X Concrete Domain Abstract Domain (a)(a) ^

20 Dagstuhl Seminar "Applied Deductive Verification" November 2003 assume[  ]( a) a 〚〛〚〛 The assume[  ](a) Algorithm X Concrete Domain Abstract Domain (a)(a) ^  (X)

21 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Outline Shape Analysis The “assume” Operation  The assume Algorithm canonical abstraction  Main Results  Future Work ^ ✔ ✔

22 Dagstuhl Seminar "Applied Deductive Verification" November 2003 C Concrete Domain Abstract Domain Abstraction Function  (C)(C)     (C) = {  (S) | S  C} 2-valued logical structures sets of 3-valued logical structures

23 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Describing Heap Using Logical Structure  Definition of linked list  Cyclic linked list of length 4 pointed to by variable x structure S = universe U = {u 1, u 2, u 3, u 4 }, unary relation x = {u 1 } binary relation n = {,,, } unary relation r x = {u 1, u 2, u 3, u 4 } unary relation c = {u 1, u 2, u 3, u 4 } struct List { int d; struct List *n; } x u1u1 u2u2 u3u3 u4u4 c,r x nnn

24 Dagstuhl Seminar "Applied Deductive Verification" November Valued Logical Structures  Relation meaning over {0, 1, ½}  Kleene 1: True 0: False ½ : Unknown  A join semi-lattice: 0 ⊔ 1 = ½   ½ Information order

25 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Canonical Abstraction  x u1u1 u2u2 u3u3 u4u4 c,r x x u1u1 u2u2 u 2 summary node x u1u1 u2u2 u3u3 u4u4 c,r x

26 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Canonical Abstraction  x u1u1 u2u2 u3u3 u4u4 c,r x x u1u1 u2u2 :: u 2 summary node  Unary relations have definite values x

27 Dagstuhl Seminar "Applied Deductive Verification" November 2003 a Concretization Function  Concrete Domain Abstract Domain (a)(a)  (a)  a ≜ ∃ v 1,v 2 :node u1 (v 1 ) ⋀ node u2 (v 2 ) ⋀∀ w: node u1 (w) ⋁ node u2 (w) ⋀ ∀ w 1,w 2 :node u1 (w 1 ) ⋀ node u1 (w 2 ) ⇒(w 1 =w 2 )⋀⌝n(w 1,w 2 )  (a) ≜  a ⋀ IR ^ S  (a) ⇔ S ⊧  (a) ^ Formulas ^ x u1u1 u2u2 c,r x

28 Dagstuhl Seminar "Applied Deductive Verification" November 2003 a Concretization Function  Concrete Domain Abstract Domain (a)(a)  (a) IR = unique[x] ⋀ function[n] ⋀ reachable[x] ⋀ cyclic[n] reachable[x] ≜ ∀ v:r x (v) ⇔∃ v 1 : x(v 1 ) ⋀ n*(v 1,v) cyclic[n] ≜ ∀ v:c(v) ⇔∃ v 1 :n(v,v 1 ) ⋀ n*(v 1,v)  (a) ≜  a ⋀ IR ^ S  (a) ⇔ S ⊧  (a) ^ Formulas ^ unique[x] ≜ ∀ v 1,v 2 :x(v 1 ) ⋀ x(v 2 ) ⇒ v 1 =v 2 function[n] ≜ ∀ v,v 1,v 2 :n(v,v 1 ) ⋀ n(v,v 2 ) ⇒ v 1 =v 2

29 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Outline Shape Analysis The “assume” Operation  The assume Algorithm canonical abstraction  Main Results  Future Work ^ ✔ ✔ ✔

30 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Example x u1u1 u2u2 c,r x y==x->n  ≜ ∀v 1 :y( v 1 ) ↔ ∃ v 2 : x(v 2 ) ⋀ n(v 1, v 2 ) y,r y x u1u1 uyuy c,r x r y x u1u1 u2u2 y uyuy y a: assume[  ](a) ^ IR = unique[x] ⋀ unique[y] ⋀ reachable[x] ⋀ reachable[y] ⋀ cyclic[n] ⋀ function[n]

31 Dagstuhl Seminar "Applied Deductive Verification" November 2003 The assume[  ](a) Algorithm assume[  ](a) : set of 3-valued structures // initialization for all S ∈ a if  (S) ⋀  is satisfiable then W  S // phase 1: node materialization while there is S ∈ W with p(u)=1/2 do duplicate nodes and deduce their unary relations using calls to theorem prover // phase 2: relation refinement while there is S ∈ W with p(u1,u2)=1/2 do duplicate structures and deduce their binary relations using calls to theorem prover return W ^ ^ ^

32 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Example - Materialization materialization u 2  u y, u 2 y(u y ) = 1, y(u 2 ) =0 x u1u1 u2u2 c,r x y,r y S x u1u1 u2u2 c,r x y,r y y y(u 2 )=0 S0 ryry S1 y(u 2 )=1 x u1u1 u2u2 c,r x y,r y y ryry u2u2 x u1u1 uyuy c,r x y,r y y rxrx y ryry ryry

33 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Example - Materialization x u1u1 uyuy c,r x y,r y y rxrx y x u1u1 u2u2 c,r x r y y u2u2 x u1u1 u2u2 c,r x y,r y y ryry ryry ryry x u1u1 uyuy c,r x r y y u2u2

34 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Example – Refinement x u1u1 uyuy c,r x r y y u2u2 n(u 2,u y ) x u1u1 uyuy c,r x,r y y u2u2 c,r x r y c,r x,r y S0 x u1u1 uyuy c,r x,r y y u2u2 c,r x r y u y n(u 1,u y ) n(u y,u y ) n(u 1,u 2 ) n(u y,u 1 )

35 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Example x u1u1 u2u2 c,r x y==x->n  ≜ ∀v 1 :y( v 1 ) ↔ ∃ v 2 : x(v 2 ) ⋀ n(v 1, v 2 ) y,r y x u1u1 uyuy c,r x r y x u1u1 u2u2 y uyuy y a: assume[  ](a) ^ IR = unique[x] ⋀ unique[y] ⋀ reachable[x] ⋀ reachable[y] ⋀ cyclic[n] ⋀ function[n]

36 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Algorithm assume[  ](a) : set of 3-valued structures for all S ∈ a if  (S) ⋀  is satisfiable then W  S // phase 1: materialization while there is S ∈ W with p(u)=1/2 do W  W/S if  (S) ⋀  ⋀  p,u is satisfiable then W  S' if  (S0) ⋀  is satisfiable then W  S0 if  (S1) ⋀  is satisfiable then W  S1 // phase 2: relation refinement while there is S ∈ W with p(u1,u2)=1/2 do if  (S) ⋀  ⋀  p,u1,u2 is not satisfiable then W  W/S if  (S0) ⋀  is satisfiable then W  S0 if  (S1) ⋀  is satisfiable then W  S1 return W ^ ^ ^ ^ ^ ^ ^ ^

37 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Theorem Prover  Satisfiability of FO TC  Calls to theorem prover need not terminate  Experience with SPASS  Solutions ?

38 Dagstuhl Seminar "Applied Deductive Verification" November 2003 SPASS Experience  Handles arbitrary FO formulas  Can diverge  Converges in our examples Captures older shape analysis algorithms  How to handle FO TC ? Overapproximations are not good enough Lead to too many structures

39 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Theorem Prover  Satisfiability of FO TC  Calls to theorem prover need not terminate  Experience with SPASS  Solutions timeout and return ½ decidable logic  Bad news Even ∃∀ TC is undecidable Reduction to halting problem

40 Dagstuhl Seminar "Applied Deductive Verification" November 2003 ∃∀ DTC[E] Logic  Neil Immerman, Alexander Rabinovich  ∃∀ DTC[E] is subset of FO TC ∃∀ form arbitrary unary relations single binary relation E deterministic transitive closure E*(v,w) E-path through individuals with at most one successor  Decidable for satisfiability NEXPTIME -complete

41 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Simulation Technique  Simulate regular data structures using ∃∀ DTC[E] Singly linked list shared/cyclic/nested Doubly linked list (Shared) Trees  Preserved under mutations

42 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Outline Shape Analysis The “assume” Operation The assume Algorithm canonical abstraction  Main Results  Future Work ^ ✔ ✔ ✔ ✔

43 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Most-precise Operations  Most-precise abstract value  Best transformer statement loop-free fragment ^  (  ) = assume[  ]( ) ^ BT(a,τ) = assume[τ]( ) ^

44 Dagstuhl Seminar "Applied Deductive Verification" November 2003 (a)(a)  Concrete DomainAbstract Domain Best Transformer BT(a,τ) a τ τ   (C) C BT(a,τ)= τ

45 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Most-precise Operations  Most-precise abstract value  Best transformer statement loop-free fragment  Meet operation  Assume guarantee reasoning procedure specifications ^  (  ) = assume[  ]( ) ^ ^ ^ ^ m(a,a') =  (  (a) ⋀  (a')) ^ BT(a,τ) = assume[τ]( ) ^

46 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Conclusions  Employ decision procedure/theorem prover for shape analysis most precise modular - assume guarantee reasoning

47 Dagstuhl Seminar "Applied Deductive Verification" November 2003 Future Work  Implementation  Assume guarantee of “real” programs specification language write procedure specifications  Extend to other domains

Dagstuhl Seminar "Applied Deductive Verification" November THE END