1. 1 Lecture 08(a) – Shape Analysis – continued Lecture 08(b) – Typestate Verification Lecture 08(c) – Predicate Abstraction Eran Yahav

2. Previously  Shape Analysis 2

3. Today  Shape Analysis – continued  Concurrent Shape Analysis  Typestate Verification  Predicate Abstraction (optimistically!) 3

4. Shape Analysis Automatically verify properties of programs manipulating dynamically allocated storage Identify all possible shapes (layout) of the heap 4

5. Shape Analysis via 3-valued Logic 1) Abstraction  3-valued logical structure  canonical abstraction 2) Transformers  via logical formulae  soundness by construction  embedding theorem, [SRW02] 5

6. Collecting State Semantics 6  {  st(w)  (S) | S  CSS[w] }  (w,v)  E(G), w  Assignments(G) { } CSS [v] = if v = entry  { S | S  CSS[w] }  (w,v)  E(G), w  Skip(G)  { S | S  CSS[w] and S  cond(w)}  (w,v)  True-Branches(G)  { S | S  CSS[w] and S   cond(w)} (w,v)  False-Branches(G) othrewise empty universe empty interpretation set of states at node v set of states at node w effect of the statement at node w all predecessors of v in the CFG

7. Collecting Semantics  At every program point – a potentially infinite set of two-valued logical structures  Representing (at least) all possible heaps that can arise at the program point  Next step: find a bounded abstract representation 7

8. 3-Valued Logical Structures  A set of individuals (nodes) U  Relation meaning  Interpretation of relation symbols in P p 0 ()  { 0, 1, 1/2 } p 1 (v)  { 0, 1, 1/2 } p 2 (u,v)  { 0, 1, 1/2 }  A join semi-lattice: 0  1 = 1/2 8

9. Property Space  3-struct[P] = the set of 3-valued logical structures over a vocabulary (set of predicates) P  Abstract domain   (3-Struct[P])   is   We will see alternatives later (maybe) 9

10. Canonical Abstraction Top nn u1u2u3 u1u2u3 Top 10

11. Canonical Abstraction (  )  Merge all nodes with the same unary predicate values into a single summary node  Join predicate values  ’(u’ 1,..., u’ k ) =  {  (u 1,..., u k ) | f(u 1 )=u’ 1,..., f(u k )=u’ k }  Converts a state of arbitrary size into a 3-valued abstract state of bounded size   (C) =  {  (c) | c  C } 11

12. Abstract Semantics Top s = Top  n ? r Top Top r Top s s ’ (v) =  v1: Top(v1)  n(v1,v)  s = Top->n  12

13. Semantic Reduction  Improve the precision of the analysis by recovering properties of the program semantics  A Galois connection (C, , , A)  An operation op:A  A is a semantic reduction when   l  L 2 op(l)  l and   (op(l)) =  (l) l CA   op

14. The Focus Operation  Focus: Formula  (  (3-Struct)   (3-Struct))  Generalizes materialization  For every formula   Focus(  )(X) yields structure in which  evaluates to a definite values in all assignments  Only maximal in terms of embedding  Focus(  ) is a semantic reduction  But Focus(  )(X) may be undefined for some X

15. Top r Top Top r Top Partial Concretization Based on Transformer (s=Topn) Abstract Semantics Partial Concretization Canonical Abstraction Abstract Semantics Top s r Top r Top, r s Top s r Top Top r Top s Top s r Top Top r Top s ’ (v) =  v1: Top(v1)  n(v1,v) 15  u: top(u)  n(u, v) Focus (Top  n)

16. Partial Concretization  Locally refine the abstract domain per statement  Soundness is immediate  Employed in other shape analysis algorithms [Distefano et.al., TACAS’06, Evan et.al., SAS’07, POPL’08]  Emplyed in other analysis algorithms [Typestate verification, ISSTA’06] 16

17. The Coercion Principle  Another Semantic Reduction  Can be applied after Focus or after Update or both  Increase precision by exploiting structural properties possessed by all stores (Global invariants)  Structural properties captured by constraints  Apply a constraint solver

18. Apply Constraint Solver Top r Top x Top r Top x x rxrx r x, r y n n n n y n Top r Top  x rxrx r x, r y n n y n

19. Sources of Constraints  Properties of the operational semantics  Domain specific knowledge  Instrumentation predicates  User supplied

20. Example Constraints x(v1)  x(v2)  eq(v1, v2) n(v, v1)  n(v,v2)  eq(v1, v2) n(v1, v)  n(v2,v)   eq(v1, v2)  is(v) n*(v3, v4)  t[n](v1, v2)

21. Abstract Transformers: Summary  Kleene evaluation yields sound solution  Focus is a statement-specific partial concretization  Coerce applies global constraints

22. Abstract Semantics 22  { t_embed(coerce(  st(w)  3(focus F(w) (SS[w] ))))  (w,v)  E(G), w  Assignments(G) { } SS [v] = if v = entry  { S | S  SS[w] }  (w,v)  E(G), w  Skip(G)  { t_embed(S) | S  coerce(  st(w)  3(focus F(w) (SS[w] ))) and S  3 cond(w)}  (w,v)  True-Branches(G)  { t_embed(S) | S  coerce(  st(w)  3(focus F(w) (SS[w] ))) and S  3  cond(w)}  (w,v)  False-Branches(G) othrewise

23. Recap  Abstraction  canonical abstraction  recording derived information  Transformers  partial concretization (focus)  constraint solver (coerce)  sound information extraction 23

24. void push (int v) { Node  x = alloc(sizeof(Node)); x  d = v; x  n = Top; Top = x; } Stack Push Top x x x emp xx x Top  v:  c(v)  v: x(v)  v1,v2: n(v1, v2)  Top(v2) … 24

25. Non-blocking Stack [Treiber 1986] [1] void push(Stack *S, data_type v) { [2] Node *x = alloc(sizeof(Node)); [3] x->d = v; [4] do { [5] Node *t = S->Top; [6] x->n = t; [7] } while (!CAS(&S->Top,t,x)); [8] } [9] data_type pop(Stack *S){ [10] do { [11] Node *t = S->Top; [12] if (t == NULL) [13] return EMPTY; [14] Node *s = t->n; [15] data_type r = t->d; [16] } while (!CAS(&S->Top,t,s)); [17] return r; [18] } #define EMPTY -1 typedef int data_type; typedef struct node t { data_type d; struct node t *n } Node; typedef struct stack t { struct node t *Top; } Stack; 25

26. Concurrent Shape Analysis  a thread is represented as a thread object  add predicates to vocabulary  Recipe 1) abstraction: canonical abstraction 2) transformers: interleaving + as before  Bounded threads  Static thread names  Unbounded threads  thread objects abstracted via canonical abstraction 26

27. U = { u1, u2, u3,…,u7 } isThread = { u1,u2 } at[pc=1] = { } … at[pc=6] = { u3 } at[pc=7]= { u1 } Top = { u5 } … x = { (u1,u4), (u2,u3)} t={ (u1,u5),(u2,u6)} n = { (u5,u6), (u6,u7) } t 1= { u1 } t2 = { u2 } Top n x n x t n pc=7 pc=6 t Concrete State u1 u2 u5 u6 u7 u3 u4 thread object with program counter thread-local variable list field list object 27

28. [1] void push(Stack *S, data_type v) { [2] Node *x = alloc(sizeof(Node)); [3] x->d = v; [4] do { [5] Node *t = S->Top; [6] x->n = t; [7] } while (!CAS(&S->Top,t,x)); [8] } Top n n 5 x 5 x n n x x t 6 5 n n x x t 5 6 n n x x t t 6 7 n n n x x t t 6 8 n n n x t 6 1 n n n x t 7 1 n n  v:  c(v) Exploration Top n n x x t t 6 6 … n n x t 5 1 n n 28

29. Top n n pc=5 Representing an Unbounded Number of Threads [1] void push(Stack *S, data_type v) { [2] Node *x = alloc(sizeof(Node)); [3] x->d = v; [4] do { [5] Node *t = S->Top; [6] x->n = t; [7] } while (!CAS(&S->Top,t,x)); [8] } pc=5 x x x 29

30. Top n n pc=5 Representing an Unbounded Number of Threads [1] void push(Stack *S, data_type v) { [2] Node *x = alloc(sizeof(Node)); [3] x->d = v; [4] do { [5] Node *t = S->Top; [6] x->n = t; [7] } while (!CAS(&S->Top,t,x)); [8] } pc=5 x x x Top n n pc=5 x rbyX hasX pc=5 n 30

31. Top n n Abstract Semantics [1] void push(Stack *S, data_type v) { [2] Node *x = alloc(sizeof(Node)); [3] x->d = v; [4] do { [5] Node *t = S->Top; [6] x->n = t; [7] } while (!CAS(&S->Top,t,x)); [8] } pc=5 x rbyX Top n n pc=5 x rbyX pc=6 x rbyX t hasX 31

32. Example - Mutual Exclusion 1 shared Initial configuration [1] while (true) { [2] lock(shared) [C] // critical actions [3] unlock(shared) [4] }  t 1,t 2 : (t 1  t 2 )   (at[pc=c](t 1 )  at[pc=c](t 2 )) 1 shared C held_by A thread enters the critical section 2 shared C held_by blocked Other threads may be blocked or just beginning execution 1 shared 32

33. Recap [1] void push(Stack *S, data_type v) { [2] Node *x = alloc(sizeof(Node)); [3] x->d = v; [4] do { [5] Node *t = S->Top; [6] x->n = t; [7] } while (!CAS(&S->Top,t,x)); [8] } [9] data_type pop(Stack *S){ [10] do { [11] Node *t = S->Top; [12] if (t == NULL) [13] return EMPTY; [14] Node *s = t->n; [15] data_type r = t->d; [16] } while (!CAS(&S->Top,t,s)); [17] return r; [18] } #define EMPTY -1 typedef int data_type; typedef struct node t { data_type d; struct node t *n } Node; typedef struct stack t { struct node t *Top; } Stack; Dynamic Allocation Destructive Updates Concurrency No null dereferences Structural shape invariantsLinearizability 33