Shape Analysis by Graph Decomposition R. Manevich M. Sagiv Tel Aviv University G. Ramalingam MSR India J. Berdine B. Cook MSR Cambridge

2 Motivation Challenge: precise and efficient shape analyses Prove properties of dynamically allocated linked data structures Observation: often many correlations irrelevant for proving shape properties Our approach: develop a flexible abstraction that takes advantage of this

3 h1t1... h2t2... h1t1h2t2 Example program – 2 lists h1!=null && h1==t1 && h1.n==null && // h2!=null && h2==t2 && h2.n==null // Reach(h1,t1) && // Reach(h2,t2) && // DisjointLists(h1,h2) EnqueueEvents() { L1: while (...) { List temp = new List(getEvent()); if (nondet()) { t1.n = temp; t1 = temp; } else { t2.n = temp; t2 = temp; } } } Correlation between two lists irrelevant for proving loop invariant

4 size>2 size=2size=1 size>2 size=2size=1 Abstract states - full heaps [VMCAI’05] h1 >1 t1 h2t2 1 h2t2 h1t1 >1 h2t2 1 h1t1 >1 h2t2 >1 h1t1 1 h2t2 1 h1t1 1 h2t2 >1 h1t1 1 h2t2 h1t1 >1 h2t2 h1t1 h1t1 h2t2

5 Graph decomposition 1 h2t2 1 h1t1 >1 h2t2 1 h1t1 h1 >1 t1 h2t2 >1 h2t2 >1 h1t1 1 h2t2 >1 h1t1 1 h2t2 h1t1 1 h2t2 h1t1 >1 h2t2 h1t1 h1t1 h2t2

6 Connected component 1 Connected component 2 Graph decomposition 1 h2t2 1 h1t1 Connected components by undirected reachability 1 h2t2 1 h1t1 decompose

7 Abstract states – decomposed heaps h1t1 h1 1 t1 h1 >1 t1 h2t2 h2 1 t2 h2 >1 t2 For k lists: full heap abstraction generates 3 k abstract states decomposed heap abstraction generates 3×k abstract states Coarser abstraction precise enough to prove invariant but generates fewer states

8 Overall view h1t1... h2t2... h1t1 h2t2 h1t1 h2t2 h1t1 h2t2 >1 1 1 h1t1 h2t2 h1t1 h2t2 >1 1 1 Concrete domain: concrete heaps Full heaps domain: shape graphs Decomposed heaps domain: shape subgraphs  FH  FH  GD  GD Shape graphs track ALL correlations Shape subgraphs track SOME correlations

9 Main results New abstraction for shape analysis reduces exponential factors by: Connected component decomposition Abstracting away null-value correlations Sound and sufficiently precise transformers Most precise transformers are FNP-complete Polynomial time efficient transformers Sufficiently precise Implementation and empirical results Sufficiently precise on set of benchmarks, including Windows device driver models State space/time reduced by factor of 33/212

10 Outline Full heap abstraction [VMCAI’05] Reference abstraction Further abstraction by decomposition Connected component decomposition Abstracting away null-value correlations (details in paper) Abstract transformers Concretization by composition Experimental results

11 Full heap abstraction [VMCAI’05] h1t1... h2t2... h1t1 h2t2 h1t1 h2t2 h1t1 h2t2 >1 1 1 h1t1 h2t2 h1t1 h2t2 >1 1 1 Concrete domain: concrete heaps Full heaps domain: shape graphs Decomposed heaps domain: shape subgraphs  FH  FH  GD  GD

12 Full heap abstraction [VMCAI’05] Abstraction for singly-linked lists Basic concepts: Interruptions (bounded number of) Uninterrupted list segments (bounded number of) Abstraction keeps interruptions and abstracts segment lengths to {1,>1} Result is a shape graph x y Concrete heap x y 1 >1 Shape graph β FH  FH by point-wise extension

13 Graph decomposition abstraction h1t1... h2t2... h1t1 h2t2 h1t1 h2t2 h1t1 h2t2 >1 1 1 h1t1 h2t2 h1t1 h2t2 >1 1 1 Concrete domain: concrete heaps Full heaps domain: shape graphs Decomposed heaps domain: shape subgraphs  FH  FH  GD  GD

14 Graph decomposition abstraction Abstraction of shape graphs Further abstraction over shape graphs Decouples connected components Intuitively different components = different logical data structures Result = set of shape subgraphs

15 Connected components decomposition 1 h2t2 h1t1 h1 >1 t1 h2t2  GD h1t1 h2 1 t2 h1 >1 t1 h2t2

16 Abstracting null-value correlations Actual shape graph representation captures null-value correlations (null node not shown in other slides) Abstraction reduces exponential factor due to null-value correlations Details in paper y >1 null x1 x2 xn … Null-value correlations abstraction  GD y >1 null … x1 null xn

17 Concretization  GD h1t1... h2t2... h1t1 h2t2 h1t1 h2t2 h1t1 h2t2 >1 1 1 h1t1 h2t2 h1t1 h2t2 >1 1 1 Concrete domain: concrete heaps Full heaps domain: shape graphs Decomposed heaps domain: shape subgraphs  FH  FH  GD  GD

18 1 h2t2 h1t1 h1 >1 t1 h2t2  GD Abstracting correlations  GD 1 h2t2 h1t1 h1 >1 t1 h2t2 h1t1 h2 t2 h2 1 t2 h1 >1 t1 h1t1 h2 1 t2 h1 >1 t1 h2t2

19 Abstract transformers Need transformers for program statements x=new List() x=null x=y x=y.n x.n=y assume(x!=y) assume(x==y) …

20 Abstract transformers outline Induced transformers by concretization (from subgraphs and shape graphs) Problem: concretization introduces exponential space blow-up Most precise transformers by partial concretization Avoids exponential space blow-up Requires oracle to test strong feasibility Strong feasibility test NP-complete Conservative transformers Give up on strong feasibility test Avoids exponential time blow-up

21 Most precise transformer [CC’77] h1t1... h2t2... h1t1 h2t2 Concrete domain: concrete heaps Full heaps domain: shape graphs Decomposed heaps domain: shape subgraphs  FH  FH  GD  GD  st Problem: concretization is exponential space in worst-case

22 Partial concretization Compose weakly-feasible subgraphs Subgraphs that do not share any variables Compose only subgraphs in footprint of statement Compose at most any 2 or 3 subgraphs h1t1 h2 1 t2 h1 >1 t1 h2 1 t2 h1t1h1t1 h1 >1 t1 h1t1

23 Transformer example temp h1t1 h1 1 t1 h2t2  t1.n = temp temp h1 1 t1  t1.n = temp temp h1 1 t1 1  t1.n = temp h2t2  t1.n = temp h2t2 temp h1 1 t1 temp h1t1

24 Most precise transformer xz wx ywy z Can we extend to have variable w? M1M1 M2M2 M3M3 M4M4 M5M5 xzy Most precise requires strong feasibility test Check that subgraphs can be extended to include all variables

25 Most precise transformer Inconsistency: shared variable x xz wx ywy z M1M1 M2M2 M3M3 M4M4 M5M5 xzy Most precise requires strong feasibility test Check that subgraphs can be extended to include all variables

26 Most precise transformer Inconsistency: shared variable y Conclusion: can’t extend with w M 1 and M 4 are weakly-feasible but not strongly-feasible in {M 1,…,M 5 } Strong feasibility NP-complete Therefore most precise transformer FNP- complete xzy xz wx ywy z M1M1 M2M2 M3M3 M4M4 M5M5

27 Making the transformers efficient Vanilla transformer inefficient in practice Incremental transformers Reuse results of previous iterations Details in paper Engineering optimizations Avoid unnecessarily composing subgraphs … Optimized transformers linear time in practice

28 Prototype implementation Implemented in Java Supports assertions assertReach(x,y) assertDisjointLists(x,y) assertAcyclicList(x) assertCyclicList(x) assert(x==y)assert(x!=y) Check cleanness properties Absence of null derefs Absence of memory leaks No misuse of dangling pointers

29 Experiments – precision Precision lost in just 2/21 benchmarks getLast Unable to prove x points to last cell Due to imprecise transformer Can be avoided by simple and efficient heuristics queue_2_stack Intentionally constructed Loss of correlations important to prove property Same precision as full heap analysis on other benchmarks

30 Experiments – “standard” suite Programs operating on 1-2 lists insert, delete, reverse, merge… New analysis slightly less efficient But running times < 0.6 seconds so…

31 Experiments – multiple lists (89,430 / 7,733) number of shape graphs number of subgraphs x

32 Experiments – multiple lists full shape graph analysis time graph decomposition analysis time x (552.6 / 2.6)

33 Properties of the abstraction No loss of precision when connected components represent completely independent lists Reduces state space exponentially Loss of precision when mixing abstract states  GD (X 1  X 2 )   GD (X 1 )   GD (X 2 ) So where is this technique useful?

34 Related work Partial isomorphism join [Manevich et al. SAS’04] Applied in more generic context but does not reduce exponential blow-ups addressed in this paper Heap analysis by separation [Yahav et al. PLDI’04] [Hackett et al. POPL’05] Decompose verification problem itself and conservatively approximate contexts Heap decomposition for interprocedural analysis [Rinetzky et al. POPL’05] [Rinetzky et al. SAS’05] [Gotsman et al. SAS’06] [Gotsman et al. PLDI’07] Decompose/compose at procedure boundaries Predicate/variable clustering [Clark et al. CAV’00] Statically-determined decomposition

35 Conclusions New abstraction scheme to control precision/cost trade-off for shape analyses Efficient algorithms for abstract domain operations Abstraction Partial concretization Transformers … Applicable beyond singly-linked lists E.g., class of graphs supported by Lev-Ami et al. [CAV’06] Doubly-linked lists Trees …

36 Ongoing work Extension for concurrent program analysis Future work: Tune abstraction by counterexample-guided refinement

37 Questions?

38 Conservative transformer Computes superset of subgraph computed by most precise transformer Algorithm sketch: Compose components in footprint of statement Apply local  st on footprint and decompose result Test consistency instead of strong feasibility Pass other components as is Time(  st ) polynomial in #vars in st x=null : linear x.n=y: quadratic assume(x==y) : cubic

39 Concretization  GD Maps sets of shape subgraphs to sets of full shape graphs Mathematically:  GD (XG) = {G | β(G)  XG} Algorithmically: by composing weakly-feasible subgraphs Subgraphs that do not share any variables Full shape graph includes all program variables