Using Checkers for End-User Shape Analysis National Taiwan University – August 11, 2009 Bor-Yuh Evan Chang 張博聿 University of Colorado, Boulder If some of the symbols are garbled, try either installing TexPoint ( or the TeX fonts ( Collaborators: Xavier Rival (INRIA and ENS Paris), George C. Necula (UC Berkeley)

2 Why think about the analyzer’s end-user? Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis UserTool Accessibility end-users are not experts in verification and logic want adoption of our tools and techniquesAccessibility end-users are not experts in verification and logic want adoption of our tools and techniques Expressivity, Efficiency, and Feasibility end-users are not completely incompetent either can provide guidance to tools, understand the code best Expressivity, Efficiency, and Feasibility end-users are not completely incompetent either can provide guidance to tools, understand the code best

3 Splitting Splitting of summaries (materialization) To reflect updates precisely summarizing And summarizing for termination (summarization) Shape analysis is an abstract interpretation on abstract memory descriptions with … cur l “sorted dl list” l cur l l l l Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis Main Design Decision: Summaries and their operations Main Design Decision: Summaries and their operations

4 Our Approach: Executable Specifications validation code Utilize “run-time validation code” as specification for static analysis. assert(l.purple_dll(null)); for each node cur in list l { make cur red; } assert(l.red_dll(null)); ll cur l Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis h.dll(p) := if (h = null) then true else h ! prev = p and h ! next.dll(h) checker Automatically generalize checkers for intermediate states (generalized segment) p specifies where prev should point h.dll(p) := h = null Æ emp Ç 9 n.  p ¤  n ¤ n.dll(h) Build the abstraction for analysis directly out of the developer- supplied validation code

5 Problem: Checkers are incomplete specs Xisa shape analyzer abstract interpretation splitting and interpreting update summarizing Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis h.dll(p) = if (h = null) then true else h ! prev = prev and h ! next.dll(h) checkers program analysischecker analysis (“pre-program analysis”) Derives information about checkers to use them effectively How do we decide where to unfold? 1 1 How do we decide where to fold? 2 2 What about different checkers for the same structure? 3 3 Defining a program analysis: 1.The abstraction (e.g., separation logic formulas with inductive definitions) and operations on the abstraction (e.g., unfolding, update) 2.How to effectively apply the operations (harder!) Defining a program analysis: 1.The abstraction (e.g., separation logic formulas with inductive definitions) and operations on the abstraction (e.g., unfolding, update) 2.How to effectively apply the operations (harder!)

6 Outline Memory abstraction Guide unfolding (materialization) with level-type analysis on checker definitions Guide folding (summarization) with iteration history –a binary, non-symmetric widening operator Prove lemmas amongst checkers with our parametric shape domain –for a reduction operator Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis

7 memory cell (points-to: °! next = ± ) Abstract memory as graphs h.dll(p) = if (h = null) then true else h ! prev = p and h ! next.dll(h) Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis l ® dll(null)dll( ¯ ) cur ° dll( ° ) ¯ prev next ± Make endpoints and segments explicit l dll( ±, ° ) ± “dll segment” cur ° ® segment summary checker summary (inductive pred) memory address (value) Some number of memory cells (thin edges) ¯  ¯ ¤  ± ¤ ¤ ±.dll( ° ) ¤= ¤ ( ®.dll(null) ¤= °.dll( ¯ )) ¤ Segment generalization of a checker (Intuitively, ®.dll(null) up to °.dll( ¯ ).) Segment generalization of a checker (Intuitively, ®.dll(null) up to °.dll( ¯ ).)

8 Segments as Partial Checker “Runs” (conceptually) ®.dll(null) ¯.dll( ® ) °.dll( ¯ ) ±.dll( ° ) null.dll( ± ) Complete Checker “Run” Instance Summary c0(¯,°0)c0(¯,°0) c( ®, ° ) …… ……… ®¯ c( ° )c0(°0)c0(°0) i i i i = 0 ii 00 c = c 0 ® = ¯ ° = ° 0 ® = ° ¯ = null null next ° ± prev null Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis [POPL’08]

9 Outline Memory abstraction Guide unfolding (materialization) with level-type analysis on checker definitions Guide folding (summarization) with iteration history –a binary, non-symmetric widening operator Prove lemmas amongst checkers with our parametric shape domain –for a reduction operator Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis

10 Types for deciding where to unfold ® dll(null) dll( ¯ ) ° dll( ®,null) dll( ¯, ® ) dll( °, ¯ ) dll( ±, ° ) dll(null, ± ) Checker “Run” Checker “Run” (call tree/derivation) Instance Summary h.dll(p) = if (h = null) then true else h ! prev = p and h ! next.dll(h) If it exists, where is: °! next ? ¯! next ? If it exists, where is: °! next ? ¯! next ? Checker Definition Says Says: from For h ! next/h ! prev, unfold from h before For p ! next/p ! prev, unfold before h Says Says: from For h ! next/h ! prev, unfold from h before For p ! next/p ! prev, unfold before h Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis

11 Types make the analysis robust with respect to how checkers are written ¯ dll( ® )dll( ¯ ) ° Instance Summary h.dll(p) = if (h = null) then true else h ! prev = p and h ! next.dll(h) Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis Instance ¯ dll 0 ° Summary h.dll 0 () = if (h ! next = null) then true else h ! next ! prev = h and h ! next.dll 0 () Alternative doubly-linked list checker Doubly-linked list checker (as before) Different types for different unfolding

12 Summary of checker parameter types wherewhich Tell where to unfold for which fields robust Make analysis robust with respect to how checkers are written Learn where in summaries unfolding won’t help Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis inferred automatically Can be inferred automatically with a fixed- point computation on the checker definitions

13 Outline Memory abstraction Guide unfolding (materialization) with level-type analysis on checker definitions Guide folding (summarization) with iteration history –a binary, non-symmetric widening operator Prove lemmas amongst checkers with our parametric shape domain –for a reduction operator Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis

14 Summarize by folding into inductive predicates last = l; cur = l ! next; while (cur != null) { // … cur, last … if (…) last = cur; cur = cur ! next; } list l, last next cur list l next curlast list l next curlast summarize list last list next cur list l Challenge: Precision (e.g., last, cur separated by at least one step) Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis

15 Use iteration history to guide folding Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis list l next curlast summarize list last list next cur list l Previous approaches guess where to fold for each graph i.e., which nodes to drop e.g., not pointed by variables Previous approaches guess where to fold for each graph i.e., which nodes to drop e.g., not pointed by variables list l, last next cur list l next curlast Contribution: Determine where by comparing graphs across history discover which nodes to drop and edges to fold simultaneously Contribution: Determine where by comparing graphs across history discover which nodes to drop and edges to fold simultaneously

16 Outline Memory abstraction Guide unfolding (materialization) with level-type analysis on checker definitions Guide folding (summarization) with iteration history –a binary, non-symmetric widening operator Prove lemmas amongst checkers with our parametric shape domain –for a reduction operator Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis

17 Problem: Non-Unique Representations With user-guided abstraction, different summaries may have the same (or related) concretizations. Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis l.dll(p) := if (l = null) then true else l ! prev = p and l ! next.dll(l) l.dll_back(n) := if (l = null) then true else l ! next = n and l ! prev.dll_back(l) dll(null) h ht h dll_back(null) t checker summary concrete instance

18 Need: Convert between related summaries 1.Prove lemmas about related checkers –e.g., “dll, dll_back” Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis Observation Observation: Our widening operator can derive these facts on an appropriate program Basic Idea Basic Idea : l.dll(p) := … semantics of dll_back parametric abstract domain summarization (widening) S

19 Need: Convert between related summaries 2.Find out which lemmas are needed and when to apply them during program analysis –work-in-progress –not in this talk Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis

20 New “Pre-Program Analysis Analysis” Xisa shape analyzer abstract interpretation splitting and interpreting update summarizing level-type inference for unfolding Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis h.dll(p) = if (h = null) then true else h ! prev = prev and h ! next.dll(h) checkers program analysischecker analysis (“pre-program analysis”) lemma proving for reduction SS

21 Example: User-Defined List Segments Want Want a decision procedure for these inclusions: Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis l.ls(e) := if (l = e) then true else l ! next.ls(l) l.list() := if (l = null) then true else l ! next.list() checker summary “a list segment”“a segment of a list” ® list() ¯ le Can reuse our parametric abstract domain! ls( ¯ ) ® l ¯ e v ? ® l ¯ e ® list() ¯ le

22 An Alternative Semantics for Checkers Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis °° set of concrete stores summary ls( ¯ ) ® l ¯ e … le addrof( ® )addrof( ¯ ) generator of “concrete” graphs ® l ¯ e ® = ¯ ® l next ®0®0 ¯ e ® 0 = ¯ ¯ e ® 00 = ¯ ® l next ®0®0 ® 00 …

23 Show Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis ® l ¯ e ® = ¯ ® l next ®0®0 ¯ e ® 0 = ¯ ¯ e ® 00 = ¯ ® l next ®0®0 ® 00 … Apply abstract interpretation using only list as a checker parameter to the domain v ls( ¯ ) ® l ¯ e ® list() ¯ le ® l ¯ e X ® l ¯ e Our widening is a non-symmetric binary operator interleaves region matching and summarizing Our widening is a non-symmetric binary operator interleaves region matching and summarizing Widening Properties Soundness: computes an over-approximation Termination: ensures chain stabilizes Algorithm 1.Iteratively split regions by matching nodes (ok by ¤ ) 2.Find common abstraction for matched regions (calling on v to check inclusion) [SAS’07]Widening Properties Soundness: computes an over-approximation Termination: ensures chain stabilizes Algorithm 1.Iteratively split regions by matching nodes (ok by ¤ ) 2.Find common abstraction for matched regions (calling on v to check inclusion) [SAS’07]

24 Inclusion Check Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis ® l next ®0®0 ¯ e ® 0 = ¯ ® l ¯ e list() v ¯ e ® l next ®0®0 ® 0 = ¯ ¯ e ® l next ®0®0 ® l ®0®0 ® l ®0®0 Inclusion Check Algorithm 1.Iteratively split regions by matching nodes 2.Check inclusion by unfolding and matching edges until obvious (emp v emp) Inclusion Check Algorithm 1.Iteratively split regions by matching nodes 2.Check inclusion by unfolding and matching edges until obvious (emp v emp)

25 Summary: Reuse domain to decide relations amongst checker definitions Xisa shape analyzer abstract interpretation splitting and interpreting update summarizing level-type inference for unfolding Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis dll(h, p) = if (h = null) then true else h ! prev = prev and dll(h ! next, h) checkers program analysischecker analysis (“pre-program analysis”) lemma proving for reduction SS

26 Reduction: Next steps Non-unique representation problem magnified with user-supplied checkers –Need reduction to convert between representations –Ordering on checkers needed to apply reduction Ordering shown by applying Xisa to a checker def To put into practice –Needed lemmas: pre-compute ordering or on-demand? –When to apply: level types for unfolding may help –Derive new checkers (e.g., dll_back from dll)? Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis

27 Summary: Using checkers as specs Constructing the end-user program analysis Generalized segment Intermediate states: Generalized segment predicates types with levels Splitting: Checker parameter types with levels History-guided Summarizing: History-guided approach Reduction: Prove lemmas by reusing our domain on checkers next list ®¯ c( ° )c0(°0)c0(°0) Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis l.dll(p) := … semantics of dll_back S

28 Conclusion Checkers are useful specifications Developer View:Global, Expressed in a familiar style Analysis View:Capture developer intent, Not arbitrary inductive definitions Yet they are incomplete for program analysis –With an executable interpretation, can apply program analysis to checker definitions –Such “pre-analysis analysis” guides the code analysis Bor-Yuh Evan Chang 張博聿, University of Colorado - Using Checkers in End-User Shape Analysis