Deductive Techniques for synthesis from Inductive Specifications Dagstuhl Seminar Oct 2015 Sumit Gulwani

Vu Le Collaborators Dan Barowy Ted Hart Alex Polozov Dileep Kini Rishabh Singh Mikael Mayer Gustavo Soares Ben Zorn Bill Harris

2 Reference “Programming by Examples (and its applications in Data Wrangling)”, Gulwani; 2016; In Verification and Synthesis of Correct and Secure Systems; IOS Press [based on Marktoberdorf Summer School 2015 Lecture Notes]

Deductive Synthesis Refers to synthesis using deductive methods. Has traditionally been applied to synthesis in the presence of logical specifications. Inductive Synthesis Refers to synthesis from inductive (example-based) specifications. Various kinds of techniques have been applied including constraint solving, stochastic, and enumerative search. This talk describes techniques for synthesis from inductive specifications using deductive methods! 3 Deductive Synthesis vs Inductive Synthesis

4 PBE Architecture Example-based specification Search Algorithm Challenge 1: Ambiguous/under-specified intent may result in unintended programs. Challenge 2: Designing efficient search strategy. Ranking Function Program Ordered set of Programs

Key Ideas Restrict search to an appropriately designed domain- specific language (DSL) specified as a grammar. –Expressive enough to cover wide range of tasks –Restricted enough to enable efficient search 5 Challenge 2: Efficient search strategy “Spreadsheet Data Manipulation using Examples” [CACM 2012 Research Highlights] Gulwani, Harris, Singh

6 FlashFill DSL Concatenate(A, C) SubStr(X, P, P) “Automating string processing in spreadsheets using input-output examples”; POPL 2011; Gulwani

7 FlashExtract DSL substr expr S[z] := [z] [z]) “FlashExtract: A Framework for data extraction by examples”; PLDI 2014; Vu Le, Sumit Gulwani

8 Challenge 2: Efficient search strategy “Spreadsheet Data Manipulation using Examples” [CACM 2012 Research Highlights] Gulwani, Harris, Singh

9 Problem Reduction Spec for T Spec for L Spec for S FlashExtract DSL

substring expr E := SubStr(y, P 1, P 2 ) position expr P := K | Pos(y, R 1, R 2, K) 10 Problem Reduction Redmond, WA Spec for P 1 Spec for P 2 Spec for E SubStr grammar

11 Programming by Examples Example-based specification Program Search Algorithm Challenge 1: Ambiguous/under-specified intent may result in unintended programs. Challenge 2: Designing efficient search strategy. Challenge 3: Lowering the barrier to design & development.

Developing a domain-specific robust search method is costly: Requires domain-specific algorithmic insights. Robust implementation requires good engineering. DSL extensions/modifications are not easy. Key Ideas: PBE algorithms employ a divide and conquer strategy, where synthesis problem for an expression F(e1,e2) is reduced to synthesis problems for sub-expressions e1 and e2. –The divide-and-conquer strategy can be refactored out. Reduction depends on the logical properties of operator F. –Operator properties can be captured in a modular manner for reuse inside other DSLs. 12 Challenge 3: Lowering the barrier “ FlashMeta: A Framework for Inductive Program Synthesis ” [OOPSLA 2015] Polozov, Gulwani

13 Programming by Examples Example-based specification Program Search Algorithm Challenge 1: Ambiguous/under-specified intent may result in unintended programs. Challenge 2: Designing efficient search strategy. Challenge 3: Lowering the barrier to design & development. DSL

14 Search Strategy

15 Output properties Elements belonging to the output list Elements not belonging to the output list Contiguous subsequence of the output list Prefix of the output list Task

16 Output properties Task Prefix of the output table (seq of records) We do not require explicit (magenta) record boundaries in which case the spec is: Prefixes of projections of the output table

17 Search Strategy

18 Search Strategy Key concepts in problem reduction: VSAs & Witness functions

19 Version Space Algebra (VSA)

20 VSA Operations

21 Problem Reduction Rules

22 Intersect Operation

23 Problem Reduction Rules

24 Cluster Operation

25 Filter Operation

26 Problem Reduction Rules

27 Problem Reduction Rules

28 Problem Reduction Rules

29 Problem Reduction Rules

Provides efficient implementations of VSA operations Provides a library of witness functions Role of synthesis designer Can add new operators and witness functions. Can provide ranking strategies. Can specify tactics to resolve non-determinism in search –Which witness function to use? –How to order search branches? 30 FlashMeta Framework

Project FlashFill FlashExtractText FlashRelate FlashNormalize FlashExtractWeb 31 Comparison of FlashMeta with hand-tuned implementations OriginalFlashMeta N/A2.5 OriginalFlashMeta N/A1.5 Lines of Code (K) Development time (months) Running time of FlashMeta implementations vary between x of the corresponding original implementation. Faster because of some free optimizations Slower because of larger feature sets & a generalized framework