1. Synthesizing Framework Models for Symbolic Execution Xiaokang Qiu Massachusetts Institute of Technology Joint work with Jinseong Jeon, Jonathan Fetter-Degges, Jeff Foster (University of Maryland, College Park) and Armando Solar-Lezama (MIT)

2. Event-Driven Framework 2 Event-Driven Framework GPS Screen Application Event B API Calls Event A [Environment] Timer GPS User Battery Phone …

3. Symbolic Execution? Apps make lots of calls into framework Framework manages critical control flow Framework is much bigger than apps Android platform: over 12 MLoC 3 Event-Driven Framework GPS Screen Application Event B API Calls Event A [Environment] Timer GPS User Battery Phone …

4. Android Symbolic Execution? 4 Application public class MainActivity extends Activity implements onClickListener { Button button; String msg; @Override public void onCreate(Bundle savedInstanceState) { …… button = (Button) findViewById(R.id.button); button.setOnClickListener(this); msg = “Hellow world!”; } @Override public boolean onClick() { assert msg != null; } Activity Manager Context ContextImpl Instrumentation ActivityManagerNative ActivityManagerService ActivityThread ActivityStack 1 2 3 45 6 7 8 9 InputEventReceiver ViewRootImpl Handler View 10 11 12 13

5. Symbolic Execution? Solution: Use framework model (manually) Simplify and abstract actual framework Need not be precise, e.g., may not need to know exact layout of GUI items ) 5 Application Event B API Calls Event A Event-Driven Framework GPS Screen [Environment] Timer GPS User Battery Phone … Simplified Model

6. Synthesis of Framework Models Easy to revise When analysis changed/fixed When bugs/mistakes in model spec are found Easy to update as frameworks evolve Easy to apply to new frameworks 6 Idea: Use program synthesis technology

7. Pasket: Pattern Sketcher First step toward automatically synthesizing event- driven framework models Main target: control flow (with some data flow) Generated from example usage and reproduce same callbacks Can replace manually-written models for symbolic execution 7 95 classes 297 methods 2677 LoC 50 classes 169 methods 1419 LoC

8. 8 Motivation/Overview Key ideas behind Pasket Implementation/Evaluation

9. Pasket Architecture 9 Pasket Framework Framework Model

10. What is the Input to Pasket? 10 Application Event B API Calls Event A Simplified Model Same API Framework’s API Can be extracted from the JAR or AAR files of the framework Provides the skeleton of the expected model

11. API info extracted from Swing 11 class AbstractButton implements Jcomponent {private List olist;private String s; public AbstractButton() ; olist = new LinkedList (); void addActionListener(ActionListener l) ; olist.add(l); String getActionCommand() ; return fld1; void setActionCommand(String ac) ; fld1 = ac; }

12. What is the Input to Pasket? Framework’s API Can be extracted from the JAR or AAR files of the framework Provides the skeleton of the expected model Logging Tutorial Apps Tutorials available from official SDK, web, courses, etc. Manually run under typical scenarios to obtain logs Serves as examples of behaviors at API boundary 12 Application Event B API Calls Event A Simplified Model Same API

13. Logging Tutorial Apps 13 ButtonActivity Android ButtonActivity.onCreate(Button.ButtonActivity@11776102, null) ButtonActivity.setContentView(Button.ButtonActivity@11776102, 1) …… android.widget.Button. (android.widget.Button@262241329, ButtonActivity@11776102) android.widget.Button.setId(android.widget.Button@262241329, 2) android.widget.Button.setText(android.widget.Button@262241329, "Press Me!") …… android.app.Activity.findViewById(ButtonActivity@11776102, 2) …… ButtonActivity.onStart(Button.ButtonActivity@11776102) ButtonActivity.onResume(ButtonActivity@11776102) android.view.MotionEvent.MotionEvent(2, 1) ButtonActivity.onClick(ButtonActivity@11776102, android.widget.Button@262241329) …… ( Log Conformity as Behavioral Constraints )

14. Pasket Architecture 14 Pasket Framework Encoder Logs Logger Design Patterns Synthesis Problem Synthesizer Framework Model API Tutorial Synthesis-by-Example: Vague specificatioin Easily overfit solution

15. Design Patterns General solutions to common problems in software design A lot of frameworks are built using design patterns Observer pattern, e.g., button click  ButtonActivity.onClick() Accessor pattern, e.g., CompoundButton check/uncheck Adapter pattern Singleton pattern … Serve as structural hypotheses of possible models More likely to generalize and behave correctly with other applications Challenge: design patterns are vague concepts Provide precise definition of roles/behavior Leave unclear details unknown 15

16. Observer Pattern 16 An unknown class from API Triggered by event dispatcher, not part of the API Triggered by event dispatcher, not part of the API An (optional) unknown method from Subject An (optional) unknown method from Subject Unknown mapping from event types to handle methods Unknown mapping from event types to handle methods Implementation scaffold

17. Pasket Architecture 17 Pasket Framework Encoder Logs Logger Design Patterns Synthesis Problem Synthesizer Framework Model API Tutorial

18. Encoder: from Java to Sketch Extended from our tool JSketch [ESEC/FSE 2015 Tool Demo] Assign every method/class a unique type id, describe search space Add all possible design pattern code + log conformity assertions 18 /* setOnClickListener is method #19 */ Void setOnClickListener(V_Object self, V_Object l) { int[] params = { 19, self.obj_id, l.obj_id }; check_log(params); /* check next method called in the log*/ if (attach == 19) { /* code for attach */ } else if (detach == 19) { /* code for detach */ } else… int[] ret = { 19 }; check_log(ret); /* check next method return in the log*/ } int Subject = [[ 1 | 2 | … ]]; int Observer = [[ 1 | 2 | … ]]; int attach = [[ 18 | 19 | … ]]; int detatch = [[ 18 | 19 | … ]];

19. 19 Pasket API Tutorial Framework Encoder Logs Logger Design Patterns Synthesis Problem Synthesizer Framework Model

20. Backend: Program Synthesis Synthesis engine: Sketch Input: a C-like program template describing search space Uses Counterexample-Guided Inductive Synthesis (CEGIS) Run Sketch and gather solution Output: mapping roles to classes/methods Map solution back to design pattern, generate code Merge together results from multiple tutorials Quicker to run tutorials separately, synthesis pieces, and then join 20 int Subject = 1; /* View */ int Observer = 2; /* ButtonActivity */ int attach = 18; /* setOnClickListener */

21. 21 Motivation/Overview Key ideas behind Pasket Implementation/Evaluation

22. Performance 22 AC: Adaptive Concretization [CAV 2015]

23. Patterns 23

24. Correctness Ran tutorials again with model, verified executes same path Manually inspected model, verified correct Compared with JPF’s own manually crafted models 24

25. Applicability to Other Programs O’Reilly’s Java Swing, 2 nd Edition, using JPF ClickRelease Subject Programs, using SymDroid 25

26. Conclusion Pasket (Pattern Sketcher) Input = Tutorials + APIs + design patterns Key Results Synthesized executable models for Java GUI Swing and Android Very large synthesis problem, but synthesis performs well Yields correct results Applies to programs other than tutorials A first step toward automatic synthesis of executable framework models More design patterns? User-provided design patterns? Stronger behavioral guarantee than log conformity? 26