1. SIF: A Selective Instrumentation Framework for Mobile Apps Shuai Hao, Ding Li, William G.J. Halfond Ramesh Govindan

2. Background 2 App Behaviors are Complex Source: www.zdnet.com, 3/7/2013www.zdnet.com Source: www.zdnet.com, 6/6/2012www.zdnet.com, 2/9/2013

3. MotivationDesignImplementationEvaluation Motivation 3 Many Studies on App Behavior AppInsight [OSDI12] AppInsight [OSDI12] AppInsight [OSDI12] AppInsight [OSDI12]

4. MotivationDesignImplementationEvaluation Motivation 4 Many Studies on App Behavior FreeMarket [NDSS12] FreeMarket [NDSS12] AppInsight [OSDI12] AppInsight [OSDI12] FreeMarket [NDSS12] FreeMarket [NDSS12]

5. MotivationDesignImplementationEvaluation Motivation 5 Many Studies on App Behavior AppInsight [OSDI12] AppInsight [OSDI12] FreeMarket [NDSS12] FreeMarket [NDSS12] All use app instrumentation Dr. Android [SPSM12] Dr. Android [SPSM12] Dr. Android [SPSM12] Dr. Android [SPSM12]

6. Server Client App App Cloud Phone In-App Billing API In-App Billing MotivationDesignImplementationEvaluation Motivation 6 App Instrumentation Process is Complex 1 1 2 2 3 3 4 4 5 5 6 6 Original Instrumented FreeMarket

7. It raises the level of abstraction for performing app instrumentation. Goal 7 Instrumentation Framework for Mobile Apps

8. Approach 8 Instrumentation Framework for Mobile Apps Instrumentation Framework High-level Instrumentation Specification App Instrumented

9. Find critical paths todebug slow apps Study vulnerability of In-AppBilling service Support fine-grained permissions Remove ads from app Traceapps access to both user location and Internet Binary? Instrument what? Modify app functionality? R1: Instrument binary R2: Instrument at different levels of granularity: bytecode, method, path R2: Instrument at different levels of granularity: bytecode, method, path Requirements 9 Survey of Recent Research R3: Permit user-specified instrumentation Task AppInsight FreeMarket Dr. Android AdCleaner Location Auditor bytecode, method,path bytecode path

10. Requirements 10 Survey of Recent Research R1: Instrument binary R2: Instrument at different levels of granularity: bytecode, method, path R3: Permit user-specified instrumentation

11. Challenges 11 Finding the Right Abstractions R2: Instrument at different levels of granularity: bytecode, method, path How to find the high-level abstractions for what to instrument? Gmail Involves ~150,000 path! Involves ~150,000 path! How to succinctly specify code paths of interest?

12. R3: Provide user-specified instrumentation Arbitrary user-level instrumentation can change execution time of instrumented app 1 1 2 2 3 3 4 4 5 5 6 6 Challenges 12 Providing Overhead Feedback CFG 1 1 2 2 3 3 4 4 5 5 6 6 BAD Instr. Spec. BAD Instr. Spec. 1 1 2 2 3 3 4 4 5 5 6 6 GOOD Instr. Spec. GOOD Instr. Spec. How to provide quick offline overhead feedback?

13. Contributions Novel and flexible abstractions for specifying instrumentation points Support for user-specified instrumentation with quick overhead feedback Extensive evaluation that demonstrates expressivity and efficiency 13 SIF: Selective Instrumentation Framework

14. MotivationDesignImplementationEvaluation SIF Workflow 14 Instrumenter SIFScript App Instrumented Overhead Estimator User Overhead Feedback SIF Refine High execution time

15. Task AppInsight FreeMarket Dr. Android AdCleaner Location Auditor Insert before/after code point Replace code point Track paths A B MotivationDesignImplementationEvaluation SIF Abstractions 15 Survey of Recent Research Operations on set of instructions that share common attributes Codepoint Set Abstraction Set of dynamic traces between two user-specified codepoints Set of dynamic traces between two user-specified codepoints Path Set Abstraction

16. User-specified instrumentation MotivationDesignImplementationEvaluation SIFScript 16 AdCleaner Instrumentation code Specify codepoint filter Hook with user defined code Select target codepoints Place instrumentation Before After

17. MotivationDesignImplementationEvaluation SIFScript 17 Location Auditor 1 1 2 2 3 3 4 4 5 5 C 1 1 2 2 3 3 4 4 5 5 B 1 1 3 3 4 4 5 5 6 6 A 2 2 GPS Internet 1 1 2 2 3 3 4 4 5 5 D 1 1 2 2 4 4 1 1 2 2 3 3 3 3 4 4 5 5 2 2 GPS Internet 1 1 2 2 Find API invokes to GPS Find API invokes to INTERNET Place instrumen- tation

18. MotivationDesignImplementationEvaluation Realizing Path Set Abstraction 18 1 1 2 2 3 3 4 4 5 5 C 1 1 2 2 3 3 4 4 5 5 B 1 1 3 3 4 4 5 5 6 6 A 2 2 GPS Internet 1 1 2 2 3 3 4 4 5 5 D Reachability Analysis Find potential set of methods M Invoke instructions Class hierarchy 4 4 5 5 4 4 M = {A, B, C, D}

19. MotivationDesignImplementationEvaluation Realizing Path Set Abstraction 19 Path Profiling Ball-Larus approach Extension to inter- procedural profiler 1 1 3 3 4 4 5 5 6 6 2 2 1 1 3 3 4 4 6 6 2 2 1 1 3 3 5 5 6 6 2 2 Control Flow Graph Path 1 Path 2 Runtime Statistics - Path 1: 3 times - Path 2: 5 times Runtime Statistics - Path 1: 3 times - Path 2: 5 times Unfolding path statistics Record call-site info P 1,P 2,P 2,P 1,P 2,P 2,P 1,P 2 Call Site 2 P 1,CS 1,CS 2,P 2,CS 1,… Call Site 1

20. MotivationDesignImplementationEvaluation Realizing Path Set Abstraction 20 Path Stitching Reproduce inter- procedural paths by simulated call stack analysis 1 1 2 2 3 3 4 4 5 5 C 1 1 2 2 3 3 4 4 5 5 B 1 1 3 3 4 4 5 5 6 6 A 2 2 GPS Internet 1 1 2 2 3 3 4 4 5 5 D 5 5 4 4 1 1 2 2 3 3 4 4 5 5 C 1 1 2 2 3 3 4 4 5 5 B 1 1 3 3 4 4 5 5 6 6 A 2 2 GPS Internet 1 1 2 2 3 3 4 4 5 5 D A: P2, CS2 C: P2, CS2 D: P1 A: P2, CS2 C: P2, CS2 D: P1

21. MotivationDesignImplementationEvaluation Estimating Overhead Feedback 21 CFG 1 1 2 2 3 3 4 4 5 5 6 6 User Code Overhead Instructions inserted by SIF User-specified instrumentation code Introduced Overhead instruction i in 6 6 User Code and Dynamic execution statistics e.g. by workload generator Profiled estimates for instruction execution time

22. MotivationDesignImplementationEvaluation Evaluation 22 Overview Completeness Classicaltasks: e.g. timing profiler Recentresearch: e.g.AppInsight,FreeMarket Conciseness #Source lines of code Efficiency Instrumentationtime Runtime overhead Accuracy of overhead feedback

23. MotivationDesignImplementationEvaluation Evaluation 23 Conciseness of SIF SIF abstractions permit concise specifications

24. MotivationDesignImplementationEvaluation Evaluation 24 Efficiency of SIF: Runtime Overhead SIF introduces <5% runtime overhead 0.8 4.4 0.6 1.3 1.0 0.4

25. MotivationDesignImplementationEvaluation Evaluation 25 Efficiency of SIF: Accuracy of Overhead Feedback SIFs estimation error is within 7%

26. Conclusion 26 Abstractions for specifying instrumentation points Support for user-specified instrumentation with quick overhead feedback Extensive evaluations that demonstrates expressivity and efficiency SIF: A Selective Instrumentation Framework Future Work Port SIF to other platforms Use SIF to study various kinds of app behaviors Extend overhead estimator to support more sophisticated instrumentation