1. The Deconstruction of Dyninst: Experiences and Future Directions Drew Bernat, Madhavi Krishnan, Bill Williams, Bart Miller Paradyn Project 1

2. Why components? Share tools Build new tools quickly 2

3. Share Tools 33

4. Dataflow API Dyninst Components 4 Patch API Instruction API Parse API Stackwalker API ProcControl API CodeGen API Symtab API DynC API DyninstAPI

5. StackwalkerAPI A Dyninst Component Dyninst Component Users 5 SymtabAPI A Dyninst Component

6. Build New Tools Quickly: Dataflow Analysis 6 PowerPC jump tables and return instruction detection Malware return address tampering Behavior-preserving relocation

7. Build New Tools Quickly: Binary Rewriter 7 SymtabAPI A Dyninst Component

8. Build New Tools Quickly: Unstrip 8 targ8056f50 targ805c3bd0 targ805ee40 targ8057220 ParseAPI A Dyninst Component SymtabAPI A Dyninst Component getpidkill Symbol Table

9. July 2007 Down The Memory Lane SymtabAPI – version 1.0 DynStackwalker – coming soon InstructionAPI – proposed BinInst – proposed 9

10. PatchAPI A Dyninst Component ParseAPI A Dyninst Component DataflowAPI A Dyninst Component DynC API A Dyninst Component SymtabAPI A Dyninst Component StackwalkerAPI A Dyninst Component InstructionAPI A Dyninst Component ProcControlAPI A Dyninst Component Dyninst Components Timeline 10 200620072008200920102011 Design and Implementation Beta Release First Release Integration into Dyninst SymtabAPI StackwalkerAPI InstructionAPI ParseAPI PatchAPI ProcControlAPI DataflowAPI DynCAPI

11. Componentization: Design Decisions Define the scope of the component 11 Block Edge Function Cached register liveness info Instrumentability InstPoints Dyninst CFG model ParseAPI CFG model

12. Componentization: Design Decisions Balance internal and external user requirement 12 StackwalkerAPI A Dyninst Component

13. Componentization: Design Decisions Refine requirements 13 PatchAPI A Dyninst Component

14. Componentization: Design Decisions Create right level of abstractions 14 SymtabAPI A Dyninst Component

15. Componentization: Design Decisions Design extensible and adaptable interfaces 15 StackwalkerAPI A Dyninst Component PatchAPI A Dyninst Component Stack frame stepper Standard frame Debug frame Signal frame ParseAPI A Dyninst Component

16. Componentization: Design Decisions Plan for reintegration 16 StackwalkerAPI A Dyninst Component ProcControlAPI A Dyninst Component

17. Ongoing Research 17

18. Ongoing Research Lightweight, Self-Propelled Instrumentation Wenbin Fang Binary Editing Andrew Bernat Malware Analysis and Instrumentation Kevin Roundy Binary Provenance and Authorship Nate Rosenblum Instrumenting Virtualized Environments Emily Jacobson 18

19. Lightweight Instrumentation Analyze intermittent bugs and fine-grained performance problems Autonomy Little perturbation High level of detail Rapid activation Ability to analyze black-box systems User level and kernel level 19

20. User Mutator Self-Propelled Instrumentation 20 Snippet PatchAPI A Dyninst Component

21. void foo() { { bar() } void bar() { baz() } How it Works 21 Instrumenter.so Process Snippet PatchAPI A Dyninst Component ProcControlAPI A Dyninst Component

22. 22 Binary Instrumentation PatchAPI A Dyninst Component ParseAPI A Dyninst Component

23. Binary Editing 23 Insert error checking and handling Predicate switching Dynamic patching Code surgery

24. Malware Analysis and Instrumentation 24 Unpacking Code Overwriting Code Self- Checksumming Address Space Sensitive

25. SR-Dyninst 25 ParseAPI A Dyninst Component PatchAPI A Dyninst Component ProcControlAPI A Dyninst Component DataflowAPI A Dyninst Component Parse Reachable Code Catch Exceptions Dynamic Code Discovery Overcome Sensitivity

26. CFG of Conficker A 26

27. Forensics 27 linux-vdso.so.1 libpthread.so.0 libasound.so.2 libdl.so.2 libstdc++.so.6 libm.so.6 libgcc_s.so.1 libc.so.6 /lib64/ld-linux-x86-64.so.2 librt.so.1 Debugging remote deployments Why Provenance?

28. 010111010110... I C++ Binary Provenance and Authorship

29. Provenance System Overview 29 01110101 1010101 01010111 0101001 01101 TRAINING DATABINARY ANALYSIS TOOL ParseAPI A Dyninst Component LEARNING FRAMEWORK provenance model 01110101 10101010 10111010 11101010 01101101 PROGRAM

30. 30 Language.999 Compiler.998 Optimization.993 LOHI Version.910 175 programs x 2,686 binaries 955k functions Acc. Provenance Evaluation

31. Instrumenting Virtualized Environments 31

32. Status Update 32

33. 33 Dyninst 7.0.1 Major new features: New platforms for binary rewriter x86 and x86_64 - statically linked binaries ppc32 and BlueGene/P - dynamically linked binaries Improvements to parsing speed Reductions in memory usage Deprecated Solaris and IA64 platforms AIX pending due to support difficulties

34. Component Status Update SymtabAPI 7.0.1 Speed and space optimizations InstructionAPI 7.0.1 PowerPC (ppc32, ppc64) platform Full integration with Dyninst ParseAPI 7.0.1 - Platform independent API for parsing binaries Control flow graph representation Interprocedural edges (call and return) Built on InstructionAPI and SymtabAPI Full integration with Dyninst 34

35. Component Status Update StackwalkerAPI 2.1 Significant reduction in memory usage ProcControlAPI 1.0.1 - Platform independent interface for creating, monitoring and controlling processes High level abstraction for process control, breakpoints and callbacks for process events DynC API 1.0.1 - Instrumentation language for specifying snippets C like instrumentation snippets for easy and more legible mutator Handles creation and destruction of snippet-local variables 35

36. Dyninst 8.0 ProcControl API - Windows and BlueGene Stackwalker API - Windows and VxWorks Stackwalker & ProcControl integration into Dyninst PatchAPI and integration into Dyninst SR Dyninst for tamper resistant and obfuscated binaries New platforms for binary rewriter Dynamically linked binaries on ppc64 and Windows Statically linked binaries on ppc32 and BlueGene/P Dataflow API official release 36

37. MRNet 3.0.1 37 Support for loading several filters from the same library Lightweight MRNet back-end support for non- blocking receives CrayXT support for staging files using ALPS tool helper Improved build structure that permits configuration for multiple platforms from a single source distribution Numerous bug fixes and enhancements

38. 38 Software and Manuals Dyninst 7.0.1, MRNet 3.0.1: available now! Downloads: http://www.paradyn.org/html/downloads.html http://www.paradyn.org/html/manuals.html Dyninst 8.0 – 4th quarter, 2011 MRNet 3.0.2 – coming soon!

39. New Environments Virtual Machines Whole-system profiling (guest + VMM) using instrumentation VMM-level information to understand how and why an application's performance is affected by the virtualized environment Expand performance profiling in the virtualized environment, where traditional approaches do not work or may not be sufficient Mobile environments – VxWorks, ARM GPUs 39

40. Questions 40

41. Unstrip: Semantic Descriptors We take a semantic approach Record information that is likely to be invariant across multiple versions of the function 41 unstrip : Restoring Function Information to Stripped Binaries : mov %ebx, %edx mov %0x66,%eax mov $0x5,%ebx lea 0x4(%esp),%ecx int $0x80 mov %edx, %ebx cmp %0xffffff83,%eax jae 8048300 ret mov %esi,%esi int $0x80 mov %0x66,%eax mov $0x5,%ebx { }, 5

42. unstrip Identifying Functions in a Stripped Binary 42 unstrip : Restoring Function Information to Stripped Binaries stripped binary unstripped binary Descriptor Database For each wrapper function { 1. Build the semantic descriptor. 2. Search the database for a match (two stages). 3. Add label to symbol table. }

43. Performance: Capturing Fine-grained behavior 43 Introduction to the PatchAPI User Mutator DyninstAPI PatchAPI find point insert snippet delete snippet Process void foo () { } void bar () { } void baz () { } Snippet Process void foo () { bar() } void bar () { baz() } void baz () { } Instrumenter.so PatchAPI Snippet Dyninst (3 rd party instrumentation) Self-propelled instrumentation (1 st party instrumentation)

44. Address Space Snippet CFG Parsing Instrumentation Engine Plugin Interface Public Interface New Component: PatchAPI 44 Introduction to the PatchAPI DyninstInternal PatchAPI

45. Dyninst Analysis tool Dyninst Dyninst is a toolbox for analysts Mutator  Specifies instrumentation  Gets callbacks for runtime events  Builds high-level analysis program binary 7a 77 0e 20 e9 3d e0 09 e8 68 c0 45 be 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 6a d0 56 4b fe 92 57 af 40 0c b6 f2 64 32 f5 07 b6 66 21 Control flow analyzer Instrumenter Data flow analyzer CFG 45 loop, block, function, instruction instrument- ation function replace- ment call stack walking forward & backward slices loop analysis process control library injection symbol table reading, writing binary rewriting machine language parsing

46. What we could do because of components? SymtabAPI & StackwalkerAPI DyninstAPI Instrumentor ROSE semantics engine Tools we developed - quickly Binary rewriter, unstrip 46

47. Componentization Trade-offs Internal requirements vs. external requirements Early feedback vs. interface stability Development time vs. scope Structured vs. organic Lesson learned Keep the project details where they belong Change code incrementally Test new interfaces 47

48. Binary rewriter Read binary file format from disk Parse binary code and build CFG Generate code for instrumentation Patch code Emit new binary file 48 SymtabAPIPatchAPIDyninstAPIParseAPI

49. Binary rewriter 49 SymtabAPI A Dyninst Component ParseAPI A Dyninst Component PatchAPI A Dyninst Component StackwalkerAPI A Dyninst Component ProcControlAPI A Dyninst Component DataflowAPI A Dyninst Component

50. Componentization: Design decisions Define the scope of the component Balance internal and external user requirement Refine the assumptions Create right level of abstractions Build from scratch or improve existing code Early feedback vs. interface stability 50 Dyninst Paradyn SymtabAPI A Dyninst Component ProcControlAPI A Dyninst Component InstructionAPI A Dyninst Component StackwalkerAPI A Dyninst Component PatchAPI A Dyninst Component

51. DyninstAPI Patch API Dyninst and the components AST Binary Process Symtab API Symtab API Binary DynCAPI Symtab API Instruction API Parse API Dataflow API Stackwalker API ProcControl API CodeGen API Symtab API