Automated Tracing and Visualization of Software Security Structure and Properties Symposium on Visualization for Cyber Security 2012 (VizSec’12) Seattle, WA, USA Oct. 15, 2012 Wenbin Fang, Barton P. Miller, and James A. Kupsch Computer Sciences Department University of Wisconsin-Madison

Motivation Visualization: an intrinsic part of in-depth security assessment First Principles Vulnerability Assessment (FPVA) Microsoft Threat Modeling Diagrams as road map for later analysis Key components and interaction The privilege level of each component Access to high-value resources 2

Example Diagrams From FPVA 3

4

5

6

Diagram Creation Problems Manual (time consuming) data collection Collected from many sources Potentially inaccurate Manual diagram construction Deferred until confident in data collection Limits diagrams produced Approach: Automate diagram construction 7

Data Collection Automatically collect trace data during runtime Visualization Construct diagrams/animation from trace data Web-based interface 8 SecSTAR: Security System Tracing, Analysis and Reporting Data Collection Instrumented Binary Code Trace Data Visualization Diagram Display Interface

Data Collection Overview Goal: automate system data collection Unmodified binaries Follows control flows to other processes Easy to extend to trace new security events SecSTAR: Uses self-propelled instrumentation Simple code snippets determine what to trace 9

Self-propelled Instrumentation Instrument unmodified binary code No special preparation Inject code snippet into a target process Instrumentation follows control flow Within a process Across thread boundaries Across process and even host boundaries 10

Self-propelled Instrumentation 11 Application Process Injector: Process to inject shared library Agent: Shared library Injector process a.out libc.so libpthread.so Agent.so Payload Functions Instrumentation Engine

12 void payload(SpPoint* pt){ if IsExit(pt) { trace(“exit” …) } else if IsConnect(pt) { trace(“connect” …) } else if... // detect other events } void main () { pthread_create(foo …) … } void foo () { connect(…) exit(0) } Host A Host B Process P Process Q Agent.so network Process R Injector Call How it works

Detect system events Process creation and destruction Privilege level changes Communication Resource access Query runtime info related to the current call Arguments / Return value Query Control Flow Graph (or CFG) structures Functions / Basic blocks / Edges Enables sophisticated code analysis 13 Payload Function

Visualization Overview Goal: Same-style same-quality diagrams as those constructed by skilled analysts Animate temporal data Interactive interface 14 Data Collection Instrumented Binary Code Trace Data Visualization Diagram Display Interface

Notation 15

Diagram, Animation and SecSTAR Interface Demo 16

Case Study Using SecSTAR to produce FPVA-style diagrams for Condor Condor: high-throughput job scheduling system Used worldwide ~700,000 lines of code pages of documentation Multiple processes, multiple hosts 17

Original FPVA vs SecSTAR Original FPVA diagram construction Manual data collection from Many processes and hosts Documentation and code Correlated and distilled artifacts Manual diagram creation Months SecSTAR Automated data collection Automated diagram construction Hours, mostly to learn how to install and operate Condor 18

Diagram comparison 19 SecSTAR Original FPVA

Future Work Capture and visualize more events Capture and visualize resources Improve the web-based interface Integrating with Microsoft Threat Modeling 20

Summary SecSTAR Automated data collection Automated diagram/animation construction Case study Diagram construction for Condor Original FPVA vs SecSTAR 21

Questions? 22

Backup 1: Intra-process Propagation 23 a.out main 8430: 8431: 8433: 8444: 8449: 844b: 844e: 844f: push %ebp mov %esp,%ebp... call printf mov %ebp,%esp xor %eax,%eax pop %ebp ret foo call jmp Patch1 payload(foo) foo 0x8405 Agent.so call jmp payload(printf) printf 0x8449 Patch2 patch jmp push %ebp mov %esp,%ebp... call foo mov %ebp,%esp pop %ebp ret 83f0: 83f1: 83f3: 8400: 8405: 8413: 8414: Inject ActivatePropagate jmp Patch1 jmp Patch2

Backup 2: Inter-process Propagation 24 Main procedure for inter-process propagation 1.Detect the initiation of communication at the local site. connect, write, send … 2.Identify the remote process 3.Inject the agent into the remote process 4.Start following the flow of control in the remote site void main () { connect(…) recv(…) } void main () { accept(…) send(…) } Agent.so inject call payload() Process A Process B