1. Machine-Learning Assisted Binary Code Analysis
N. Rosenblum, X. Zhu, B. Miller
Computer Sciences Department
University of Wisconsin - Madison
K. Hunt
National Security Agency

2. Supporting Static Binary Analysis
Binary Analysis is a Foundational Technique for Many Areas
Example Uses
Why Analyze Binaries?
Malware detection
Vulnerability analysis
Static and Dynamic Instrumentation
Formal verification
Source code unavailable
e.g., malware
Source code is inaccurate
Compiler transforms structure
Provides most accurate representation
Code is found through symbol information and parsing
MUCH HARDER without symbols
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

3. Many Binaries are Stripped
BINARY
Stripped binaries lack symbol & debug information
Headers
EXAMPLES:
Code Segment
(functions?)
Malicious programs
Operating system distributions
Commercial software packages
Legacy codes
Data Segment
Standard Approach: Parse from entry point
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

4. Stripped Binaries Exhibit Gaps
Code Segment
After static parsing, gap regions remain
Indirect (pointer-based) control ambiguity
Deliberate calls/branch obfuscation
Gaps in code segment may not contain code
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

5. Stripped Binaries Exhibit Gaps
Code Segment
Gap contents may vary
.__gmon_start__.libc.so.6.stpcpy.strcpy.__divdi3.printf.stdout.strerror.memmove.getopt_long.re_syntax_options.__ctype_b.getenv.__strtol_internal.getpagesize.re_search_2.memcpy.puts.feof.malloc.optarg.btowc._obstack_newchunk.re_match.__ctype_toupper.__xstat64.abort.strrchr._obstack_begin.calloc.re_set_registers.fprintf.
String data
Dialog Constants
Import names
Other strings
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

6. Stripped Binaries Exhibit Gaps
Code Segment
Gap contents may vary 0x
0x802434b
0x80243ad
0x80403d0
0x80503d0 0x 0x
0x806000b
0x802321a 0x
0x804132a
0x8050ca0
Tables or lists of addresses
Jump tables
Virtual function tables
Data objects
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

7. Stripped Binaries Exhibit Gaps
Code Segment
Gap contents may vary
gap_funcA {
. . . }
Code unreachable through standard static parsing
gap_funcB {
. . .
Function pointers
Virtual methods
Obfuscated calls
gap_funcC {
. . . }
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

8. Stripped Binaries Exhibit Gaps
Code Segment
Gap contents may vary
7a fd a2 b3
d 70 6c 65 f
e 2e 2e
But… all of these just look like bytes
Every byte in gaps may be the start of a function
How can we find code in gaps?
Our approach: Use information in known code to model code in gaps
Previous work (Vigna et al., 2007) augments parsing with simple instruction frequency information
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

9. Problem reduces to finding function entry points
Modeling Binary Code
Problem reduces to finding function entry points
Task: Classifying every byte in a gap as entry point or non-entry point
Two types of features:
Content: Idiom features of function entry points
Based on instruction sequences
Structure: Control flow & conflict features
Capture relationship of candidate function entry points
Requires joint assignment over all function entry point candidates
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

10. Content-based Features
Entry idioms are common patterns at function entry points
Idioms are preceding and succeeding instruction sequences with wildcards
Candidate
For each idiom u, C1
Entry idioms
push ebp
push ebp|mov esp,ebp
push ebp|*|sub esp
push ebp|*|mov esp,ebp
*|mov_esp,ebp
*|sub 0x8,esp
*|mov 0x8(ebp),eax
PRE nop
PRE ret|nop
PRE pop ebp|*|nop
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

11. Call Consistency & Overlap
Call & conflict features relate candidate FEPs over entire gap
Candidates
y1 = 1
y3 = -1
y2 = 1
y4 = 1 C1 C2 C3
Don’t be formal: use example subscripts C4
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

12. Markov Random Field Formalization
Joint assignment of yi = {1,-1} for each FEP xi in binary P
Unary idiom features fu
Weights u trained through logistic regression
Binary features fo (overlap), fc (call consistency)
Weights o, c large, negative
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

13. Experimental Setup
Large set (100’s) of binaries from department Linux servers and Windows workstations
Additional binaries compiled with Intel compiler
Binaries have full symbol information
Model implemented as extensions to Dyninst instrumentation library
Strip binary copies and parse to obtain training set
Select top idiom features by forward feature selection
Perform logistic regression to build idiom model
Evaluate model on test data from gap regions in Step 1.
Unstripped copies of binaries provide reference set
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

14. Idiom Feature Selection & Training
1. Obtain training data from traditional parse
2. Use Condor HTC to drive forward feature selection on idioms
Statically reachable functions …
Corpus is hundreds of stripped binaries
Features:
Feat1
Feat2
Feat3
...
Featk
3. Perform logistic regression on the selected idiom features to obtain model parameters t
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

15. Evaluation Data Sets GNU C Compiler Simple, regular function preamble
MS Visual Studio
High variation in function entry points
Intel C Compiler
Most variation in entry points; highly optimized
Compiler
Programs examined
Total Training Examples (pos+neg)
Total Test Examples (pos+neg)
Actual number of functions in gaps
GCC
625
8,412,711
22,806,449
85,870
MS VS
443
8,020,828
11,231,721
70,620
ICC
112
1,364,598
13,169,487
47,841
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

16. Preliminary Results Comparison of three binary analysis tools:
Original Dyninst
Scans for common entry preamble
IDA Pro Disassembler
Scans for common entry preamble
List of Library Fingerprints (Windows)
Dyninst w/ Model
Model replaces entry preamble heuristic
Compiler
Orig. Dyninst
IDA Pro
Dyninst w/ Model FP FN
GCC
2,833
2,012
14,576
38,074
403
1,860
MS VS
79,320
65,586
9,044
21,491
725
14,143
ICC
3,786
40,195
14,422
26,970
2,337
16,220
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

17. Classifier Comparisons
GCC
MSVS
ICC
Model-based Dyninst extensions outperform vanilla Dyninst and IDA Pro
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

18. Model Component Contributions
ICC Test Set
Structural information improves classifier accuracy
Conflict resolution contributes the most
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

19. So Far We’ve…
Framed stripped binary parsing as a machine learning problem
Combined idiom and structural information to consider gap regions as a whole
Extended Dyninst with classifier of Function Entry Points in gaps
Obtained significant improvement in parsing stripped binaries over existing tools
Shown how the HTC approach makes expensive ML techniques tractable for large scale systems
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

20. Future Work: Extensions
We’d like precision-recall AUC  1. How?
More detailed instruction sequence models (e.g. Hidden Markov Model)
Additional information sources (e.g. pointer tables)
Caveat: this is where IDA Pro often goes wrong
Code provenance
First task: identify source compiler (needed to choose appropriate model)
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

21. Future Work: Targets Malicious code Obfuscated code
Lots of hand-coded assembly
Usually packed (see Kevin Roundy’s talk)
Obfuscated code
Obfuscation/deobfuscation arms race
Signal-based obfuscation is latest salvo
Can not trust control flow (e.g. non-returning calls, branch functions, opaque branches)
Maybe model block-level structural properties?
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

22. Backup Slides Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis

23. Tool Performance Comparison
Classifier maintains high precision with good recall
Model performance highly system-dependent
MS Visual Studio & Intel C Compiler FEPs are highly variable
Rosenblum, Zhu, Miller, Hunt
ML Assisted Binary Code Analysis