Automatic Discovery of API-Level Exploits Vinod Ganapathy, Sanjit Seshia, Somesh Jha, Thomas Reps & Randal Bryant

ICSE 2005Automatic Discovery of API-Level Exploits2 Two Definitions Exploit A sequence of operations that attacks the vulnerability Vulnerability An error in a software package that allows for unintended behavior

ICSE 2005Automatic Discovery of API-Level Exploits3 Motivation //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } Format-string vulnerability buf = %s%s%s fprintf(LOG,%s%s%s) Insufficient arguments to fprintf. Possible outcomes Unintelligible log entry. Program crash. Hacker takes over program!

ICSE 2005Automatic Discovery of API-Level Exploits4 Motivation Format-string vulnerabilities: Well-understood class of security vulnerabilities. Commonly used by attackers [CERT] Tools to find format-string vulnerabilities exist Examples: Percent-S [Shankar et al. USENIX Security 2001], Avots et al. [ICSE 2005 (later in this session!!)] What about systematic exploit-finding tools?

ICSE 2005Automatic Discovery of API-Level Exploits5 Vulnerabilities versus Exploits ClassificationVulnerability-finding toolsExploit-generation Buffer- overflows BOON, Archer, CSSV, Rats, Its4, Codesurfer, … Solar-designer, Aleph-one,… Format- strings Percent-S, Avots et al., … Newsham, Thuemmel,… Race- conditions Eraser, RacerX, SVD, Atomicity,… Several online manuals. SQL- injection JDBC-checker, …Several online manuals.

ICSE 2005Automatic Discovery of API-Level Exploits6 Vulnerabilities versus Exploits ClassificationVulnerability-finding toolsExploit-generation Buffer- overflows BOON, Archer, CSSV, Rats, Its4, Codesurfer, … Solar-designer, Aleph-one,… Format- strings Percent-S, Avots et al., … Newsham, Thuemmel,… Race- conditions Atomicity, Eraser, RacerX,…,… Several online manuals. SQL- injection JDBC-checker, …Several online manuals. Huge body of research Based upon systematic techniques

ICSE 2005Automatic Discovery of API-Level Exploits7 Vulnerabilities versus Exploits ClassificationVulnerability-finding toolsExploit-generation Buffer- overflows BOON, Archer, CSSV, Rats, Its4, Codesurfer, … Solar-designer, Aleph-one,… Format- strings Percent-S, Avots et al., … Newsham, Thuemmel,… Race- conditions Atomicity, Eraser, RacerX,…,… Several online manuals. SQL- injection JDBC-checker, …Several online manuals. Ad-hoc (but clever!) techniques. Mostly restricted to the hacker community.

ICSE 2005Automatic Discovery of API-Level Exploits8 Vulnerabilities versus Exploits All programs Programs with vulnerabilities

ICSE 2005Automatic Discovery of API-Level Exploits9 Vulnerabilities versus Exploits All programs Programs with vulnerabilities Existing work finds this set of programs

ICSE 2005Automatic Discovery of API-Level Exploits10 Vulnerabilities versus Exploits All programs Programs with vulnerabilities Programs with expoitable vulnerabilities

ICSE 2005Automatic Discovery of API-Level Exploits11 Vulnerabilities versus Exploits All programs Programs with vulnerabilities Programs with expoitable vulnerabilities This paper: What does it take to identify these programs?

ICSE 2005Automatic Discovery of API-Level Exploits12 Main Message of This Talk Exploit-finding can benefit, and improve the quality of, vulnerability-finding tools

ICSE 2005Automatic Discovery of API-Level Exploits13 Exploit-finding: Possible Benefits Benefits to vulnerability-finding tools: Enhance vulnerability reports with exploits. Helps differentiate between exploitable and benign vulnerabilities. Prioritizes bug-fixes. Test cases for future versions of the software. Derive signatures for signature-based misuse- detection systems (e.g., Snort, Bro)

ICSE 2005Automatic Discovery of API-Level Exploits14 Exploit-finding: Possible Benefits Benefits to vulnerability-finding tools: Enhance vulnerability reports with exploits. Helps differentiate between exploitable and benign vulnerabilities. Prioritizes bug-fixes. Test cases for future versions of the software. Derive signatures for signature-based misuse- detection systems (e.g., Snort, Bro)

ICSE 2005Automatic Discovery of API-Level Exploits15 Exploit-finding: Challenges Key difference: Modeling low-level details of the runtime-environment. Format-string exploits [CERT, Newsham, Thuemmel,…] IBM CCA API exploit [Bond and Anderson, 2001] Machine architecture. Layout of the runtime execution stack. Pointer movements along the runtime stack. Discussed in detail in the rest of this talk. Intruder knowledge. Bit operations on cryptographic keys. Analysis is similar to protocol verification techniques. Paper has details.

ICSE 2005Automatic Discovery of API-Level Exploits16 Exploit-finding: Challenges Key difference: Modeling low-level details of the runtime-environment. Format-string exploitsIBM CCA API exploit [Bond and Anderson, 2001] Machine architecture. Layout of the runtime execution stack. Pointer movements along the runtime stack. Discussed in detail in the rest of this talk. Intruder knowledge. Bit operations on cryptographic keys. Analysis is similar to protocol verification techniques. Paper has details. Analysis of both case studies is done using the same generic framework.

ICSE 2005Automatic Discovery of API-Level Exploits17 Talk structure Motivation and Overview. Framework for finding API-level exploits. Example: format-string exploit-detector. Overview of printf and format-string exploits. Instantiating printf in our framework. Results. Comparison with other tools. Related work and Conclusions.

ICSE 2005Automatic Discovery of API-Level Exploits18 API-Level Exploits Exploit A sequence of operations that attacks the vulnerability API-Level Exploit A formal way to think about any exploit. View steps of the exploit as operations from an appropriately defined API.

ICSE 2005Automatic Discovery of API-Level Exploits19 API-Level Exploits Example [Chen and Wagner, CCS 2002] setuid(0) followed by execl can lead to root privileges. View system calls as API to UNIX. API-Level Exploit View steps of the exploit as operations from an appropriately defined API.

ICSE 2005Automatic Discovery of API-Level Exploits20 API-Level Exploits Example [Chen and Wagner, CCS 2002] setuid(0) followed by execl can lead to root privileges. View system calls as API to UNIX. API-Level Exploit View steps of the exploit as operations from an appropriately defined API. This sequence of system calls is allowed by UNIX, but compromises security

ICSE 2005Automatic Discovery of API-Level Exploits21 Framework for Modeling APIs Goal: Find exploits at the API-Level. Requirement 1: Model low-level details, reason about control and data. Stack layouts. Pointer movements. Solution: Model using expressive logics. Requirement 2: Only check allowed sequences. Solution: Encode sets of allowed sequences.

ICSE 2005Automatic Discovery of API-Level Exploits22 Framework for Modeling APIs System modeled using: Set of variables that describe its state. API operations that change state. Language of allowed API operations. System S

ICSE 2005Automatic Discovery of API-Level Exploits23 Framework for Modeling APIs System modeled using: Set of variables that describe its state. API operations that change state. Language of allowed API operations. System S OP1 OP2 OP3

ICSE 2005Automatic Discovery of API-Level Exploits24 Framework for Modeling APIs Checker: Basic idea Specify what is Bad for the system. Check all allowed inter-leavings of API operations. Problem: Arbitrarily long API operation sequences. Solution: Only check finite prefixes. Use bounded model checking. Implication: Only find exploits of bounded length. See paper for details.

ICSE 2005Automatic Discovery of API-Level Exploits25 Talk structure Motivation and Overview. Framework for finding API-level exploits. Example: format-string exploit-detector. Overview of printf and format-string exploits. Instantiating printf in our framework. Results. Comparison with other tools. Related work and Conclusions.

ICSE 2005Automatic Discovery of API-Level Exploits26 Format-string vulnerabilities Allow intruder to assume privileges of the victim program. Highly prevalent. [CERT] Vulnerability-detection tools available. Example: Percent-S, Avots et al. Goals of our tool: Systematically find exploits against such vulnerabilities. Work with real-world applications.

ICSE 2005Automatic Discovery of API-Level Exploits27 Overview of printf //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } Stack growth Pointers High addresses Low addresses

ICSE 2005Automatic Discovery of API-Level Exploits28 Overview of printf //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } buf LEN

ICSE 2005Automatic Discovery of API-Level Exploits29 Overview of printf //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } buf LEN Pointer to buf

ICSE 2005Automatic Discovery of API-Level Exploits30 Overview of printf //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } buf LEN Stack frame of log

ICSE 2005Automatic Discovery of API-Level Exploits31 Overview of printf //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } buf LEN Pointer to buf

ICSE 2005Automatic Discovery of API-Level Exploits32 Overview of printf //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } buf LEN Stack frame of fprintf

ICSE 2005Automatic Discovery of API-Level Exploits33 Overview of printf //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } buf LEN argptr DIS fmtptr

ICSE 2005Automatic Discovery of API-Level Exploits34 Overview of printf //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } buf LEN argptr DIS fmtptr buf = %x%x%s

ICSE 2005Automatic Discovery of API-Level Exploits35 Overview of printf //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } buf LEN argptr DIS fmtptr buf = %x%x%s 4 bytes, integer

ICSE 2005Automatic Discovery of API-Level Exploits36 Overview of printf //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } buf LEN argptr DIS fmtptr buf = %x%x%s 4 bytes, integer

ICSE 2005Automatic Discovery of API-Level Exploits37 Overview of printf //Format & enter into LOG void log(char *fmt,...){ fprintf(LOG,fmt,...); return; } //Call log on user input int foo(void){ char buf[LEN]; … fgets(buf,LEN-1,FILE); log(buf); … } buf LEN argptr DIS fmtptr buf = %x%x%s 4 bytes, address

ICSE 2005Automatic Discovery of API-Level Exploits38 Format-string Exploits buf LEN argptr DIS fmtptr What if we move argptr into buf ? Remember, attacker can control buf !

ICSE 2005Automatic Discovery of API-Level Exploits39 Format-string Exploits LEN argptr DIS fmtptr Example exploit scenario: fmtptr is at a %s buf contains an attacker- chosen address. argptr points to this location within buf Can read from arbitrary memory location! Writes also possible, paper has details. %s address

ICSE 2005Automatic Discovery of API-Level Exploits40 Format-string Exploits buf LEN argptr DIS fmtptr Exploit technique just discussed is well-known Our key observations: 1.DIS and LEN completely characterize any printf call. 2. Each byte in buf instructs printf what to do next.

ICSE 2005Automatic Discovery of API-Level Exploits41 Format-string Exploits buf LEN argptr DIS fmtptr Each byte to printf is an instruction. View format-string as sequence of API-ops. Format-string exploits = API-level exploits.

ICSE 2005Automatic Discovery of API-Level Exploits42 Finding Format-string Exploits Format-string = sequence of API operations. Modeling printf : from the code in libc. Various flags and variables that describe printf. API operations: all ASCII characters (|Σ| = 256). Allowed API operations (Σ*). printf %x s d g f E.g., # of bytes printed, # of bytes to read off the stack,…

ICSE 2005Automatic Discovery of API-Level Exploits43 Finding Format-string Exploits buf LEN argptr DIS fmtptr Construct model of printf (one-time activity) Model is parameterized by DIS and LEN. At each printf call site, DIS and LEN can be obtained by disassembly.

ICSE 2005Automatic Discovery of API-Level Exploits44 Finding Format-string Exploits Specialize printf to call site using values of DIS and LEN. Formulate Bad. Paper has examples of Bad that can be used to read from, or write to, an arbitrary memory location. Run the bounded model checker. Only need to check format-strings shorter than LEN Have a bound for the model checker!

ICSE 2005Automatic Discovery of API-Level Exploits45 Finding Format-string Exploits The model of printf : Requires precise reasoning about stack locations, in particular, the format-string. Has integer operations: pointer arithmetic to advance fmtptr and argptr. Quantifier-free Presburger-arithmetic with theory of uninterpreted functions and arrays. UCLID tool. [Bryant et al. CAV 2002]

ICSE 2005Automatic Discovery of API-Level Exploits46 Format-string Exploit-detector Finds exploits against vulnerabilities in real- world software packages. Can find different kinds of exploits. Can find an arbitrary number of variations of a given exploit. Can work on binary executables. Can improve the quality of format-string vulnerability-detection tools.

ICSE 2005Automatic Discovery of API-Level Exploits47 Use Scenario Percent-S [Shankar et al. USENIX Security 2001] finds possibly vulnerable locations. No exploits. Run our tool at each vulnerable location: Exploit generated: true vulnerability. No exploit generated: possibly a false alarm. Format-string vulnerability finding tool List of vulnerabilities Our tool Exploit Possible false alarm Exploit Possible false alarm Exploit

ICSE 2005Automatic Discovery of API-Level Exploits48 Results Exploits against vulnerabilities in real- world software: See paper for details SoftwareDISLENExploit description php Overwrite memory location qpopper Read a memory location wu-ftpd Overwrite memory location

ICSE 2005Automatic Discovery of API-Level Exploits49 Results DIS LEN Read exploitWrite exploit 07 a 1 a 2 a 3 a 4 %sNo exploit a 1 a 2 a 3 a 4 %d%s%234Lg%na 1 a 2 a 3 a %Lx%ld%sa 1 a 2 a 3 a 4 a 1 a 2 a 3 a 4 %%229x%n 816 a 1 a 2 a 3 a 4 %Lx%sa 1 a 2 a 3 a 4 %230g%n 16 %Lg%Lg%sa 1 a 2 a 3 a 4 a 1 a 2 a 3 a 4 %137g%93g%n 20 a 1 a 2 a 3 a 4 %Lg%g%sa 1 a 2 a 3 a 4 %210Lg%20g%n 2420 a 1 a 2 a 3 a 4 %Lg%Lg%sa 1 a 2 a 3 a 4 %61Lg%169Lg%n 3224 a 1 a 2 a 3 a 4 %g%Lg%Lg%sa 1 a 2 a 3 a 4 %78Lg%80g%72Lg %n

ICSE 2005Automatic Discovery of API-Level Exploits50 Results DIS LEN Read exploitWrite exploit 07 a 1 a 2 a 3 a 4 %sNo exploit a 1 a 2 a 3 a 4 %d%s%234Lg%na 1 a 2 a 3 a %Lx%ld%sa 1 a 2 a 3 a 4 a 1 a 2 a 3 a 4 %%229x%n 816 a 1 a 2 a 3 a 4 %Lx%sa 1 a 2 a 3 a 4 %230g%n 16 %Lg%Lg%sa 1 a 2 a 3 a 4 a 1 a 2 a 3 a 4 %137g%93g%n 20 a 1 a 2 a 3 a 4 %Lg%g%sa 1 a 2 a 3 a 4 %210Lg%20g%n 2420 a 1 a 2 a 3 a 4 %Lg%Lg%sa 1 a 2 a 3 a 4 %61Lg%169Lg%n 3224 a 1 a 2 a 3 a 4 %g%Lg%Lg%sa 1 a 2 a 3 a 4 %78Lg%80g%72Lg %n Ability to find false alarms

ICSE 2005Automatic Discovery of API-Level Exploits51 Results DIS LEN Read exploitWrite exploit 07 a 1 a 2 a 3 a 4 %sNo exploit a 1 a 2 a 3 a 4 %d%s%234Lg%na 1 a 2 a 3 a %Lx%ld%sa 1 a 2 a 3 a 4 a 1 a 2 a 3 a 4 %%229x%n 816 a 1 a 2 a 3 a 4 %Lx%sa 1 a 2 a 3 a 4 %230g%n 16 %Lg%Lg%sa 1 a 2 a 3 a 4 a 1 a 2 a 3 a 4 %137g%93g%n 20 a 1 a 2 a 3 a 4 %Lg%g%sa 1 a 2 a 3 a 4 %210Lg%20g%n 2420 a 1 a 2 a 3 a 4 %Lg%Lg%sa 1 a 2 a 3 a 4 %61Lg%169Lg%n 3224 a 1 a 2 a 3 a 4 %g%Lg%Lg%sa 1 a 2 a 3 a 4 %78Lg%80g%72Lg %n Ability to find different kinds of exploits: Parameterized by the predicate Bad

ICSE 2005Automatic Discovery of API-Level Exploits52 Results DIS LEN Read exploitWrite exploit 0 7 a 1 a 2 a 3 a 4 %sNo exploit a 1 a 2 a 3 a 4 %d%s%234Lg%na 1 a 2 a 3 a %Lx%ld%sa 1 a 2 a 3 a 4 a 1 a 2 a 3 a 4 %%229x%n 816 a 1 a 2 a 3 a 4 %Lx%sa 1 a 2 a 3 a 4 %230g%n 16 %Lg%Lg%sa 1 a 2 a 3 a 4 a 1 a 2 a 3 a 4 %137g%93g%n 20 a 1 a 2 a 3 a 4 %Lg%g%sa 1 a 2 a 3 a 4 %210Lg%20g%n 2420 a 1 a 2 a 3 a 4 %Lg%Lg%sa 1 a 2 a 3 a 4 %61Lg%169Lg%n 3224 a 1 a 2 a 3 a 4 %g%Lg%Lg%sa 1 a 2 a 3 a 4 %78Lg%80g%72Lg %n Ability to find variants of an exploit

ICSE 2005Automatic Discovery of API-Level Exploits53 Talk Structure Motivation and Overview. Framework for finding API-level exploits. Example: format-string exploit-detector. Overview of printf and format-string exploits. Instantiating printf in our framework. Results. Comparison with other tools. Related work and Conclusions.

ICSE 2005Automatic Discovery of API-Level Exploits54 Related Work Software Model Checking [Blast, SLAM, Magic, CBMC, Saturn] Counter-example guided abstraction refinement. Exploits Concrete counter-examples. Test generation [Beyer et al. ICSE04, Boyapati et al. ISSTA02] Bounded exhaustive checking. Exploits can be used as test cases. Supercompilers [Massalin ASPLOS87, Joshi et al. PLDI02] Model low-level semantics. Exhaustive checking to generate compact code sequences. Ad-hoc techniques [Thuemmel 2001, Newsham 2000] No soundness guarantees. Cannot find variants.

ICSE 2005Automatic Discovery of API-Level Exploits55 Summary of Important Ideas Exploit-finding requires modeling low-level details of the runtime system. Exploit-finding can benefit vulnerability-finding tools. Framework for producing API-level exploits. Real-world instantiation: format-string exploits.

Automatic Discovery of API-Level Exploits Vinod Ganapathy, Sanjit Seshia, Somesh Jha, Thomas Reps & Randal Bryant