1. Design of a Framework for Testing Security Mechanisms for Program-Based Attacks Ben “Security” Breech and Lori Pollock University of Delaware

2. Motivation o Unsafe programming practices –Exploited by malicious input (program-based attack) o Vulnerabilities difficult to find and fix –Expensive (time, resources required) –Often wait until exploit is found o Mechanisms proposed to protect programs –Poorly tested –Low confidence in security protection

3. Testing Security Mechanisms o Testing is often poor and non-systematic –Current process: Find program with vulnerability and known exploit. Apply the mechanism and try the exploit. –Finding test subjects can be difficult o Need a framework that provides a systematic, and automatic method of testing security mechanisms –More thorough testing of mechanisms

4. Program-Based Attacks o Malicious attacks initiated as input –Usually buffer overflow o Examples –Stack smashing –Function pointers –Heap variables

5. Example Attack int proc_inp (int a, int b, int c) { char buf1 [3]; int z; char buf2 [5]; gets (buf2) …. } 42 Input: ABCDEFGHIJKLMNOPQRST c -> 6 7 0x1200 b -> a -> return address -> saved fp -> buf1 [] -> 0x150 z -> buf2 [] -> QRST MNOP JKL FGHI ABCDE Caller AR proc_inp AR

6. Example Mechanism: RAD A() { char s [5]; gets (s); B(s) … ret } B(char *s) { C(s) … ret } C(char *s) { … ret } A AR: 0x1200 C AR: 0x1280 B AR: 0x1250 0x1200 0x1250 0x1280 Call Stack: RAD Stack: (Chiueh and Hsu, ICDCS 2001) Input: ABC

7. Our Framework… o Present the design of a framework to test security mechanisms –Enables systematic and automatic testing o Key insight: use dynamic compiler to simulate attacks

8. Framework Design

9. Framework Key Components

10. Framework Requirements o General -- support different languages, vulnerabilities and mechanisms o Systematic -- insert attacks at appropriate points o Automatic -- little user interaction o Support testing of evolving programs o Robust -- few false positives o Low overhead -- both space and time

11. The Testing Process A AR: 0x1200 A AR: 0x5000 C AR: 0x1280 B AR: 0x1250 0x1200 0x1250 0x1280 Call Stack: RAD Stack: B AR: 0x5000 C AR: 0x5000 A() { char s [5]; gets (s); B(s) … ret } B(char *s) { C(s) … ret } C(char *s) { … ret } Input: ABC

12. Prototype Implementation Use the DynamoRIO 1 dynamic compiler 1 (Bruening et al. CGO 03) Native Binary Code Initialization Basic Block Construction Initialization Basic Block Execution Exit Analysis or Optimization Cleanup And Exit DynamoRIO (simplified view) Client Module

13. Implementation Issues o Simulation of attacks -- may not perfectly mimic all effects of attack o Efficiency -- overhead tradeoff o Monitoring -- what to look for? o Key component goals -- tradeoff? –No extra code compiled in –All possible attack points tested o Automatically specifying attacks

14. Summary and Current Status o Presented design of framework for testing security mechanisms for program-based attacks o Proof of concept implementation