1. Buffer Overflow Attack- proofing of Code Binaries Ramya Reguramalingam Gopal Gupta Gopal Gupta Department of Computer Science University of Texas at Dallas

2. ALPS LAB@UTD Buffer Overflow: A Serious Problem  Percentage of buffer overflows listed in CERT advisories each year  Some examples include Windows 2003 server, sendmail, windows HTML conversion library Percentage of Buffer Overflows Per Year as listed by CERT [1]

3. ALPS LAB@UTD Overview of our approach  Buffer Overflow Attacks (B.O.A): A majority of attacks for which advisories are issued are based on B.O.A.  Other forms of attacks, such as distributed denial of service attacks, sometimes rely on B.O.A.  B.O.A. exploit the memory organization of the traditional activation stack model to overwrite the return address stored on the stack.  This memory organization can be slightly changed so as to prevent buffer overflows overwriting return addresses.  Our system automatically transforms code binaries in accordance to this modified memory organization, thereby preventing most common forms of B.O.A.s.  Our tool can be used on third-party s/w and off-the-shelf products, & does not require access to source code.

4. ALPS LAB@UTD Traditional Memory Organization  Sample Code void function (int a, int b, int c){ char buffer1[8]; } void main( ){ function(5, 8, 2); } Stack at the start ESP Stack Heap Data Code 00 ff ff

5. ALPS LAB@UTD Stack Organization: After a Call  Sample Code void function (int a, int b, int c){ char buffer1[8]; } void main( ){ function(5, 8, 2); } Stack after a function call Buffer variables... Stack Param 3 = 2 Param 2 = 8 Param 1 = 5 Return address ebp Buffer variables Heap, Data & Code EBP ESP Parameters

6. ALPS LAB@UTD Buffer Overflow  Sample Code void function (char *str){ char buffer1[8]; strcpy (buffer1, str); } void main( ){ char large_str[256] ; for (int i=0; i<255; i++) large_str[i] = ‘A’; function(large_str);}  New return address =41414141 Stack showing buffer overflow Stack Large_str  64 Return address ebp Buffer1  2 Direction Strcpy writes

7. ALPS LAB@UTD Abusing the Buffer Overflow  Step 1: Overwrite the return address with an address that points ‘back’ to the buffer area  Step 2: Insert code that you wish to execute in the buffer area  Step 3: Buffer start of inserted code with NOP instructions  Step 4: Eliminate any null values in inserted code sssssssssssssssssssebpret

8. ALPS LAB@UTD Buffer Overflow Attack-proofing  B.O.A. done by overwriting return addresses  If return addresses are recorded in a separate area, away from the direction of the buffer overflow, then they cannot be overwritten  So modify the memory organization to add a new return address stack, allocated in an area opposite to the direction in which buffers are allocated.  When a function call exits it uses the return address in this new return address stack

9. ALPS LAB@UTD B.O.A-proofing (Cont’d)  Transform the executable binary code so it is consistent with this new memory organization  Conditions that must hold for this transformation to be semantics preserving: Code must be re-entrant Code must be re-entrant Code should not modify the stack pointer or use it for computation in the program. Code should not modify the stack pointer or use it for computation in the program. Processor: Intel x386 Processor: Intel x386 Compiler: Dev C++ compiler 4.9.9.1 Compiler: Dev C++ compiler 4.9.9.1 Platform: Windows Platform: Windows

10. ALPS LAB@UTD Transforming the Binary  Step 1: Analyze the binary PE format and develop flow graph for the code  Step 2: Modify the binary such that return address is stored on a separate stack (within the stack area); This separate stack is allocated in an area opposite to the direction of buffer growth This separate stack is allocated in an area opposite to the direction of buffer growth  Step 3: Convert the modified hexadecimal code back to.exe

11. ALPS LAB@UTD Step 1 - Sample Source Code  Sample Code int func1 (int a, int b){ int temp = a+ b; return (temp); } void main( ){ int a, b; a = 7; b = func1(3,4); a +=b; }  The code is compiled  The.exe file generated is disassembled MASM 32

12. ALPS LAB@UTD Step 1 - Disassembled Code  Sample Code int func1 (int a, int b){ int temp = a+ b; return (temp); } void main( ){ int a, b; a = 7; b = func1(3,4); a +=b; }  55 89 E5 83 EC 04 8B 45 03 0C 45 08 89 45 FC 8B 45 FC C9 C3 55 89 E5 83 EC 18 83 E4 F0 B8 00 00 00 00 89 45 F4 8B 45 F4 E8 A3 04 00 00 E8 0E 01 00 00 C7 45 FC 07 00 00 00 C7 44 24 04 04 00 00 00 00 C7 04 24 03 00 00 00 E8 B3 FF FF FF 89 45 F8 8B 55 F8 8D 45 FC 01 10 C9 C3

13. ALPS LAB@UTD Step 1 - Intel Opcode Decoded  55Push ebp  89 E5Mov ebp, esp  83 EC 04 Sub esp, 0x04  8B 45 0C Mov eax, [ebp+0x0c]  03 45 08 Add eax, [ebp+0x08]  89 45 FC Mov [ebp-0x04],eax  8B 45 FCMov eax, [ebp-0v04]  C9Leave  C3Ret

14. ALPS LAB@UTD Step 2 – Modification of Binary 1. Before CALL push return address onto another stack Call = Push (eip) Call = Push (eip) 2. Store new stack pointer in a general purpose register whose value is saved in memory 3. Before RET is executed use new stack pointer to obtain value of return address

15. ALPS LAB@UTD Step 2 – Example of Modification  Mov edx, [0x13000000] Location of stack pointer  Mov [edx], eip+0x04 Return Address is stored  Sub edx, 0x04 Decrement Stack Pointer  Call function1  Add edx, 0x04 Increment Stack Pointer to get value  Mov [esp], [edx] Move stored address to current stack  Ret

16. ALPS LAB@UTD Advantages  Binary code is analyzed and transformed. Our approach can be used on third-party software when one does not have access to source code. Run-time checks require modification to the source code Run-time checks require modification to the source code Compiler modifications are costly and performing changes to all available compilers is not possible. Compiler modifications are costly and performing changes to all available compilers is not possible.  Return addresses are stored on the stack itself. Hence overhead incurred while accessing addresses in other areas is reduced.

17. ALPS LAB@UTD Disadvantages  The stack has to store a list of return addresses. Storage overhead = depth of the flow graph is incurred.  The code is machine dependent. But, it covers machines from 80x86 upwards. A large number of machines fall in this category.