Presentation on theme: "Dean Carlson and Beth Anne Byrd CpSc 420. What is reverse engineering? Brief History Usefulness The process Bagle Virus example."— Presentation transcript:
Dean Carlson and Beth Anne Byrd CpSc 420
What is reverse engineering? Brief History Usefulness The process Bagle Virus example
“[T]he process of analyzing a subject system to create representations of the system at a higher level of abstraction” (Chikofsky, 1990). Going through the software development cycle backwards
Started as analyzing hardware in an attempt to gain an advantage. The first time this was applied to a piece of malware was in Bernt Fix disassembled and neutralized the Charlie virus.
Analysis of a product Recreating lost or nonexistent documentation Academic use Curiosity With Malware Contain it Remove it Prevent it
Diffuse “time bombs” Conficker virus ▪ The Y2K of today
1. Set up a controlled, isolated laboratory 2. Perform behavioral analysis to examine the specimen’s interactions with its environment. 3. Perform static code analysis to further understand the specimen’s inner-workings. 4. Perform dynamic code analysis to understand the more difficult aspects of the code. 5. If necessary, unpack the specimen. 6. Repeat steps 2, 3, and 4 (order may vary) until sufficient analysis objectives are met. 7. Document findings and clean-up the laboratory for future analysis.
HOST: Windows XP in Virtual Machine DataRescue IDA Pro Microsoft Visual C++ ▪ Dumpbin UltraEdit SERVER Solaris 9 (SPARC) Snoop BIND (DNS) GCC GDB
The and DNS programs on the server were setup to log all of their activity and network traffic in order to see the virus interact with the server.
Open in IDA Pro Breaks it down into assembly and hex
Open in dumpbin to determine type PE (Portable Executable)
Walk through the virus step by step with a debugger and look at register values. Especially EAX, EIP, ZF bit of EFLAGS EAX = return values from functions ZF = flag used for comparisons and decisions EIP = useful for thread usage
Use IDA to chart subroutines
Use IDA to identify function parameters and variables arg_8 can be accessed by adding “10h” to the EBP Register
The Bagle virus was not packed Compressed or encrypted It also was not polymorphic Changing the assembly, usually by inserting “noop” thus changing the virus signature but not changing the effectiveness The Bagle virus has many removal tools
Reverse engineering malware started in 1987 It is good to contain, remove, and prevent malware 7 steps 1. Set up lab 2. Behavioral analysis 3. Static code analysis 4. Dynamic code analysis 5. Unpack 6. Repeat steps 2, 3, and 4 7. Document and clean-up