KEVIN COOGAN, GEN LU, SAUMYA DEBRAY DEPARTMENT OF COMUPUTER SCIENCE UNIVERSITY OF ARIZONA 報告者：張逸文 Deobfuscation of Virtualization- Obfuscated Software ADLab 1

Outline ADLab 2 Introduction Deobfuscation Experimental Evaluation Related Work Conclusion

Introduction （ 1/4 ） Basic about Reverse Engineering  Compilation  Decompilation ADLab 3

Introduction （ 2/4 ） ADLab 4 Virtualization obfuscators  VMProtect, Code Virtualizer VMProtectCode Virtualizer { VIRTUALIZER_START your code VIRTUALIZER_END }

Introduction （ 3/4 ） ADLab 5 The virtualization-obfuscated programs are resistant to static and dynamic analysis techniques  The executed code reveals only the structure and logic of the byte- code interpreter  Randomness VM Outside-in approach  Reverse engineer the VM interpreter  Individual byte code instructions  Recover the logic  The structure of the interpreter meets certain requirements

Introduction （ 4/4 ） ADLab 6 Programs interact with the system through system calls Identifying instructions that interact with the system Not recovering the original instructions Capturing behavior of the code General, using in a wide range

Deobfuscation ADLab 7 Static analysis v.s dynamic trace Identifying instructions that are known to be part of the original code No information about the specific structure of the interpreter

Deobfuscation ADLab 8 Overall approach ： 1. Tracing tool  Low level execution trace 2. Identifying system calls and their arguments  database 3. Instruction trace  Relevant instructions 4. Building a subtrace  Relevant subtrace

Deobfuscation ADLab 9 Value-based Dependence Analysis  Not recovering the original code  The process of deobfuscation must be semantics-preserving  Identifying instructions that affect the values of the arguments to system calls  Slicing algorithms --- control-dependent  Data dependencies  Use-definition chains --- link instructions that use a variable to the instruction that define it  Problem ：

Deobfuscation ADLab 10  Value-based dependence if( I defines a location l S) { I is marked as relevant; l is removed from S; the set of locations used by I is added to S; }  Problem ： a pointer to a structure I uses some locations  l 1, l 2, …, l d if ( I uses l i P to define l d ) l d is added to P if ( l i access a memory location ) [l i ] is added to M

Deobfuscation ADLab 11 Relevant Conditional Control Flow  Value-based dependence analysis doesn’t identify the associated control flow instructions  The occurring of conditional control flow  IA-32 architecture  setting the condition code flags in the eflags register  Not such simple!!  Examining target address  Equational Resoning System ： translate each instruction in the dynamic trace into an equivalent set of equations

Deobfuscation ADLab 12  Equational Resoning System  Identifies conditional dependencies  The left hand side variables in an equation is numbered by the order of its instruction appears  The right hand side variables is numbered by the instruction that defined it  Example 1.

Deobfuscation ADLab 13  Example 2.  Example 3.  Indirect jump

Deobfuscation ADLab 14  Example 4.  Used in VMProtect Target 20 = index1*4+0x10000

Deobfuscation ADLab 15

Deobfuscation ADLab 16

Deobfuscation ADLab 17 Relevant Call-Return Control Flow  Identifying functions ： the behavior of calls and returns  Knowing how them work allows one to use for other purposes  Behavior of Function Calls and Returns

Deobfuscation ADLab 18 registers call 改成 push 無法解決

Deobfuscation ADLab 19  Identification Approach  Call ： a code address is saved at the call site  Return ： the saved address is used for a control transfer at the return point

Deobfuscation ADLab 20 Relevant Dynamic Trace

Experimental Evaluation ADLab 21 Experimental Methodology  Compile original source code  Generate an original dynamic trace  Build an original subtrace  Virtualization-obfuscation technique  Generate an obfuscated dynamic trace  Build a relevant subtrace of the obfuscated subtrace  The obfuscated subtrace is matched to the original subtrace and scores are produced  The relevance score and obfuscation score are calculated

Experimental Evaluation ADLab 22 VX Heavens website

Related Work ADLab 23 Deobfuscation of code obfuscated via virtualization obfuscators  Rolles, Sharif, Falliere Programming language community  Partial evaluation

Conclusions ADLab 24 Virtualization-obfuscated programs are difficult to reverse engineer We present a different approach to identifying the flow of values to system call instructions

XD ~ ADLab 25