1. Software Reverse Engineering Binary analysis: concepts, methods and tools. Catalin Patulea http://vv.carleton.ca/~cat/ Mar 5, 2008

2. Overview Blackbox analysis Whitebox analysis Surgical analysis aka. names-I-made-up-to-split-up-this-talk- into-multiple-sections

3. Relevance Malware analysis, signature and countermeasure creation Detecting IP infringement Compatibility –Protected by law in certain countries Backup copies –Teehee.. ;) “How’d it do that!?” Learning?

4. Blackbox Analysis Analyze the binary’s interaction with the OS –Files, registry, network Usually only good as a first idea of what you’re dealing with –Does it keep a log? Where does it keep user settings? Does it “call home”? Helps to keep the rest of the analysis bounded –After all, the binary can’t do any anything without the OS’ help

5. Blackbox Tools Dependency Walker Oleview Process Explorer FileMon RegMon Wireshark strace Google it, Luke!

6. Blackbox Demo

7. Whitebox Analysis Except for the simplest binaries, you have to get your hands dirty Convert the binary into assembly code (disassemble it), read, and understand it Requires intimate knowledge of: –Assembly language –Operating system interfaces –Calling conventions –Typical instruction sequences and their meaning –etc.

8. Whitebox Analysis: Challenges Binaries over a few hundred KB (depending on your comfort level and your disassembler) become very time- consuming Some binaries are obfuscated –Packing (ala UPX, PECompact, ASPack) – very common –Polymorphism (Morphine) – typically found in malware

9. Whitebox Analysis: Demo UPX DataRescue’s IDA Pro

10. Surgical Analysis Context is as important as the code itself –Knowing the value of a particular register at some point can be crucial –Helps to determine code paths, program state Potentially dangerous (malware) as it involves running the binary on your computer –Could run inside insolated VM (sometimes inconvenient) Debuggers allow us to control the execution flow of the binary –Microsoft-provided symbol files help understand OS interaction

11. Surgical Analysis: Demo Visual Studio OllyDbg IDA Pro