CSS 548 Dan Chock

 What are some ways that compilers can affect application security? ◦ Improving Application Security  Checking for and preventing potential vulnerabilities ◦ Additional Effects on Security  Understanding compiler optimization for binary obfuscation ◦ Reducing Application Security  Compiler vulnerabilities and backdoors

 More Secure Code ◦ Preventing stack buffer overflow exploitation via canaries “Anatomy of a Stack Smashing Attack and How GCC Prevents It”, Dr. Dobbs, Oliver Mueller, June 19, 2012

 GCC Options ◦ -fstack-protector ; -fstack-protector-all  Adds a guard variable (aka canary) to functions that are initialized when a function is entered and checked when exited. If a guard check fails the program exits.  Visual Studio Switches ◦ /GS  Shadowing to protect class member definitions, stack data reordering to protect function pointers

◦ /sdl – Superset of /GS, expands scope and treats select security warnings as errors “Compiler Security Enhancements in Visual Studio 11”, SDL Team, MSDN, Dec. 2, 2011

 Binary Obfuscation ◦ volatile keyword “Binary Obfuscation from the Top Down”, Sean Taylor

 Binary Obfuscation ◦ volatile keyword

 Compiler Vulnerabilities ◦ Insecure Compiler Optimization  Improperly removing sensitive data from memory  Dead store removal - code that is used to overwrite memory is removed by the compiler  Compiler backdoors ◦ Modifying compiler to recognize specific code while compiling and include a backdoor in the output  Ken Thompson - “Reflections on Trusting Trust” describes modified C compiler that would backdoor the login command to accept backdoor password

 Compiler/hardware approaches to embedded systems security [1] ◦ Secure compiler inserts hidden code into the compiled code that is validated during run-time by hardware  Security-preserving compiler for distributed programs [2] ◦ Four stages: slicing, control flow protection (to keep track of state), replication (implement distributed program variables as local replicas), and cryptography (insert crypto operations to protect variable changes)

 [1] “Performance Study of a Compiler/Hardware Approach to Embedded Systems Security”, Kripashankar Mohan, Bhagi Narahari, Rahul Simha, Paul Ott1,Alok Choudhary, and Joe Zambreno  [2] “A Security-Preserving Compiler for Distributed Programs From Information-Flow Policies to Cryptographic Mechanisms”, Cédric Fournet, Gurvan Le Guernic, Tamara Rezk