1 Achieving Trusted Systems by Providing Security and Reliability (Research Project #22) Project Members: Ravishankar K. Iyer, Zbigniew Kalbarczyk, Jun Xu, Shuo Chen, Nithin Nakka and Karthik Pattabiraman

2 Objective and Approach Objective design and validate secure and reliable computing systems to support critical infrastructures. design and validate secure and reliable computing systems to support critical infrastructures.Approach analyze raw data on security vulnerabilities and attacks analyze raw data on security vulnerabilities and attacks generate stochastic and state machine models depicting security threats generate stochastic and state machine models depicting security threats apply formal method to uncover security vulnerabilities due to inconsistencies between system specifications and implementations apply formal method to uncover security vulnerabilities due to inconsistencies between system specifications and implementations implement defensive techniques at compiler, operating system and hardware levels implement defensive techniques at compiler, operating system and hardware levels

3 Accomplishments Study impact of hardware errors on system security IEEE Dependable Systems and Networks (DSN’01 and DSN’02) IEEE Dependable Systems and Networks (DSN’01 and DSN’02) State machine modeling of real-world security vulnerabilities DSN’03 DSN’03 Non-control-data attack: a new security threat USENIX Security (Security’05) USENIX Security (Security’05) Memory layout randomization-based defensive technique IEEE Reliable Distributed Systems (SRDS’03) IEEE Reliable Distributed Systems (SRDS’03) Architecture level support for reliability and security EASY’02, DSN’04 and DSN’05 EASY’02, DSN’04 and DSN’05 Formal reasoning on security vulnerabilities IFIP Information Security (SEC’04) IFIP Information Security (SEC’04)

4 Modeling and Analyzing Security Vulnerabilities An extensive study on security vulnerabilities in Bugtraq and CERT Each vulnerability is decomposed to multiple simple predicates. State machine modeling for buffer overflow, format string bug, heap corruption, integer overflow, … A more formal way for reasoning about security vulnerabilities. WU-FTP Server Format String Attack NULL-HTTP Server Heap Corruption Attack (DSN’03)

5 New Security Threat: Non-Control-Data Attack Most current attacks are control-data attacks Corrupting function pointers or return addresses to run malicious code. Corrupting function pointers or return addresses to run malicious code. Many defensive techniques are proposed to defeat control-data attacks. Many defensive techniques are proposed to defeat control-data attacks. New threat: non-control-data attacks are generally applicable to attack real-world systems User identity data, configuration data, user input data and decision-making Booleans are security critical. User identity data, configuration data, user input data and decision-making Booleans are security critical. Non-control-data attacks can obtain the root privilege on FTP, SSH, HTTP and Telnet servers. Non-control-data attacks can obtain the root privilege on FTP, SSH, HTTP and Telnet servers. A comprehensive defensive technique is needed to defeat both types of attacks. A comprehensive defensive technique is needed to defeat both types of attacks. (USENIX Security’05)

6 Pointer Taintedness Detection for Security The root cause of many attacks (control-data attacks and non-control-data attacks): pointer taintedness Pointer taintedness: a pointer value is derived directly or indirectly from user input. Prevent pointer taintedness  defeat many real-world attacks, e.g., stack smashing, format string, heap corruption, integer overflow, and globbing attacks. e.g., stack smashing, format string, heap corruption, integer overflow, and globbing attacks.format stringformat string Pursued directions Pointer taintedness avoidance – uncover vulnerabilities by source code analysis Pointer taintedness avoidance – uncover vulnerabilities by source code analysis Pointer taintedness detection – check pointers at runtime. Pointer taintedness detection – check pointers at runtime.

7 Source Code Analysis to Uncover Pointer Taintedness Formal semantic definition of pointer taintedness using equational logic Develop a theorem proving technique to analyze C source code at machine code level Extract a set of preconditions for each analyzed function Satisfaction of preconditions  no possibility of pointer taintedness inside this function Satisfaction of preconditions  no possibility of pointer taintedness inside this functionEvaluation Analyze strcpy(), printf(), free() and socket read functions of HTTP servers Analyze strcpy(), printf(), free() and socket read functions of HTTP servers Negations of extracted preconditions suggest scenarios of potential vulnerabilities. Negations of extracted preconditions suggest scenarios of potential vulnerabilities. (SEC’04)

8 Runtime Pointer Taintedness Detection A processor architectural level mechanism to detect pointer taintedness On SimpleScalar simulator On SimpleScalar simulator Implemented a taintedness-aware memory system Extended ALU instructions to propagate taintedness in memory Evaluation using several network applications and SPEC benchmarks Effective in detecting both control-data attacks and non-control-data attacks Effective in detecting both control-data attacks and non-control-data attacks Transparent to applications, precompiled binary can run. Transparent to applications, precompiled binary can run. No known false alarm. No known false alarm. (DSN’05)

9 Future Directions Combination of static code analysis and architecture support To automatically derive predicates to be checked by processor at runtime To automatically derive predicates to be checked by processor at runtime Reliability and security support for embedded systems Migrate our current techniques to embedded systems Migrate our current techniques to embedded systems New topics: cell phone virus, reduced power consumption, tamper-resistant hardware, crypto and authentication hardware/software New topics: cell phone virus, reduced power consumption, tamper-resistant hardware, crypto and authentication hardware/software

10 Backup

11 ap: argument pointer fmt: format string pointer Internals of Format String Attack In vfprintf(), if (fmt points to “%n”) then **ap = (character count) Vulnerable code: recv(buf); printf(buf); /* should be printf(“%s”,buf) */ \xdd \xcc \xbb \xaa %d %d %d %n …%n%d%d%d0xaabbccdd fmt: format string pointer ap: argument pointer High Low Stack growth *ap is a tainted value. back

12 Extracting Security Specifications by Theorem Prover C source code of a library function formal semantic representation Automatically translated to formal semantic representation Theorem generation Theorem proving A set of sufficient conditions that imply the validity of the theorems. They are the security specifications of the analyzed function. For each pointer dereference in an assignment, generate a theorem stating that the pointer is not tainted