The Impact of Programming Language Theory on Computer Security Drew Dean Computer Science Laboratory SRI International

2 What I’m not Talking About Cryptographic Protocol Verification  See, e.g., Computer Security Foundations Workshop Type Systems for Non-Interference  See, e.g., POPL

3 Much of security is: “Program P exactly implements Specification S and no more.” For this talk, we assume that the specification is correct

4 Security Tripos No undefined user mode behavior Proper system call use Correctness wrt critical requirements

5 Correctness wrt Security No system that misses security checks can be secure  Program Verification  Architectural Support Stack inspection Security Passing Style [WAF]

6 Program Verification Obvious connections  Lambda calculus, Curry-Howard  Hoare Logic  …

7 Architectural Support Stack Inspection  Access control based on endorsement of code: answers “Who called me?”  Designed to prevent untrusted code from bypassing access controls, while allowing higher level code to assert that it knows what it’s doing

8 Stack Inspection Example Applet wants to use the Helvetica font May require JVM to read a file Solution:  Font handling code checks arguments  If successful, asserts privilege  Attempts to read file Which notes that font code (privileged) has asserted everything’s OK

9 Stack Inspection: Critique Exposes call stack  Tail call elimination painful  Function inlining also painful  Goodbye, Church-Rosser, goodbye!

10 Security Passing Style Wallach, Appel, Felten, TOSEM 9/00 A la CPS, pass security context as an extra (implicit) argument Restores tail call elimination and function inlining Doesn’t restore Church-Rosser

11 Observation SPS is in closer analogy to CPS than its authors say Shivers: “Threads are paths through continuation space” Continuations are the right semantic object to attach permissions to Would a dependent type system work out?

12 Properly Using System Calls If a program handles its own security, e.g., ftpd, it better use system calls correctly Many programs don’t  Wu-ftpd  Sendmail  …

13 How Can PLT help? Joint work with David Wagner and Hao Chen, UC Berkeley Given a program, morph control flow graph into an automaton that accepts language of system calls

14 IEEE S&P 2001 Take automaton, check runtime trace of system calls for anomaly detection (Most of) Benefits of specification- based intrusion detection without needing the non-existent spec

15 Current Work Take abstracted specification, throw it and library of security “best practices” (and known attacks) at (custom) model checker But this requires understanding system calls Usually the POSIX spec is reasonable But not for set*uid()

16 Understanding set*uid Absolutely necessary for writing secure setuid Unix programs Linux, FreeBSD, Solaris all subtly different  Even if all POSIX compliant Kernel code unreadable Reverse engineer formal model Will appear at USENIX Security 2002

17 No Undefined User-mode Behavior Buffer overflows are still a problem in 2002 PL people think this is stupid  It is Like it or not, most of the world codes in C or unsafe C++

18 Not Just Buffer Overflows Any corruption of program state can cause vulnerability  Nearly science fiction attack based on a C program double freeing a pointer

19 Observation Memory comes in two colors  Storage of variables  Compiler/runtime support

20 Partition Property “All variables only refer to memory locations that the compiler has mapped to program variables, not compiler/runtime support (e.g., return addresses, temporaries for evaluating expressions, memory management overhead, etc.)”

21 Partition Properties Note that this is weaker than non- interference  Values obviously depend on program values Stronger than some forms of memory & type safety Should be a theorem of modern (safe) languages

22 Conclusions This was a brief survey of a wide field “and no more” is hard to implement Hopefully, breaking it down helps No undefined behavior Proper system call use Correctness wrt critical requirements