1. Secure Virtual Architecture:
A Novel Foundation for Operating System Security
Vikram Adve
University of Illinois at Urbana-Champaign
Joint work with:
John Criswell, Dinakar Dhurjati, Sumant Kowshik, Andrew Lenharth, Balpreet Pankaj
Thanks: NSF, SRC, DARPA, Motorola, Apple

2. Acknowledgements Funding
NSF (several programs), SRC, DARPA, Motorola, Apple
LLVM, pointer analysis (DSA), Automatic Pool Allocation
Chris Lattner
Other LLVM developers, past and present
SVA PrivOps design “consultants”
Pierre Salverda
David Raila
Other input and feedback
Sam King, Roy Campbell
Many reviewers

3. Increasing threats to system software
The Context - 1
Increasing threats to system software
US-CERT: 5198 O.S. vulnerabilities reported in 2005(A)
812 on Windows, 2328 on Unix/Linux, 2058 on multiple systems
Month of Kernel Bugs (Nov. 2006)
One new bug every day
Linux: 8, MacOS: 8, FreeBSD: 2, Solaris: 1, Windows: 1
Code injection: 13, DOS: 13, Memory corruption: 4
Situation could get dramatically worse
Incentives are increasing: “botnets,” online banking, identity theft
Generations entering college are much more computer-savvy

4. Vast majority of security-critical software is in C/C++
The Context - 2
Vast majority of security-critical software is in C/C++
Existing Software
“Commodity” OS: Windows, Linux, MacOS, BSD family, …
Special-purpose OS: Secure64, QNX, …
OS services: sshd, xinetd, named, sendmail, …
Servers: Apache, Squid, MySQL,…
Existing Solutions
Only ad-hoc, partial solutions are used in production systems
E.g., non-executable stack/data, SFI, StackGuard, interface annotations, etc.
Linux kernel has 2.5M lines of code in ring 0, Windows has 5M (B)
“Eliminating Malware and Rootkits,” Secure64 White Paper, July 2006

5. Not Even Secure64 …
“Because [buffer] overflows are not preventable by the system architecture, software developers must still review their code either manually or with code-scanning tools to find and correct possible buffer overflow vulnerabilities.”
-- Secure64 White Paper, July 2006 .

6. OS on a Safe Runtime: Vision
A complete OS in a safe execution environment
Eliminates an important class of security holes
Improves system reliability
Enables novel OS design techniques
Application-specific extensions
Run entire user processes in kernel address space
Typed inter-process communication
First-class HLL virtual machines
Well-defined multithreading
Novel compiler+run-time solutions to high-level security problems …

7. OS on a Safe Runtime: So Far
To date, all rely on a safe programming language:
Genera and others: LISP
SPIN: Modula 3
JavaOS, KaffeOS, … : Java
Singularity: C# …
These projects led to many novel OS design techniques.
But what about today’s commodity kernels?
Linux, MacOS, BSD family, Windows, …

8. SVA: Secure Virtual Architecture
Designed to work with legacy kernels, e.g., Linux
Applications
Cached
translations
Profile info
Storage
OS, daemons, SSL, …
Kernel
Drivers
Virtual instruction set
SVA Execution
Engine
State HW
Control
Cache
SVA PrivOps API
Code
gen.
Optimize &
profile
Safety checking
SVA Virtual Machine
Processor
Native instruction set

9. Categories of Security Problems
Solved with a safe execution env.
Buffer overflows
Dangling pointers
Format string errors
Uninitialized pointers
New solutions enabled by a compiler-based VM
Excessive s/w privilege
Application data secrecy
Detecting rootkits
Information flow
Incorrect security checks
DOS by resource exhaustion
Dynamic code insertion
Race conditions …
Need application-level solutions
Configuration errors
Excessive admin privilege
Cross-site scripting
Network-level DOS
worms
Phishing attacks
Spam …

10. Some Progress Has Been Made
New comprehensive solutions for C in the last 5 years:
Memory safety, type safety, sound analysis
Cyclone [PLDI 02]
(+) No dangling pointers, intraprocedural compilation
(-) Some GC, substantial porting effort
CCured [PLDI 03]
(+) No dangling pointers
(-) Only garbage collection (GC); some porting effort
SAFECode [PLDI 06]
(+) Fully automatic, no GC (preserves full memory model of C)
(-) Makes dangling pointers harmless, does not eliminate them
(-) Sound analysis is more restricted
BUT:
None handle a complete
kernel; hypervisor; etc.
Porting effort precludes
use for large systems

11. The State of Commodity OSes
Total size
Kernel size
Windows XP
30M 5M
Red Hat Linux 7.1
20M
2.5M
Bill Worley and Mark Beckett, Eliminating Malware and Rootkits, Secure64 White Paper, July 2006.
Tanenbaum, Can We Make Operating Systems Reliable and Secure?, Computer Magazine, May 2006.

12. Surveys, Bugs Reports, Data, References
US-CERT: 5198 O.S. vulnerabilities reported in 2005 (A)
812 on Windows, 2328 on Unix/Linux, 2058 on multiple systems
Not even Java is safe: CERT #388289: buffer overflow in Sun JRE
Month of Kernel Bugs (Nov. 2006)
One new bug every day
Linux: 8, MacOS: 8, FreeBSD: 2, Solaris: 1, Windows: 1
Code injection: 13, DOS: 13, Memory corruption: 4
Month of Apple Bugs (Jan 2007): 23 new bugs in MacOS!
Cross-site Scripting
Overview: See
https (SSL) doesn’t help! Exploit scripts are embedded in legitimate HTML
Ordinary web server security policies (e.g., “local source”) don’t help!
Kernel Sizes and Bug Estimates (B,C)
Windows: 40M total, 5M kernel, 5,000-35,000 bugs
Linux: M total, 2.5M kernel, 2,500-17,500 bugs
Assumes 1-7 errors per 1K lines, as estimated by SEI for commercial software
Bill Worley and Mark Beckett, Eliminating Malware and Rootkits, Secure64 White Paper, July 2006
Tanenbaum, Can We Make Operating Systems Reliable and Secure?, Computer Magazine, Feb. 2006

13. SVA: Secure Virtual Architecture
Outline
SVA: Secure Virtual Architecture
SVA Overview
SVA Virtual Instruction Set and Compiler System
SVA Safety Guarantees
Future Work: Security Applications of SVA

14. E.g., This is how we ported Linux 2.4.22
Porting an OS to SVA
E.g., This is how we ported Linux
Safety-checking
C/C++ compiler
Existing
OS in
C/C++
OS in SVA
bytecode
Port kernel to PrivOps API
Port kernel allocators to checker API

15. Trusted Computing Base
Running an OS on SVA
Trusted Computing Base
SVA
bytecode
verifier
SVA to
native
codegen OS
in SVA
bytecode
Boot
Link
PrivOps
Library
Offline or online
Local  incremental
TCB components are far simpler than the safety checking compiler
Incremental  dynamic loading is easy

16. SVA Design Features
Comprehensive: All security-sensitive software can be protected
Kernel, OS extensions, device drivers, daemons, SSL
Incontrovertible: Security criteria cannot be bypassed
VM and OS can enforce safety policies at install time or load-time
Robust: Reduce trusted computing base (TCB):
Complex safety-checking compiler is outside the TCB
Extend to other security problems: Information flow, security automata, …
Whole-system ‡: Security analysis, transforms across programs
Can perform analysis, transformations across application/kernel boundary
Hardware assisted ‡: Can exploit upcoming hardware support
TPM (secure boot, secure PKI), IOMMU (secure DMA)
‡Capability exists but not yet used

17. SVA Prototype SVA PrivOps Library Safety Principles
C and i386 assembly code for Pentium 3
Compiled to native code library ahead of time
Safety Principles
Same safety guarantees for standalone programs and kernel
Untrusted safety-checking compiler, trusted type checker
Linux port to SVA
Like a port to a new architecture
Assembly code replaced with SVA operations
kmem_cache_alloc, kmalloc, vmalloc: port to checker API

18. SVA: The Secure Virtual Architecture Project
Outline
SVA: The Secure Virtual Architecture Project
SVA Overview
SVA Virtual Instruction Set
SVA Safety Principles
Future Security Applications of SVA

19. SVA Virtual Instruction Set
Unprivileged operations:
Derived from IR of the LLVM compiler system
Extended with security annotations:
Current: type system for safety properties
Privileged operations (PrivOps) API:
API for kernel-hardware interactions
Mechanisms, not policy

20. LLVM Provides Compiler Foundation
Framework for “lifelong compilation”
Compile-time, link-time, install-time, load-time, run-time, “idle”-time
IR: Compact, persistent, designed for effective optimization
Commercial-quality compiler infrastructure
JIT: Apple (MacOS 10.5), Adobe, Hue AS
Static back end: Cray, Ageia, Ascenium, Wind River
Silicon compilation: AutoESL
Unknown: Aerospace, Microchip Tech, Wind River
Also many academic, open-source, users; many contributors
Available at llvm.org .

21. LLVM IR = Core Unprivileged Operations
/* C Source Code */
int SumArray(int Array[],
int Num) {
int i, sum = 0;
for (i = 0; i < Num; ++i)
sum += Array[i];
return sum; }
;; SVA Code
int %SumArray(int* %Array, int %Num) {
bb1:
%cond = setgt int %Num, 0
br bool %cond, label %bb2, label %bb3
bb2:
%sum0 = phi int [%tmp10, %bb2], [0, %bb1]
%i0 = phi int [%inc, %bb2], [0, %bb1]
%tmp7 = cast int %i0 to long
%tmp8 = getelementptr int* %Array, long %tmp7
%tmp9 = load int* %tmp8
%tmp10 = add int %tmp9, %sum0
%inc = add int %i0, 1
%cond2 = setlt int %inc, %Num
br bool %cond2, label %bb2, label %bb3
bb3:
%sum1 = phi int [0, %bb1], [%tmp10,%bb2]
ret int %sum1 }
Architecture-neutral
Low-level operations
SSA representation
Typed
Mid-level type info
[CGO 2004, MICRO 2003]

22. SVA Privileged Operations API
Hardware Control
Syscall, interrupt, trap handlers
all OS entries are monitored
Page table entries
so are page mapping events
I/O operations
ditto
State Manipulation
Context-switching
save/restore native state
Signal delivery
Interrupt Context
exploit hardware for fast interrupts
[WIOSCA 2006]

23. Virtual and Native State
Virtual State
Virtual Registers
Program Counter
Privilege Mode
Interrupt Flag
Stack Pointer
MMU State
Native State
General Purpose / FP Registers
Stack slots
CPU Control Registers
MMU Registers

24. Challenges with Virtual State
Mapping native to virtual state
Changes frequently
Many virtual registers per function (but few are live!)
Insight: native-to-virtual mapping rarely needed
Only for debugging, portable checkpointing, etc.
Optimizations:
Save and restore (opaque) native state
Reconstruct virtual state only on demand

25. State Saving/Restoring Instructions
Solution: Expose existence of native state, but no details
Define native state based on correlation to virtual state
integer state
floating point (FP) state
Instructions
void llva_save_integer (void * buffer)
void llva_load_integer (void * buffer)
void llva_save_fp (void * buffer, bool save_always)
void llva_load_fp (void * buffer)

26. Hardware Control Registration functions I/O Memory Management
void llva_register_syscall (int number, int (*f)(…))
void llva_register_interrupt (int number, int (*f)(void * icontext))
void llva_register_exception (int number, int (*f)(void * icontext))
I/O
int llva_io_read (void * ioaddress)
void llva_io_write (void * ioaddress, int value)
Memory Management
void llva_load_pgtable (void * table)
void * llva_save_pgtable ()

27. Interrupted Program State
Execution Engine must save program state when entering the OS
Problem: Want to minimize the amount of state saved
No need to save FP state
Need to use low latency interrupt facilities
shadow registers (e.g. ARM)
register windows (e.g. SPARC)

28. Solution: Interrupt Context
Definition: Reserved space on kernel stack
Conceptually: saved Integer State of interrupted program
On interrupt, Execution Engine saves subset of Integer State on the kernel stack
Can leave state in registers if kernel does not overwrite it
Kernel can convert Interrupt Context to/from Integer State
Pointer to Interrupt Context passed to system call, interrupt, and trap handlers
Kernel Stack
GPR 1:
0xBEEF0000
ControlReg 2:
0x4EF23465
ControlReg 1:
0xC025E525
Processor
GPR N: 0x

29. Manipulating Interrupt Context
Push function frames
void llva_ipush_function (void * icontext, void (*f)(…), …)
Convert Interrupt Context Integer State
void llva_icontext_save (void * icontext, void * buffer)
void llva_icontext_load (void * icontext, void * buffer)

30. Example: Signal Handler Dispatch
Save program state with llva_icontext_save()
Save FP state with llva_save_fp()
Push new function frame on to program stack with llva_ipush_function()
User Space
Kernel Space
Stack
Stack
Function 1
Interrupt Context
Trap Handler
Signal
Handler
Heap
Processor
FP Registers
Other Registers

31. SVA: The Secure Virtual Architecture Project
Outline
SVA: The Secure Virtual Architecture Project
SVA Overview
SVA Virtual Instruction Set
SVA Safety Principles
Guarantees
Standalone programs
Complete kernel
Future Security Applications of SVA

32. Safety Guarantees
Strong Guarantees: Close to a safe language, but not equal
Memory safety: No uninitialized pointer uses, no array overflows
Type safety for a subset of objects
Control flow integrity: only follow compiler-predicted paths
Sound operational semantics  sound static analysis
Primary Weakness: By Design
Tolerate but not eliminate dangling pointer errors: avoid need for GC
Tolerate  Do not invalidate the above guarantees!
Option: detect all dangling pointer uses: low overhead for some programs
Practical
Completely automatic (for programs), no wrappers, no GC
Works for arbitrary C programs
Very low overhead for most C programs
Connect back to high-level security goals. Table to compare with safe languages.
[TECS 05, PLDI 06, ICSE 06, DSN 06]

33. Background - Points-to Graph
A static summary of memory objects and their connectivity
TU H,A P Q
field
P = malloc(2 * sizeof(int));
P[i] = ….
struct BigT *Q = (struct BigT *)P;
Q->x = …
struct List (TK) H
next val
head
Filed sensitive
Dotted arrow from q
Move this
struct List* head = makeList(20);
TK : Type Known
TU : Type Unknown
[PLDI 07]

34. Background: Automatic Pool Allocation
Partitions memory into pools matching points-to graph
Each node instance uses separate pool
Pool is destroyed if not accessible
List H
next val
head
List H
next val x y
Move this
Pool 1
Pool 2
[PLDI 05]

35. Automatic Pool Allocation: Key Properties
Unique, known pool for every pointer
Used by all clients
Memory partitioned into many pools
Used by array bounds detection
Run-time partitioning corresponds to alias analysis
Used by sound analysis
Many pools are type-homogeneous
Used by sound analysis , array bounds detection
Pool lifetimes bounded, and many pools short-lived
Used by sound analysis, dangling pointer detection

36. Example: Violating Alias Analysis
TU S,A P Q
field
P = malloc(2 * sizeof(int));
P[i] = ….
struct BigT *Q = (Struct BigT *)P;
Q->field8 = …
struct List (TK) H
next val
head
struct List* head = makeList(20);
Filed sensitive
Dotted arrow from q
Move this
TK : Type Known, TU : Type Unknown

37. Enforcing Alias Analysis
struct List (TK) H
next val
tmp
Check if tmp points to [an object in] appropriate node
Problem with normal allocators:
Memory objects are scattered in the heap
Each check at run-time would be extremely expensive

38. Memory pools  Points-to graph simplifies several safety properties
Key Insight
Memory pools  Points-to graph
simplifies several safety properties
Enforce sound pointer analysis
Efficient run-time checks on loads, stores
Check expected target pool for each pointer
Enforce type information for objects in TK nodes
Free!
Just need to solve minor details like array bounds, dangling pointers
Enable efficient object lookups
Actually that’s a whole ‘nother insight …
Move this

39. The Pool Bounds Check Pool is a list of pages (2^k)
Pool maintains a hash table of the start addresses of the pages
Poolcheck on a pointer p
Mask lower k bits of p, see if it is in the hash table
Alignment check for TK pools
Poolcheck : involves hash lookups

40. Insight 2 : Eliminate most checks for TK pools
3 sufficient properties
Type Known Pools
Type Unknown Pools
Typed accesses
Free
Pool / array bounds checks on all operations
Correct alignment
Free (with careful allocation)
Array bounds
Bounds checks for array references
§ Insight 3:
Type homogeneity + careful alignment + prevent reuse across pools
Insight 4:
Release memory from pool when pool is inaccessible

41. Insight 3 - Dangling Pointers
In TK pools, dangling pointers can be made harmless
Careful pool management
Ensure identical alignment for all objects
Do not release memory from one pool for use by another pool
Insight 4 - Enhancing memory reuse

42. Enforcing Safety for C Programs
Pool-level memory safety, partial type safety
Enforce points-to graph using Automatic Pool Allocation and run-time checks
Tolerates dangling pointer errors, array bounds violations
Restriction: Flow-insensitive, unification based
Finer-grain memory safety
Explicit array bounds checks, pointer initialization
Control-flow integrity
Enforce call graph via run-time checks (+ memory safety)
(Optional) Dangling pointer checks via page protection
Detect all dangling pointer uses
Very low overhead (0-15%) for programs that are not allocation-intensive

43. Formalization as a Type System
Int x y ρ
poolinit (ρ, int) PP {
int*ρ x,y;
int*ρ’ z;
x = malloc(4);
y = x;
free(x);
y = malloc(4);
poolinit(ρ’, int) PP’ {
poolinit (ρ, int) PP {
int*ρ x,y;
int*ρ’ z;
x = poolalloc(PP, 1); //allocate one element
y = x; //type checks
poolfree(PP,x)
y = poolalloc(PP,1); // malloc semantics different } ρ’
Int z
Color
Soundness theorem : core analyses are never invalidated.
Yields sound operational semantics.

44. Underlying Principles
Idea 1: Pool allocation  static points-to graph
Generate “pool checks” on (some) loads, stores
Enforces partial type safety, sound pointer analysis
Idea 2: Exploit type-homogeneous pools
Eliminate all checks except array-bounds checks
Dangling pointers harmless (with careful object alignment)
Idea 3: Efficient array bounds checks without metadata
Jones & Kelly: Maintain lookup tables of allocated objects
Use 1 table per pool: greatly reduces overhead
Avoid metadata on pointers

45. Evaluation: Run-time Overhead
1.0 ≡ no pool allocation + no SAFECode passes
Olden, Ptrdist, 3 system daemons [Full list in PLDI06]
No source changes necessary
Detected all attacks from Zitser’s suite.
Program
SAFECode bh 3%
bisort -4
em3d 91
treeadd
tsp 1
Yacr2 30 Ks 12
anagram 23
ftpd 4
fingerd 7
ghttpd
-10
Compare CCured (Olden, Ptrdist)
Much higher time overheads except em3d
Much higher space overheads (1x-4x)
Significant porting effort
Ccured overlay , green color change , no metadata, no GC

46. Related Work GC Solution Reported overheads Safety guarantees
Porting effort
Auto. memory management
Purify, Valgrind
10-100x
None -
SafeC 5x
Strong
Jones- Kelley
5-6x
SFI
Over 2x
Few
Yong
Some
SAFECode
Upto 1.3x
CCured
Upto 1.87x
Complete
Moderate GC
Cyclone
1x-2x
High
Regions +GC
Pure C
Modified C

47. Safety Challenges in the OS
Kernels have many custom pool allocators
Retain all existing kernel allocators
Define clean interface between allocator and checker: needs manual port
Compiler: Infer mapping of pools to points-to graph nodes
Kernels have many external entry points bringing in pointers
Can still use array indexing checks for externally-accessible pools
Kernel allocators not initialized early in boot sequence
Reserve memory to use for metadata for early objects
Extensive use of non-type-safe code
More aggressive static bounds-checking, run-time tuning

48. Verifying Security of SVA Bytecode
verifier
SVA to
native
codegen OS
in SVA
bytecode
Trusted
boot
Link
PrivOps
Library
SVA type system
Stack of “regions” (logical pools); every object registered in a pool
Region annotation on every pointer
Bytecode Verifier
Simple local, type checker
Inserts run-time checks
Local  easy to support dynamically loaded modules
This strategy can be used for any type system, e.g.,
security automata [Walker, POPL2000], information flow [Myers, POPL99]

49. SVA: The Secure Virtual Architecture Project
Outline
SVA: The Secure Virtual Architecture Project
SVA Overview
SVA Virtual Instruction Set
SVA Safety Principles
Future Security Applications of SVA
Application data secrecy
Reducing privilege
Monitoring kernel-mode rootkits

50. SVA: Secure Virtual Architecture
Compiler + privileged run-time + type checker
Applications
Cached
translations
Profile info
Storage
OS, daemons, SSL, …
Kernel
Drivers
Virtual instruction set
SVA Execution
Engine
State HW
Control
Cache
SVA PrivOps API
Code
gen.
Optimize &
profile
Safety checking
SVA Virtual Machine
Processor
Native instruction set

51. Some Security Applications of SVA
Minimizing Privilege for Root Programs
Compiler  Privilege bracketing + secure system call dispatch
Compiler  “Authenticated system call” policies
Incontrovertible: transformations at install-time or load-time
Protecting Application Data From OS
Higher privilege VM  Private physical memory
Higher privilege VM  Monitor all page mappings, I/O operations
Monitoring Kernel-mode Rootkits (with Sam King)
Higher privilege VM  monitoring of kernel cannot be subverted
Compiler  fine-grain monitoring of kernel operations

52. More Security Applications of SVA
Tracking information flow within and across programs
Extend the SVM type system
Efficient, secure IPC
Run communicating processes in the same address space
Enhancing benefits of hypervisors
Memory safety, safe paravirtualization, information flow

53. Future Work Performance tuning of SVA Whole-system recovery
Detected memory safety errors become DOS holes!
Need to define clean recovery semantics for entire OS
Framework for providing additional security
Memory safety for OS kernel
Install time privilege bracketing
Translator enforced system call policies
Protect application memory from kernel

54. Summary SVA: Secure Virtual Architecture Questions?
Solved with a safe execution env.
Buffer overflows
Dangling pointers
Format string errors
Uninitialized pointers
New solutions enabled by a compiler-based VM
Excessive s/w privilege
Application data secrecy
Detecting rootkits
Information flow
Incorrect security checks
DOS by resource exhaustion
Dynamic code insertion
Race conditions …
Safe environment for programs
Fully automatic
Safe environment for entire OS
Low porting effort
Higher-level security capabilities
Wide-open research area
Collaborations welcome!
Questions?

55. The LLVM Compiler System
Developer
site
C, C++
Compiler 1
LLVA
User
site
Linker +
IP Optimizer
Fortran
OCAML
• • •
MSIL
LLVA
Idle-time
Reoptimizer
Profile
info
C, C++
Compiler N
JIT
LLVA
Runtime
Optimizer
Path
profiles
LLVA
JVM
Simple JIT
MSIL
Simple JIT
LLVA
Static
Code Gen
[CGO 2004, MICRO 2003]
llvm.cs.uiuc.edu

56. SVA Linux Prototype Performance
Max. overhead: 2%
Max. overhead: 21%
Webstone: std. workload
Max. slowdown: 8%
Compiling with GCC -O2 (CPU-bound)
Slowdown: 1%