1. Breaking Up is Hard to Do
Security and Functionality in a Commodity Hypervisor
Presented by Saad Arif

2. Virtualization in Cloud Computing
Cloud computing uses virtualization to lease small slices of large scale datacenter facilities to individual paying customers
Virtualization offers
Resource utilization
Administrative features
Support of existing software
Results in large scale hosting platforms

3. Companies in the Cloud (all these run in EC2 or Rackspace)

4. Virtualization in Cloud Computing
Control VM (Dom0)
Often a full OS
Similar privileges as hypervisor
Offers services to guest VMs

5. Hypervisors are Secure
Narrow
interface
Small
codebase
x86
x86
x86
Hypervisor
Xen: 280 KLOC (based on the current version)
Nova: 9 KLOC (microvisor) + 20 KLOC (VMM) [EuroSys’10]
SecVisor: 2 KLOC [SOSP’07]
Flicker: 250 LOC [EuroSys’08]

6. Example Attack Vectors
CERT vulnerability database for type-1 hypervisors
44 attacks
23 originated from within guest VMs
Based on attack vector,
14 showed device emulation layer vulnerabilities
2 in virtualized device layer
5 in management component
Only 2 hypervisor exploits
So 21 out 23 were attacks against service components in the control VM

7. “We are the 90%” Manage devices Create and destroy VMs
Control VM (Dom0)
IPC
User A’s VM
Management
Device Emulation
Platform
User B’s VM
Device Drivers
Manage devices
Create and destroy VMs
Arbitrarily access memory
Hypervisor

8. Isolate services into least-privileged service VMs
Exposure to Risk
Isolate services into least-privileged service VMs
Make sharing between components explicit
Contain scope of exploits in both space and time
Constraint:
Don’t reduce functionality, performance, or maintainability of the system

9. SPACE

10. Space Time Control VM IPC IPC IPC XenStore XenStore Platform Platform
Management
Management
User A’s VM
Management
Device Emulation
Device Emulation
Device Emulation
Qemu
PCI Config
PCI Config
Builder
Tools
Builder
Tools
Qemu
Platform
Device Drivers
Device Drivers
User B’s VM
Device Drivers
System Boot
System Boot
Network
Network
Block
Block
Hypervisor

11. Isolation Space Time Control VM IPC IPC IPC XenStore XenStore Platform
Management
Management
User A’s VM
Management
Device Emulation
Device Emulation
Device Emulation
Emulator
PCI Config
PCI Config
Builder
Tools
Builder
Tools
Emulator
Platform
Device Drivers
Device Drivers
User B’s VM
Device Drivers
System Boot
System Boot
Network
Network
Block
Block
Hypervisor

12. Space Time Isolation Control VM IPC IPC IPC XenStore XenStore Platform
Management
Management
User A’s VM
Management
Device Emulation
Device Emulation
Device Emulation
Qemu
PCI Config
PCI Config
Builder
Tools
Builder
Tools
Emulator
Platform
Device Drivers
Device Drivers
User B’s VM
Device Drivers
System Boot
System Boot
Network
Network
Block
Block
Hypervisor

13. Configurable Sharing User A’s Network User A’s Tools User A’s Block
User A’s VM
User A’s Tools
User A’s Block
User B’s Network
User B’s VM
User B’s Tools
User B’s Block

14. Configurable Sharing
User A’s VM
Network
Tools
Block
User B’s VM

15. Configurable Sharing User A’s Network User A’s Tools User A’s Block
User A’s VM
User A’s Tools
User A’s Block
User B’s Network
User B’s VM
User B’s Tools
User B’s Block

16. Space Time Isolation Configurable Sharing Control VM IPC IPC IPC
XenStore
XenStore
Platform
Platform
Management
Management
User A’s VM
Management
Device Emulation
Device Emulation
Device Emulation
Qemu
PCI Config
PCI Config
Builder
Tools
Builder
Tools
Emulator
Platform
Device Drivers
Device Drivers
User B’s VM
Device Drivers
System Boot
System Boot
Network
Network
Block
Block
Hypervisor

17. Auditing
User A’s VM
Which VMs were relying on the Block component while it was compromise?
Which VMs were relying on the Block component while it was compromise?
Network
User B’s VM
Block
Create
Block
Network
User C’s VM
VM B and VM C

18. Space Time Isolation Configurable Sharing Auditing Control VM IPC IPC
XenStore
XenStore
Platform
Platform
Management
Management
User A’s VM
Management
Device Emulation
Device Emulation
Device Emulation
Qemu
PCI Config
PCI Config
Builder
Tools
Builder
Tools
Emulator
Platform
Device Drivers
Device Drivers
User B’s VM
Device Drivers
System Boot
System Boot
Network
Network
Block
Block
Hypervisor

19. Time
TIME

20. Space Time Containment Configurable Sharing Auditing Control VM IPC
XenStore
XenStore
Platform
Platform
Management
Management
User A’s VM
Management
Device Emulation
Device Emulation
Device Emulation
Qemu
PCI Config
PCI Config
Builder
Tools
Builder
Tools
Emulator
Platform
Device Drivers
Device Drivers
User B’s VM
Device Drivers
System Boot
System Boot
Network
Network
Block
Block
Hypervisor

21. Disposable
PCI Config
Services
System Boot
Hypervisor

22. Space Time Isolation Configurable Sharing Auditing Disposable
Control VM
IPC
IPC
IPC
XenStore
XenStore
Platform
Platform
Management
Management
User A’s VM
Management
Device Emulation
Device Emulation
Device Emulation
Qemu
PCI Config
PCI Config
Builder
Tools
Builder
Tools
Emulator
Platform
Device Drivers
Device Drivers
User B’s VM
Device Drivers
System Boot
System Boot
Network
Network
Block
Block
Hypervisor

23. Snapshots
4-25 ms VM VM

24. Space Time Isolation Configurable Sharing Auditing Disposable
Timed Restarts
Control VM
IPC
IPC
IPC
XenStore
XenStore
Platform
Platform
Management
Management
User A’s VM
Management
Device Emulation
Device Emulation
Device Emulation
Qemu
PCI Config
PCI Config
Builder
Tools
Builder
Tools
Emulator
Platform
Device Drivers
Device Drivers
User B’s VM
Device Drivers
System Boot
System Boot
Network
Network
Block
Block
Hypervisor

25. Stateless VMs
Builder
Builder
Builder
User B’s VM
User A’s VM

26. Space Time Isolation Configurable Sharing Auditing Disposable
Timed Restarts
Stateless
Control VM
IPC
IPC
IPC
XenStore
XenStore
Platform
Platform
Management
Management
User A’s VM
Management
Device Emulation
Device Emulation
Device Emulation
Qemu
PCI Config
PCI Config
Builder
Tools
Builder
Tools
Emulator
Platform
Device Drivers
Device Drivers
User B’s VM
Device Drivers
System Boot
System Boot
Network
Network
Block
Block
Hypervisor

27. Space + Time

28. Space Space + Time Time Isolation Configurable Sharing Auditing
Disposable
Timed Restarts
Stateless
Control VM
IPC
IPC
IPC
XenStore
XenStore
Platform
Platform
Management
Management
User A’s VM
Management
Device Emulation
Device Emulation
Device Emulation
Qemu
PCI Config
PCI Config
Builder
Tools
Builder
Tools
Emulator
Platform
Device Drivers
Device Drivers
User B’s VM
Device Drivers
System Boot
System Boot
Network
Network
Block
Block
Hypervisor

29. I’ve enabled the network driver to map page 0xDEADBEEF
Composition
User A’s VM OK
XenStore
XenStore-State
XenStore-Logic
User B’s VM
B: Network can map 0xDEADBEEF
I’ve enabled 0xPWND
I’ve enabled the network driver to map page 0xDEADBEEF
Hypervisor

30. I’ve enabled the network driver to map page 0xDEADBEEF
Composition
User A’s VM OK
XenStore
XenStore-State
XenStore-Logic
User B’s VM
B: Network can map 0xDEADBEEF
A: Please shut me down
A: Please shut me down
I’ve enabled 0xPWND
I’ve enabled the network driver to map page 0xDEADBEEF
Hypervisor

31. I’ve enabled the network driver to map page 0xDEADBEEF
Composition
Monitor
User A’s VM OK
XenStore-State
XenStore
XenStore-Logic B
User B’s VM
B: Network can map 0xDEADBEEF
A: Please shut me down
A: Please shut me down
I’ve enabled the network driver to map page 0xDEADBEEF
I’ve enabled 0xPWND
Hypervisor

32. Space + Time Space Time Isolation Configurable Sharing Auditing
Composition
Time
Disposable
Timed Restarts
Stateless
Control VM
IPC
IPC
IPC
XenStore
XenStore
Platform
Platform
Management
Management
User A’s VM
Management
Device Emulation
Device Emulation
Device Emulation
Qemu
PCI Config
PCI Config
Builder
Tools
Builder
Tools
Emulator
Platform
Device Drivers
Device Drivers
User B’s VM
Device Drivers
System Boot
System Boot
Network
Network
Block
Block
Hypervisor

33. EVALUATION

34. Evaluation What do privileges look like now?
What is the impact on the security of the system?
What are the overheads?
What impact does isolation have on performance?
What impact do restarts have on performance?

35. Privileges Privilege System Boot PCI Config Builder Tools Block
Network
XenStore
Arbitrarily Access Memory X
Access and Virtualize PCI devices
Create VMs
Manage VMs
Manage Assigned Devices
Privilege
System Boot
PCI Config
Builder
Tools
Block
Network
XenStore
Arbitrarily Access Memory X
Access and Virtualize PCI devices
Create VMs
Manage VMs
Manage Assigned Devices
Privilege
System Boot
PCI Config
Builder
Tools
Block
Network
XenStore
Arbitrarily Access Memory X
Access and Virtualize PCI devices
Create VMs
Manage VMs
Manage Assigned Devices

36. Security
Of the 21 vulnerabilities against the control plane, we contain all 21
TCB is reduced from the control VM’s 7.5 million lines of code (Linux) to Builder’s 13,500 (on top of Xen)

37. Isolation Performance
Postmark performance
wget performance

38. Restart Performance
Kernel build performance

39. CONCLUSION

40. Summing it All Up Components of control VM a major source of risk
Xoar isolates components in space and time
Contains exploits
Provides explicit exposure to risk
Functionality, performance, and maintainability are not impacted