1. Modern Systems: Extensible KERNELS AND containers
Shannon Joyner
Modern Systems: Extensible KERNELS AND containers

2. Motivation Applications must conform to operating system interface
One operating system implementation is not ideal for everyone
Cannot optimize for a given application
OSs did not work well for database management systems 1
1 Operating System Support for Database Management. Michael Stonebraker. Communications of the ACM.

3. Motivation Operating systems are complex
Accessing resources require many system calls 1
This complexity in UNIX lead to Mach
Give application more control over system
1 Mach: A New Kernel Foundation for UNIX Development. Mike Acetta et al.

4. Exokernel: An Operating System Architecture for Application-Level Resource Management
Dawson R. Engler, M. Frans Kaashoek, and James O’Toole Jr.
M.I.T. Laboratory for Computer Science
Proc. 15th ACM Symposium on Operating Systems Principles (SOSP). Pages

5. Exokernel GOALS
Make small, fast kernel implementation by implementing simple primitives
Secure, fast way to access hardware resources
System abstractions can be implemented efficiently by application
Applications can have fast, specialized implementations
System abstractions (interface to hardware)

6. Exokernel LibOS Secure Bindings TLB Exokernel
Say how exokernel will try to address goals
Exokernel, Figure 1

7. Library Operating Systems
Implements OS besides interaction with hardware1
Customize based on application needs
Manages resource policies
Example LibOS customizations
Virtual Memory
Scheduling
Networking
Each instance of a LibOS can do a completely different thing.
The LibOS needs way to access resources such as memory and disk
Resource policy: How much CPU time does each process receive?
Exokernel, Figure 1 1

8. Secure bindings Used by LibOSes to access resources
Provides connection to resource
Decouples authorization from use of resource
Hardware can implement protection checks quickly
When an application wants access to a resource, the secure binding provides a connection.
If the application needs to access the resource a lot, making a connection every time, which can be expensive
Exokernel, Figure 1

9. SOFTWARE Translation look-ASIDE buffer (tlb)
Caches virtual memory to physical memory translations
Cache secure bindings in TLB to reduce number of binding connections
Improves performance
Software TLB used to cache bindings.
Software TLB
Exokernel, Figure 1

10. Exokernel Only handles resource sharing Rest left to application
Does not handle resource polices
Exokernel, Figure 1

11. Exokernel
Make small, fast kernel implementation by implementing simple primitives
Exokernel
Secure, fast way to access hardware resources
TLB + Secure Bindings
System abstractions can be implemented efficiently by application
LibOS + Exokernel
Applications can have fast, specialized implementations
LibOS
So for each goal, say which part of exokernel accomplishes task on next slide

12. Implementation Aegis: Exokernel Processor Time Slices Exceptions
Protected Control Transfer (PCT)
Dynamic Packet Filter (DPF)
Processor Time Slices: Scheduling done by iterating through slices
Exceptions: Sends hardware exceptions to applications
Protected Control Transfer: A way to switch to another processor
Asynchronous: Donate rest of the time slice to the next processor.
Synchronous: Donate the current time slice and any future time slice.
Dynamic Packet Filtering:
-Need to determine which application to send a packet to
-Application installs logic on the kernel at runtime
Exokernel, Dispatch Exceptions, Table 5

13. Implementation ExOS: LibOS Interprocess communication
Application Specific Safe Handlers
Virtual Memory
IPC abstractions
-Give up time slice if write buffer is full or read buffer is empty
ASH-application level message handlers: ExOS cannot secure map the network buffers to application space. Download code to kernel. Way to quickly perform checksums and respond to messages.
Virtual Memory: Provides application level VM with low overhead
Exokernel, Figure 2

14. Questions Who handles resource policies?
Is there a problem with how exokernels handle resource policies?
Why do we need secure bindings?
Application does. Poor Isolation, conflicting policies.
Protection / authorization of resources. Faster to do at kernel level, but kernel does not need to understand resources.
Not enough space in the hardware TLB. Can make software TLB larger.

15. Questions Who handles resource policies?
Is there a problem with how exokernels handle resource policies?
Why do we need secure bindings?
Application does. Poor Isolation, conflicting policies.
Protection / authorization of resources. Faster to do at kernel level, but kernel does not need to understand resources.
Not enough space in the hardware TLB. Can make software TLB larger.

16. Questions Who handles resource policies?
Application handles resource policies.
Application does. Poor Isolation, conflicting policies.
Protection / authorization of resources. Faster to do at kernel level, but kernel does not need to understand resources.
Not enough space in the hardware TLB. Can make software TLB larger.

17. Questions Who handles resource policies?
Is there a problem with how exokernels handle resource policies?
Why do we need secure bindings?
Application does. Poor Isolation, conflicting policies.
Protection / authorization of resources. Faster to do at kernel level, but kernel does not need to understand resources.
Not enough space in the hardware TLB. Can make software TLB larger.

18. Questions Who handles resource policies?
Is there a problem with how exokernels handle resource policies?
Poor isolation. Applications can have conflicting policies.
Why do we need secure bindings?
Why does Exokernel use a software TLB instead of the hardware TLB?
Application does. Poor Isolation, conflicting policies.
Protection / authorization of resources. Faster to do at kernel level, but kernel does not need to understand resources.
Not enough space in the hardware TLB. Can make software TLB larger.

19. Questions Who handles resource policies?
Is there a problem with how exokernels handle resource policies?
Why do we need secure bindings?
Application does. Poor Isolation, conflicting policies.
Protection / authorization of resources. Faster to do at kernel level, but kernel does not need to understand resources.
Not enough space in the hardware TLB. Can make software TLB larger.

20. Questions Who handles resource policies?
Is there a problem with how exokernels handle resource policies?
Why do we need secure bindings?
Protection / authorization of resources. Faster to do in kernel and kernel does not need to understand resources.
Why does Exokernel use a software TLB instead of the hardware TLB?
Application does. Poor Isolation, conflicting policies.
Protection / authorization of resources. Faster to do at kernel level, but kernel does not need to understand resources.
Not enough space in the hardware TLB. Can make software TLB larger.

21. Exokernel Takeaways
Can have a minimal kernel and have application customize operating system
Fast
Poor isolation
Each application implements own LibOS
No way to prevent systems from conflicting
Hardware compatibility
Need to change LibOS depending on hardware interface

22. Containers1 Hypervisor Hardware abstraction
Previous virtual machines ran on top of hypervisors
No isolation
What if one operating system is not good for the same application?
1 Container-based Operating System Virtualization: A Scalable, High performance Alternative to Hypervisors Stephen Soltesz, Herbert Potzi, Marc Fiuczynski, Andy Bavier, Larry Peterson. EuroSys ’07.

23. Containers Container Container Container MySQL Web Server MySQL
Grouping of processes
Provide isolation between groups
Containers cannot customize operating systems
Isn’t this similar to the problem exokernels tried to solve?
Operating system may not be good for all groupings.
Container
Container
Container
MySQL
Web Server
MySQL
Web Server
Web Server OS
Hypervisor

24. Unikernel: Library Operating Systems for the Cloud
Anil Madhavapeddy, Richard Mortier, Charalampos Rotsos, David Scott, Ralraj Singh, ThomasGazagnaire, Steven Smith, Steven Hand, and Jon Crowcroft
University of Cambridge, University of Nottingham, Citrix Systems Ltd, OCamlPro SAS
In Proceedings of the 18th International Conference on Architectural Support for Programming Languages and Operating Systems pg. 461–472.

25. Unikernel = EXOKERNEL + CONTAINERs
Run one application per virtual machine
One process per application
Everything compiled into a VM image
Do not compile unused code
Unikernel, Figure 1

26. Unikernel Run directly on top of standard hypervisor
Can run multiple unikernels on the same hypervisor
Unikernel
Unikernel
Unikernel
Application
Application
Application OS OS OS
Hypervisor

27. Mirage Produces unikernels Compiles OCaml code to Xen VM image
4 main components
Text + Data segment
Foreign Grants
Minor Heap
Major Heap
Chose OCaml because of flexibility (implement imperative, functional, and OOP). Brevity. High performance. Xen is implemented in OCaml, so integration is easy.
PV Boot - Way to initialize VM with a single virtual CPU and multiple Xen event channels.
Unikernel, Figure 2

28. Text and Data OCaml Runtime PVBoot Initializes VM
Initializes VM that application runs in
Unikernel, Figure 2

29. HEAP Minor Heap Short lived values in VM Fast Major Heap
Long lived values
Unikernel, Figure 2

30. Foreign Grants Used for VM communication Write data to a grant table
Exchange table between VM address spaces
Unikernel, Figure 2

31. ApACHE BENCHMARK
Mirage unikernel improvements result in better performance than having multiple cores
Unikernel, Figure 2

32. Exokernel versus Unikernel
All applications on same system
Poor isolation
Unikernel
Single application per system
Better isolation

35. protected control transfer
Implementation of interprocess communication
Put the messages in the receiver process’s context
Asynchronous: Rest of sender process's time slice goes to receiver
Synchronous: All future time slices go to receiver process
7x faster than best reported implementation
Exokernel, Table 6

36. Time to perform null procedure and system calls
Exokernel, Table 5

37. Time for IPC
Exokernel, Table 8

38. Time to perform vm operations
Exokernel, Table 10

39. Time to perform vm operations (two different page-tables)
Table 3, Exokernel

40. Roundtrip latency over ethernet
Exokernel, Table 11

41. EXOKERNEL: Separate Protection From Management
Tracking ownership of resources
Protect bindings and resource usage
Ex. Accessing memory not accessible to application
Revoking resource privileges
Go into more detail on what each bullet point means
Exokernel, Figure 1

42. EXOS Application-level stride Scheduler
Each process given a fixed proportion of resources
Implemented counter program
One process given 3x time, another 2x time, last one 1x time
Exokernel, Figure 3

43. TIme to perform lrpc extensions
Table 12

44. Time to classify TCP/ip Headers
Exokernel, Table 7

45. Time to perform 150x150 matrix multiplication
Exokernel, Table 9

46. Sample of aegis’s call interface
Table 2

47. Machine Configuration for exokernel experiments
Table 1

48. Sample of Aegis’s primitive operations
Table 3

49. Exokernel Provides interface to interact with hardware
Physical resources managed at application-level
Most of the system implemented using a Library Operating System
Separate resource protection from resource management

50. Secure bindings Used by LibOSes to access resources
Decouples authorization from use of resource
Performs authorization at bind time
Hardware can implement protection checks quickly

51. Library Operating System (Libos)
Use hardware interface
Implement rest of the operating system
Implementation can be specialized to meet needs of applications