1. The Duality of Memory and Communication in the Implementation of a Multiprocessor Operating System
Michael Young, Avadis Tevanian, Richard Rashid, David Golub, Jeffrey Eppinger, Jonathan Chew, William Bolosky, David Black, and Robert Baron
ACM Symposium on Operating System Principles, 1987
Presented By Rajesh Sudarsan
October 21, 2005

2. Agenda Introduction Key Ideas Monolithic vs Microkernel
Primitive abstractions
Implementation Details
Issues with External Memory Management
Benefits of Duality
Conclusion
CS 5204

3. Introduction Mach OS project started in 1985. Continued till 1994.
Successor to Accent OS developed at CMU
MACH NeXTSTEP OPENSTEP Mac OS X
Mach OS kernel – mainly designed to support multiprocessors.
Microkernel - a small, efficient kernel providing basic services such as process control and communication
CS 5204

4. Design goals
Object oriented interface with small number of basic system objects
Support for distributed and multiprocessing
Portability to different multiprocessor and uniprocessor architectures
Compatibility with BSD UNIX
Performance comparable to commercial UNIX distributions
CS 5204

5. Key ideas
Communication and virtual memory can play complementary roles in OS kernel
Increased flexibility in memory management
Support for multiprocessors
Improved performance
Easier task migration
Memory represented as abstract objects called memory objects
Single level store implementation
CS 5204

6. Key ideas (contd.) Virtual memory implementation using memory objects
External memory management – Structure for secondary storage management
CS 5204

7. Microkernel vs Monolithic
Monolithic kernel
Kernel interacts directly with the hardware
Kernel can be optimized for a particular hardware architecture
Kernel is not very portable
Microkernel
Kernel is very small
Most OS services are not part of the kernel and run at a layer above it
Very easily portable to other systems
CS 5204

8. Examples Microkernel Monolithic kernel
Amoeba, Minix, Chorus, Mach, GNU Hurd, NeXTSTEP, Mac OS X, Windows NT
Monolithic kernel
Traditional UNIX kernels, such as BSD, Linux, Solaris, Agnix
CS 5204

9. Architecture No direct data exchange between modules User application
Memory module
Process module
File module
Microkernel
Hardware
User mode
OS interface
Kernel mode
System Call
CS 5204
*source Distributed systems – Principles and Paradigms, Andrew Tannembaum, Maarten van Steen

10. Primitive abstractions in Mach OS
Four basic abstractions from Accent
Task, Threads, Ports, Messages
Port set
Fifth abstraction introduced in Mach
Memory Objects
Tasks and Threads – Execution control primitives
Ports and Messages - Interprocess communication
CS 5204

11. Interprocess communication
Two components of Mach IPC – ports and messages
Ports
Communication channel
Provides finite length queue
Protected bounded queue within the kernel
Messages
Fixed length header and variable size collection of typed data objects
CS 5204

12. IPC (contd.) One receiver, multiple sender
Tasks allocate ports to perform communication
Task can deallocate rights to a port
destination port
reply port
size/operation
pure typed data
port rights
out-of-line-data …
Message control
Memory cache object
Port
Format of Mach messages*
CS 5204
*source Operating System Concepts, Sixth Edition by Avi Silberschatz, Peter Baer, Galvin Greg Gagne

13. Memory Management
Virtual memory – level of abstraction between process memory requests and physical memory
Continuous address space
Demand Paging
Transparent relocation of running programs in memory
Page and Page frame
VM -> RAM – page global directory, page table, offset
CS 5204

14. Virtual Memory Management in microkernel
Each task has its own virtual address space
Restriction – virtual address space must be aligned with the system page boundaries
Supports read/write sharing of memory among tasks of common ancestry through inheritance
API provided for operation on VM
CS 5204

15. External Memory Management (EMM)
External memory management interface based on memory object
Memory object – abstract collection of data bytes with operations defined on them
Memory object represented by a port
Secondary storage objects available using message passing (data managers)
Mach kernel as cache manager for contents of memory object
CS 5204

16. External Memory Management (contd.)
Interface between kernel and data manager consists of 3 parts:
Calls made by application program to cause object to be mapped into its address space
Calls made by the kernel on the data manager
Calls made by the data manager on the Mach kernel to control use of its memory object
CS 5204

17. Fault Handling … Validate Hardware Map Victim Task External Pager Task
Thread
-----
----
Thread
-----
----
Thread
-----
----
Thread
Receive Request Find Data Send Reply (data) …
Kernel Context Check Validity Check Protection Page Lookup Do Copy-on-write Call pmap module Resume thread
Pmap Module
Validate Hardware Map
CS 5204

18. Minimal Filesystem Call from the application
Char * file_data; int i, file_size; extern float rand();
…..
fs_write_file(“filename”, file_data, file_size/2);
vm_deallocate(task_self(), file data, file_size);
return_t fs_read_file( name, data, size) { //Allocate memory object (a port) and accept request port_allocate(t…); port_enable(…);
…..
//perform file lookup. Find current file size
….. //Map the memory object into client address space vm_allocate_with_pager(…);
return (success); }
fs_read_file(“filename”, &file_data, file_size);
Call from the application
CS 5204

19. Minimal Filesystem (contd.)
void pager_data_request( memory_object, pager_request, offset, size, access)
{ //Allocate disk buffer vm_allocate (…);
//Lookup memory object and read disk data disk_read(…); //Return the data without any lock pager_data_povided(…);
//Deallocate disk buffer vm_deallocate(…); }
void port_death (request_port)
{ //find associated memory object with the port
lookup_by_request_port(…);
//Release resources port_deallocate(…);
vm_deallocate(…); }
Release filesystem resources after application deallocates its resources
File retrieval for the application
CS 5204

20. Consistent Network Shared Memory (Initialization)
Client A
Client B
A Request X
B Request X
Shared Memory Server
vm_allocate_with_pager
Client A resumed
Client B resumed
vm_allocate_with_pager
pager_init(X, request_A, name_A)
pager_init(X, request_B, name_B)
Mach Kernel B
Mach Kernel A
CS 5204

21. Consistent Network Shared Memory (Read)
Client A
Client B
Shared Memory Server
Client A resumed
Client A faults
pager_data_request(X, request_A, offset, page_size,VM_PROT_READ)
pager_data_request(X, request_B, offset, page_size,VM_PROT_READ)
Client B resumed
Client B faults
pager_data_provided( request_A, offset, page_size, VM_PROT_WRITE)
pager_data_provided( request_B, offset, page_size, VM_PROT_WRITE)
Mach Kernel B
Mach Kernel A
CS 5204

22. Consistent Network Shared Memory (Write)
Client A
Client B
Shared Memory Server
Client A resumed
Client A write faults
pager_data_unlock( X, request_A, offset, page_size, VM_PROT_WRITE)
pager_data_lock( request_A, offset, page_size, VM_PROT_NONE)
pager_flush_request(request_B, offset, page_size)
Mach Kernel B
Mach Kernel A
CS 5204

23. Implementation Details
Four basic data structures used to implement EMM
Address Map -Two level map
Top level – protection and inheritance information, link to second level
Second level – Map to memory object structures
Virtual Memory Object structures
Resident Memory Structures
Page replacement queues
Address map – Task address map is a directory that map each of many valid address ranges to a memory object and the offset within than object.
Other information stored are inheritance and protection information. Two level is used to account for sharing thru inheritance
VM object structures – An internal memory object structure is kept for each memory object used in the address map.
Components of this structure include the ports used to refer to the memory object, its size, number of address map references to the object, permission to
cache the memory object
Resident Memory structure - corresponds to the page of the physical memory. Records the memory object and offset into the object along with the access permissions by data manager
Pageout queues – active queue – contains pages currently in use
passive queue- pages ready to be pageout.
free queue – pages not caching any data.
CS 5204

24. External Memory Management Issues
Types of Memory failure – Data manager
doesn’t return data
fails to free flushed data
floods the cache
changes its own data
backs up its own data
Handling Memory failure
Timeout, notification, wait, abort
Default pager
Reserved memory pool
CS 5204

25. Benefits of duality
Multiprocessor support for UMA, NUMA, and NORMA architectures
The programmer has the option to choose between shared memory and message-based communication
Emulation of operating system environment such as UNIX achieved on Mach
Generic UNIX system calls can be implemented outside Mach kernel
Other features supported are transaction and database facilities, task migration, and AI knowledge bases
CS 5204

26. Conclusion Significant contribution to the operating system research
No experimental to support performance claim
More information could be presented in the implementation
Drawbacks
Frequent message passing may cause degradation in performance
Continuous monitoring of external services
CS 5204

27. Current Trend Pure microkernel architecture not common
Most OS kernels are hybrid models of monolithic and microkernel, e.g., Windows XP, Windows 2000
CS 5204

28. Questions?
CS 5204