1. Plan 9 from Bell Labs: Real World Distributed Operating System Case Study Plan 99Grid Bryan Kinney March 22, 2005 CPSC450/550 Distributed Operating Systems Department of Physics, Computer Science and Engineering Christopher Newport University

2. Plan 9 Case Study Introduction History Goals Definitions Features Using Plan 9 Applications Significance Summary References

3. Plan 9: Introduction Distributed System  A distributed system can be defined as a system in which its components are computers connected using a network that communicate and coordinate their actions only by passing messages [1].

4. Plan 9: Introduction Major Design Considerations of Distributed Systems [1]  Heterogeneity The system must be constructed from a variety of different networks, operating systems, computer hardware and programming languages.  Openness Distributed systems should be extensible and provide published interfaces.  Security Provide adequate protection of shared resources and keep sensitive information safe when transmitting messages.  Scalability Provide a system in which the cost to support adding a user is consistent in terms of the resources that must be added.  Failure handling Provide a system that can deal with the failure of any one of its components.  Concurrency Provide safe concurrent requests to shared resources.  Transparency Provide a single system image that allows the user to only be concerned with their application and not aspects of the system such as networks topography and CPUs.

5. Plan 9: Introduction Case Study How does Plan 9 compare to the definition of a Distributed System? Does Plan 9 take into account the major design considerations of: Heterogeneity, Openness, Security, Scalability, Failure handling, Concurrency, Transparency

6. Plan 9: History Bell Labs began development in late 1980’s 4 Releases Last Release was April of 2002  Updated June 2003 4 th Edition available from Bell Labs site  http://plan9.bell-labs.com/plan9dist/index.html

7. Plan 9: History Paradigm shift in mid 1980’s  From: Large centralized timeshared computers  To: Networks of smaller personal machines Typically UNIX ‘workstations’ Plan 9 wanted it both ways  Central Administration  Cost effective

8. Plan 9: Goals Build a time sharing system out of workstations “Build a UNIX out of a lot of little systems, not out of a lot of little UNIXs”

9. Plan 9: Goals Specialization of hardware  Different computers for different tasks  Small cheap machines in peoples offices Terminals providing access to shared resources  File servers, computing servers  File Servers and CPU Servers in Server Room

10. Plan 9: Goals Personal Workstation Systems Centralized Time shared Systems Mainframe Terminals CPU Servers File Servers High Speed Network Centrally located multiprocessor machines with large memories and high bandwidth point to point interconnects. Termina l Medium Speed Network Plan 9 Distributed Operating System

11. Plan 9: Definitions Plan 9:  Plan 9 is the name of the Distributed Operating System designed and implemented by researchers at Bell Lab in Murray Hill New Jersey. It is available for download from the Plan 9 website [7]. 9Grid:  9Grid is the name of a Plan 9 installation that provides Grid Style Computing. More information can be found at the 9 Grid website [8]. 8½:  The Plan 9 window system. It provides textual I/O and bitmap graphic services to both local and remote client programs by offering a multiplexed file service to those clients. It serves traditional UNIX files like /dev/tty as well as more unusual ones that provide access to the mouse and the raw screen. Bitmap graphics operations are provided by serving a file called/dev/bitblt that interprets client messages to perform raster operations [9].

12. Plan 9: Definitions Rc:  Rc is a command interpreter for Plan 9 that provides similar facilities to UNIXs Bourne shell, with some small additions and less idiosyncratic syntax [10]. 9P Protocol:  The 9P protocol is the Plan 9 file system protocol. It is structured as a set of transactions that send a request from a client to a (local or remote) server and return the result [2]. IL Protocol:  IL is a custom implemented network protocol to transport the remote procedure call messages 9P. It is a connection-based, lightweight transport protocol that carries datagrams encapsulated by IP. IL provides retransmission of lost messages and in-sequence delivery, but has no flow control and no blind retransmission [6].

13. Plan 9: Definitions Factotum:  Factotum is the central component of the security architecture. Factotum securely holds a copy of the user’s keys and negotiates authentication protocols, on behalf of the user, with secure services around the network [5]

14. Plan 9: Features Built upon 3 principles. 1. Resources as files Named and accessed like files in a hierarchical file system. 2. Standard protocol 9P, for accessing these resources. 3. Hierarchical file name space The disjoint hierarchies provided by different services are joined together into a single private hierarchical file name space.

15. Plan 9: Features 1. Resources as files  All resources in Plan 9 look like file systems.  File Oriented access Hierarchical name tree Accessible by name Access contents by read and write calls

16. 2.9P Protocol  Standard Protocol for accessing resources  Bell labs implemented IL Protocol for network transport of 9P messages Plan 9: Features

17. 3. Hierarchical File namespace  Every resource in the system, local or remote  User or process assembles a private view of the system by constructing a file name space that connects the resources.  Allows the user to access files that are local or remote in the same manner. When writing a program the user does not need to create code that handles for cases in which the file is not local. Those details are abstracted to the system.

18. Plan 9: Features Security  Factotum is the central component of the security architecture. Factotum securely holds a copy of the user’s keys and negotiates authentication protocols, on behalf of the user, with secure services around the network [5]

19. Plan 9: Features Security  Each box is a (typically) separate machine;  Ellipse is a process.  F X are factotum processes  P X are the pieces and proxies of a distributed program.  The authentication server is one of several repositories for users security information that factotum processes consult as required.  Secstore is a shared resource for storing private information such as keys; factotum consults it for the user during bootstrap

20. Plan 9: Features Capabilities for Parallel Programming  Kernel provides a simple process model and a few carefully designed system calls for synchronization and sharing.  Parallel programming language called Alef supports concurrent programming.  Although it is possible to write parallel programs in C, Alef is the parallel language of choice.  Alef uses a system call called rendezvous to provides a way for processes to synchronize.

21. Plan 9: Structure Main Parts  File servers  CPU servers  Terminals (user access points). Typically centrally located file servers and CPU servers CPU Servers File Servers High Speed Network Centrally located multiprocessor machines with large memories and high bandwidth point to point interconnects. Termina l Medium Speed Network Plan 9 Distributed Operating System

22. Plan 9: Using Plan 9 Example: Echo Server This Code implements a typical TCP listener. It announces itself listens for connections, and forks a new process for each. The new process echoes data on the connection until the remote end closes it. The "*" in the symbolic name means the announcement is valid for any addresses bound to the machine the program is run on.

23. Plan 9: Using Plan 9 Example: Echo Server 1. Announce() 2. Listen() 3. Accept() || Reject() 4. Process()…. 5. Close()…

24. Plan 9: Using Plan 9 Example: Echo Server 1. Announce() Returns open file descriptor for the ctl file of a connection and fills dir with the path of the protocol directory for the announcement. int announce(char *addr, char *dir) Addr is the symbolic name/address announced; if it does not contain a service, the announcement is for all services not explicitly announced.

25. Plan 9: Using Plan 9 Example: Echo Server 2. Listen() Listen returns an open file descriptor for the ctl file and fills ldir with the path of the protocol directory for the received connection. It is passed dir from the announcement. int listen(char *dir, char *ldir)

26. Plan 9: Using Plan 9 Example: Echo Server 3. Accept() and Reject() Accept and reject are called with the control file descriptor and ldir returned by listen. Some networks such as Datakit accept a reason for a rejection; networks such as IP ignore the third argument. int accept(int ctl, char *ldir) int reject(int ctl, char *ldir, char *reason)

27. Plan 9: Applications Developed in research environment Continue research news way to work with Distributed Operating Systems 9Grid  The Advanced Computing Cluster Research Lab, at Los Alamos National Laboratory uses Plan 9 for a secure grid environment:  “… utilize the distributed features of the Plan 9 operating system to create a tightly-coupled Grid environment in which running applications can cross the boundaries of the local cluster or institution and utilize resources around the globe or even further away. A distinguishing feature of 9grid is its security model. Plan 9 is a far more secure system, from the ground up, than any Unix system ever built. There is no need for add-ons such as firewalls to make Plan 9 Grid-capable. As users attach to nodes on the 9grid, their entire file system name space is visible from all the nodes which they are using -- and invisible to anyone else.”

28. Plan 9: Significance Fulfills definition of Distributed System 9P: Uses message passing to coordinate actions Provides for the major design considerations as outlined  Heterogeneity, Openness, Security, Scalability, Failure handling, Concurrency, Transparency

29. Plan 9: Summary The Plan 9 Distributed Operating System is a Distributed System which gives its users a single system image of the system of networks and hardware it manages. Plan 9 gives users the tools for conventional programming and parallel programming. Plan 9 is a distributed system built from the ground up incorporating distributed concepts and encouraging future research of concepts developed through using distributed computing environments.

30. Plan 9: References 1.Coulouris,George. Jean Dollimore, Tim Kindeberg, Distributed Systems Concepts and Design. Pearson Education Limited, 2001 2.Pike, Rob. Dave Presotto, Sean Dorward, Bob Flandrena, Ken Thompson, Howard Trickey, and Phil Winterbottom, Plan 9 from Bell Labs,http://plan9.bell-labs.com/sys/doc/index.html 3.Presotto, Dave. Phil Winterbottom. The Organization of Networks in Plan 9, http://plan9.bell-labs.com/sys/doc/index.html 4.Pike, Rob. Dave Presotto, Sean Dorward, Bob Flandrena, Ken Thompson, Howard Trickey, and Phil Winterbottom, The Use of Namespaces in Plan 9,http://plan9.bell-labs.com/sys/doc/index.html 5.Cox, Russ. Eric Grosse, Rob Pike, Dave Presotto, Sean Quinlan. Security in Plan 9. http://plan9.bell-labs.com/sys/doc/index.html 6.Presotto, Dave. Phil Winterbottom. The IL Protocol. http://plan9.bell- labs.com/sys/doc/index.html

31. Plan 9: References 7.Plan 9 Website, http://plan9.bell-labs.com/plan9dist/index.html 8.9Grid Website, http://plan9.bell-labs.com/9grid/index.html 9.Pike, Rob. 8½, The Plan 9 Window System. http://plan9.bell- labs.com/sys/doc/index.html 10.LDuff, Tom, Rc- The Plan 9 Shell. http://plan9.bell- labs.com/sys/doc/index.html 11.Advanced Computing Cluster Research Lab, Los Alamos National Laboratory. http://public.lanl.gov/cluster/projects/index.html 12.9Grid (LANL): http://www.9grid.net

32. Plan 9 from Bell Labs: Real World Distributed Operating System Case Study Plan 99Grid Bryan Kinney March 22, 2005 CPSC450/550 Distributed Operating Systems Department of Physics, Computer Science and Engineering Christopher Newport University