1. Part six: Distributed systems
Natalia Shatokhina
Ashwini Mundya

2. Outline Reasons to build distributed system
Network OS vs Distributed OS
Data/process/communication migration in distributed OS
Communication Structure
Communication Protocols
Robustness
Design Issues
An Example: Networking

3. Distributed system Site A Site C server Network Resources client
Site B

4. Why distributed systems?
4 major reasons for building distributed systems:
Resource Sharing
Computational speedup
Reliability
Communication

5. Reason 1: Resource sharing
User on one site may be able to use the resources available at another site
Using a printer at remote site
Access a file
Access a database

6. Reason 2: Computational Speedup
If particular computation can be split into subcomputations that can run in parallel, the distributed system allows to distribute computations among various sites
Speedup can be achieved when running computations in parallel
Speedup= “serial computation time”/”parallel computation time”
In addition, load sharing helps to achieve speedup: move the jobs from the site overloaded with jobs to other with lighter load
Automated load sharing is not yet common

7. Reason 3: Reliability
If one site fails in distributed system remaining sites can continue to operate and get the job done
The function of failed site can be taken over by another site
To provide reliability:
The system must ensure the correct transfer of function
Failure of a site must be detected by the system and the services of that site should not be longer used
Mechanisms must be available to integrate the recovered site back to the system

8. Reason 4: Communication
Sites can communicate over a network by sending messages to one another.
The advantage of distributed system is that the messages can be carried over a great distance (two geographically distant sites). That provides for efficient collaboration.

9. Types of network-based operating systems
Network operating systems
Distributed operating systems
What is the difference?

10. Network operating systems (NOS)
Definitions:
From book: A network operating system provides an environment in which users, who are aware of the multiplicity of machines, can access remote resources by either logging in to the appropriate machine or transferring data from the remote machine to their own machines.
From wikipedia: System designed to allow shared file and printer access among computers in a network, typically a local area network (LAN), a private network or Intranet.

11. Hardware architectures (source: http://www. cs. jhu
Parallel architecture
Multiprocessors
Tightly coupled
Shared memory
cache
CPU
Memory
Distributed architecture
Multicomputers
Loosely coupled
Private memory
Autonomous
CPU
Memory
Picture that explains architecture
Types dynamic and static

12. Network OS vs Distributed OS (source: http://www. cs. jhu
Network OS: Contains N copies of operating systems, communication between machines is via shared files.
Distributed OS: Contains N copies of operating systems, communication between nodes is via messages over a network . These messages pass the necessary parameters for the task and on completion messages return the results. Its as if the computers each other with a request and an answer.

13. Network Operating systems
Remote login:
Network OS (for example Unix) provides an Internet service that allows the user to log in remotely
telnet or ssh is used
After logging in user can compute on remote machine just as any local user.
Ssh is used to establish encrypted connection
Remote file transfer (ftp):
Protocol allows to transfer a file from remote machine to local computer
User must know the commands and the exact location of the file he wants
Anonymous method provides access to files in directory tree “anonymous” to anyone
Since ftp is not secure, scp is currently used for file transfer
Shared file access should not be confused with file transfer using the file transfer protocol (FTP)
Shared file access is normally considered as a local area network (LAN) service, while FTP is an Internet service.
Shared file access is transparent to the user, as if it was a resource in the local file system, and supports a multi-user environment.

14. Distributed OS characteristics
In a distributed OS users access remote resources in the same way they access local resources
Data and process migration is under control of OS
Abbreviated as NOS, an operating system that includes special functions for connecting computers and devices into a local-area network (LAN). Some operating systems, such as UNIX and the Mac OS, have networking functions built in. The term network operating system, however, is generally reserved for software that enhances a basic operating system by adding networking features. Novell Netware, Artisoft's LANtastic, Microsoft Windows Server, and Windows NT are examples of an NOS.

15. Examples of distributed systems (source: Wikipedia)
Network OS- UNIX; Windows 2000 Server; Netware.
Distributed OS
Mosix2
implemented as an OS virtualization layer that provides to users and applications a single system image with the Linux run-time environment
Definition: single system image (SSI) cluster is a cluster of machines that appears to be one single system.
Plan 9 (developed for research purposes, planned as UNIX successor);
Amoeba (development is frozen). Project lead: Andrew Tannenbaum (MINIX author)

16. Examples of distributed systems
Clusters:
a group of linked computers often connected in LAN, working together closely thus in many respects forming a single computer
Implementations of cluster systems (commercial and free) exists for many OS
Categories of cluster systems:
high availability( Linux-HA, Lander Cluster),
load-balancing,
compute (Beowulf , OSCAR)
grid (Rocks).
Web

17. Rocks cluster architecture
Frontend node. Nodes of this type are exposed to the outside world. Many services (NFS, NIS, DHCP, NTP, MySQL, HTTP, ...) run on these nodes
In general, this requires a competent sysadmin. Frontend nodes are where users login in, submit jobs, compile code, etc. This node can also act as a router for other cluster nodes by using network address translation (NAT).
Compute nodes. These are workhorses and they are disposable. These nodes are not seen on the public Internet.
Ethernet Network
All compute nodes are connected with ethernet on the private network. This network is used for administration, monitoring, and basic file sharing.
Application Message Passing Network
All nodes can be connected with Gigabit-class networks and required switches. These are low-latency, high-bandwidth networks that enable high-performance message passing for parallel programs.
MPI is a language-independent communications protocol used to program parallel computers.
Cluster components
Front-end node
Compute node
Ethernet network
Application Message Passing Network (optional)

18. Data migration approaches in distributed OS
Transfer entire file (automated FTP system?). It was used in Andrew file system but found inefficient
Transfer only portions of the file that are necessary for immediate task. Another portion requires another transfer. Protocols that are using this method: NFS, newer versions of Andrew, SMB (running on top of TCP/IP or Microsoft NetBEUI)
The system may also perform data translation (if sites are using different character code representations)

19. Computation migration in distributed OS
Example: to obtain a summary of large files that reside on different sites.
Rather then copy files to one location and obtain a summary, it is more efficient to access the files at the sites where they reside and the return result to the initiating site
Process P wants to access file at site A. Access is initiated by RPC. Through UDP RPC executes a routine on remote site. P ivokes a pre-defined procedure at A, the results are returned to P
P sends a message to A. OS creates a new process Q that executes the task. When done Q sends the results back to P via the message system. This way P may execute concurrently with Q.
Either method can be used to access files at different sites
RPC might result in invocation of another RPC

20. Process migration in distributed system
Migration: the process is not get executed at the site where it was initiated
Reasons:
For load balancing
For speedup: if a process may be divided into subprocesses
Hardware preference: process is more suitable for execution on some specialized processor
Software preference: process requires software that is available only at particular site
Data access: the same as in computation migration - for big chunks of data
If load balancing is needed then a system itself can move processes
when the process must satisfy hardware -software preference then the user has to specify how process should migrate
Examples: Web server/database, Java Applets

21. Networks for distributed systems
Local area networks (LAN) :
Composed of processors distributed over small area (single building)
Emerged in 1970
More economical then large mainframe computer
Data sharing in a single enterprise
Communication link: high quality cable
Communication speed: 1Mbit/sec - 1 Gbit/sec
Typical LAN consist of computers, shared peripheral devices (printers, tapes) and one or more gateways that provide access to other networks
Ethernet scheme is used to construct LAN
Wireless LAN networks can be built with only one wireless router
Compared to Ethernet systems WiFi networks run at slower speed
The main difference between the two is the way in which they are geographically distributed

22. Networks for distributed systems
Wide area networks (WAN):
Composed of autonomous processors distributed over a large area (such as US territory)
Emerged in 1960s as academic research project
First WAN - Arpanet - a cluster of four computers
Communication link: phone lines, microwave links, satellite channels

23. Networks topology Configurations used for distributed system network:
Partially connected network
Fully connected network
Tree-structured network
Star network
Ring network
Criteria to compare the configurations:
Installation cost
Communication cost
Availability

24. Network communication structure
Communication network design issues:
Naming and name resolution
Routing strategies
Packet strategies
Connection strategies
Contention

25. Naming and name resolution
How could processes at different sites specify each other to communicate?
Process identifier (PID) is used for processes within computer system, but since networked hosts share no memory different approach is needed
Pair <host name, identifier> is used for process identification
Host name is alphanumeric
Host name is human readable, but computers use host-id instead. There is a mechanism to resolve host name into host-id - Domain name system
Hosts in Internet are logically addressed with with multipart names (IP addresses)

26. Naming and name resolution
Each component in IP name has name server
A system that needs to use the DNS is configured with the known addresses of the root servers
Example: bob.cs.brown.ed
How to get IP address for IP name bob.cs.brown.edu ?
Kernel of system A issues a request to the root server for edu domain asking for the address of the name server for brown.edu
The edu root server returns the address of the host on which brown.edu name server resides
The kernel on system A then queries the name server at this address and asks about cs.brown.edu
An address is returned ; and a request to that address returns host-id
Java provides API for mapping IP names to IP addresses
Generally, OS is responsible for accepting a message from its processes for <host name, identifier> and transferring it to appropriate host
Kernel of destination host transfers the message to the process named by identifier
Name server - a process on a system that accepts name and returns address of the name server responsible for that name

27. Naming and name resolution

28. Routing strategies
When a process from site A wants to communicate with process at site B how is a message sent?
Use physical path from A->B
If multiple paths exists then use Routing table
May include info about speed & cost of communication paths

29. Routing strategies comparison
Routing schemes
Fixed routing
Path is specified in advance (shortest path) and doesn’t change
Virtual routing
Path is fixed for the duration of one session. Examples: file transfer, remote login period etc
Dynamic routing
Path is chosen when message is sent. The message is sent over the link that is the least used at that particular time
Tradeoffs
Order of messages
Fixed & virtual scheme deliver messages in order
Environment
Dynamic scheme performs better in complicated environment
Usually the combination of these schemes is used - for example: within a site each host has a static route to the gateway, but the gataway uses dynamic routing to reach any host on the network.

30. Communication Structure (Cont.)‏

31. Packet Strategies
Variable length message divided into fixed- length message called packets/frames
Packets transferred to destination using 2 kinds of services :
Reliable service
Unreliable service

32. Connection Strategies
Circuit switching - A permanent physical link is established for the duration of the communication (i.e., telephone system)‏
Message switching - A temporary link is established for the duration of one message transfer (i.e., post-office mailing system)‏
Packet switching - Messages of variable length are divided into fixed-length packets which are sent to the destination
Each packet may take a different path through the network
The packets must be reassembled into messages as they arrive

33. Connection Strategies(Cont.)‏
Circuit switching requires setup time, but incurs less overhead for shipping each message, and may waste network bandwidth
Message and packet switching require less setup time, but incur more overhead per message

34. Contention
Several sites may want to transmit information over a link simultaneously. Techniques to avoid repeated collision include
CSMA/CD
Token Passing

35. Contention (Cont.)‏
CSMA/CD - Carrier sense with multiple access (CSMA); collision detection (CD)‏
A site determines whether another message is currently being transmitted over that link. If two or more sites begin transmitting at exactly the same time, then they will register a CD and will stop transmitting
When the system is very busy, many collisions may occur, and thus performance may be degraded
CSMA/CD is used successfully in the Ethernet system, the most common network system

36. Contention (Cont.)‏
Token passing - A unique message type, known as a token, continuously circulates in the system (usually a ring structure)‏
A site that wants to transmit information must wait until the token arrives
When the site completes its round of message passing, it retransmits the token
A token-passing scheme is used by some IBM and HP/Apollo systems

37. Communication Protocols

38. Communication Protocols
Simplify the design problem by partitioning the problem into multiple layers.
The 2 types of reference models
ISO's (International Standard Organization) OSI (Open System Interconnection) Model
TCP/IP Model

39. OSI Reference Model
Defines 7 network layers

40. Physical Layer
Handles the mechanical and electrical details of the physical transmission of a bit stream
This layer is implemented in the hardware of the networking device

41. Data Link Layer
This layer ensures that messages are delivered to the proper device on LAN using hardware address
Handles the frames, or fixed-length parts of packets, including any error detection and recovery that occurred in the physical layer

42. Network Layer
Responsible for tracking the location of devices in the network and determine the best way to move the data
Routers work at this level
Check if the packet belongs to that network else forward the packet to the destination address using routing table

43. Hop-to-hop delivery

44. Transport Layer Segments and reassembles the data into a data stream
Provides the end-to-end transport services – using TCP/UDP
Also called as Reliable Networking – acknowledgment, maintaining packet order (sequencing) and flow control

45. Session Layer
Responsible for setting up, managing and tearing down the sessions between presentation layer entities
Performs message synchronization

46. Presentation Layer Presents the data to the application layer
Translator – provides coding and conversion functions (Before transmit the data it has to be in standard format. Computers are configured to receive this data and convert onto native format).
By providing the translation service presentation layer ensures that data sent from the Application layer of one system is read by the Application layers of another one.

47. Presentation Layer (Cont.)‏

48. Application Layer Provides services to the end user
Examples – File Transfer, , Remote Login

49. OSI Reference Model

50. An OSI network message

51. Summary of layers

52. TCP/IP Protocol Layers

53. Robustness
Failure Detection
Reconfiguration

54. Robustness(Cont.)‏
Distributed system may suffer from various types of failure
Failure of a link
Failure of a site
Loss of message

55. Failure Detection Detecting hardware failure is difficult
To detect a link failure, a handshaking protocol can be used
The conclusion can be :
Other Site is down
The direct link between the two Site is down
The alternate link between the two Site is down
The message has been lost

56. Reconfiguration
When Site A determines a failure has occurred, it must reconfigure the system:
If the link from A to B has failed, this must be broadcast to every site in the system
If a site has failed, every other site must also be notified indicating that the services offered by the failed site are no longer available

57. Recovery from failure
Once the link is repaired both the systems must be notified (handshake)‏
Recover about the site has been notified to all other sites (failed site must also receive information from other sites)‏‏

58. Design Issues
Transparency - the distributed system should appear as a conventional, centralized system to the user
Fault tolerance – the distributed system should continue to function in the face of failure
Clusters – a collection of semi-autonomous machines that acts as a single system
Scalability – as demands increase, the system should easily accept the addition of new resources to accommodate the increased demand

59. Silicon Graphics Cluster

60. An Example: Networking
The transmission of a network packet between hosts on an Ethernet network
Every host has a unique IP address and a corresponding Ethernet (MAC) address
Communication requires both addresses
Domain Name Service (DNS) can be used to acquire IP addresses
Address Resolution Protocol (ARP) is used to map MAC addresses to IP addresses
If the hosts are on the same network, ARP can be used
If the hosts are on different networks, the sending host will send the packet to a router which routes the packet to the destination network

61. An Ethernet Packet

62. References
Operating System Concepts 8th Edition by Abraham Silberschatz, Peter B. Galvin, Greg Gagne
citeseerx.ist.psu.edu/viewdoc

63. Thank you !