Edition 3, © Addison-Wesley 2001 CS 843 - Distributed Computing Systems Chapter 2: Architectural Models Chin-Chih Chang, chang@cs.twsu.edu From Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edition 3, © Addison-Wesley 2001

Introduction Some of the problems that the designers of distributed systems face are: Widely varying modes of use: The component parts of systems are subject to wide variations in workload. Wide range of system environments: A distributed system must accommodate heterogeneous hardware, operating systems and networks. Internal problems: Non-synchronized clocks, conflicting updates, many modes of hardware and software failure. External threats: Attacks on data integrity and secrecy, denial of service. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Introduction Systems should be designed to function correctly in any circumstance. This gives rise to the problems of common properties and design issues in the form of descriptive models. Two types of models: Architectural model - client-server model and its variations, peer processes Fundamental model - the interaction model, the failure model, the security model Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Architectural models The architecture of a system is the structure in terms of separately specified components. Architecture models provide a high-level view of the distribution of functionality between components and the relationships between them to make system reliable, manageable, adaptable and cost-effective. Architectural models determine the distribution of data and computational tasks amongst the physical nodes of the system and are helpful when evaluating the performance, reliability, scalability and other properties of distributed systems. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Building a Architectural Model It simplifies the functions of individual components of a distributed system around the concepts of process and objects. It can be achieved by classifying processes as server processes, client processes and peer processes (for cooperating and communicating). It decides the placement of the components across a network of computers. It decides the interrelationships between the components. More dynamic systems can be built as variations on the client-server model. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Software Layers (Layer Architecture) The software architecture referred as services. A server is a process that accepts requests from other process. A distributed service can be provided by one or more servers A platform is the lowest-level hardware and software layer that provide services to the layers above. Intel x86/Windows, SPARC/SunOS, Intel x86/Solaris, Intel x86/Linux, PowerPC/MacOS A middleware is a software layer that masks heterogeneity and provides a programming model. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.1 Software and hardware service layers in distributed systems Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Software Layers (Layer Architecture) Middleware (continued): It is represented by processes or objects that interact with each other to implement communication and resource-sharing support for distributed applications. It also provides essential support for distributed application development and is likely to offer greater support in the future. Examples – Remote Procedure Call (RPC), Object-oriented middleware products and standards - Common Object Request Broker Architecture (CORBA), JAVA Remote Object Invocation (RMI), Distributed Component Object Model (DCOM), Reference Model for Open Distributed Processing (RM-ODP). Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

System Architectures (Process Placement) It has major implications for the performance, reliability and security of systems. A client-server model (Fig. 2.2) is most important and widely employed. Client invocation (request) and server result (reply) Servers may in turn be clients. Services provided by multiple servers (Fig. 2.3): Each web server managers its own set of resources. Users employ a browser to access a resource at any one of the servers. Replication is used to increase performance and availability and to improve fault tolerance. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.2 Clients invoke individual servers Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.3 A service provided by multiple servers Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

System Architectures (Process Placement) Proxy servers and caches (Fig. 2.4): Cache is a store of recently use data objects that is closer than the objects themselves. Web browsers maintain a cache of recently visited web pages and other resources in the client's local file system. Peer processes (Fig. 2.5) are processes interacting cooperatively as peers to perform a distributed activity or computation without any distinction between clients and servers. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.4 Web proxy server Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.5 A distributed application based on peer processes Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Variations on the client-server model Factors to consider on the client-server model: The use of mobile code and mobile agents. The need for low-cost computers (thin clients). The requirement to add and remove mobile devices in a convenient manner. Mobile code (Fig. 2.6) – Code is downloaded from server and runs locally. Mobile agents - A running program travels from one computer to another in network. Network Computers - OS and application software are downloaded from network and runs locally. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.6 Web applets Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Variations on the client-server model Thin clients (Fig. 2.7) – A software layer supports a graphic user interface to execute applications and software on a remote computers. Mobile devices and Spontaneous networking (Fig. 2.8). Key Features of Spontaneous Networking: Easy connection to a local network Easy integration with local services Security problems from connectivity of mobile users: Limited connectivity Security and privacy Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.7 Thin clients and compute servers Network computer or PC network Application Thin Process Client Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.8 Spontaneous networking in a hotel Internet gateway PDA service Music Discovery Alarm Camera Guests devices Laptop TV/PC Hotel wireless network Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Features of Spontaneous Networking The purpose of a discovery service is to accept and store details of services that are available on the network and to respond to queries from clients. A discovery service offers two interfaces: A registration service accepts registration requests from servers and records the details in the discovery service’s database. A lookup service accepts queries concerning available services and searches its database for registered services that match the queries. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Interface and Objects The set of functions available for invocation in a process is specified by one or more interface definitions. Each server process is seen as a single entity with a fixed interface defining the functions that can be invoked in it. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Design Issues Performance issues: limited processing and communication capacities of computers and networks Responsiveness, Throughput, Balancing computational loads Quality of service - Reliability, Security, Performance, Adaptability Use of caching and replication - Web-caching protocol Dependability issues - Correctness, Security, Fault tolerance Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Fundamental models Fundamental models are concerned with a more formal description of the properties that are common in all the architectural models. What should be captured in a model? Interaction: How do components interact and coordinate to solve a problem? Failure: What are the potential failures and how do they affect the results? Security: What are the system's vulnerabilities and how should they be handled? Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Interaction model Distributed systems are composed of many processes, interacting in complex way: Multiple server processes may cooperate with one another to provide a service. For example, Domain Name Service. A set of peer processes may cooperate with one another to achieve a common goal. For example, a voice conferencing system. Their behavior and state can be described by distributed algorithm - steps to be taken by each processes, including the transmission of messages between them. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Interaction models Processes interact through message passing to solve a problem. Factors affect interacting processes: Performance of the communication channels Synchronization of the clocks Communication performance: Latency – delay between the start of a message’s transmission from one process and the beginning of its receipt by another Bandwidth – total amount of information transmitted in a given time Jitter – variation in time taken to deliver a series of messages. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Clocks and event ordering Messages are often timestamped to indicate ordering. Each computer has at least one local clock. Timestamps are often based on local clock values. Local clocks aren't synchronized. The term clock drift rate refers to the relative amount that a computer clock differs from a perfect reference clock. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Clocks and event ordering Some approaches to correct the times on computer clocks: Get time readings from GPS GPS can send timing messages to other computers in the network. Two variants of the interaction model are defined: Synchronous distributed system Asynchronous distributed system Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Synchronous Distributed System The bounds are defined: Bounded time for each processing step Bounded time to transmit each message Clock drift for each process in a known bound Modelling an algorithm as a synchronous system may be useful for simulating the real system. Synchronous distributed system can be built based on if there is sufficient processor cycles and network capacity. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Asynchronous Distributed System No bounds on: Time for each processing step Time to transmit each message Clock drift for each process. This exactly models the Internet. Actual distributed systems are very often asynchronous because of the need for processes to share the processors and for communication channels to share the network. Some design problems can be solved when some time constraints are used. For example, multimedia. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Event ordering When does an event (sending or receiving a message) from one process occur relative to an event from another process? A mailing list contains users X, Y, Z and A: X sends an email to the list with subject Meeting: (m1) Y and Z reply by sending each sending a message to the list with subject Re: Meeting (m 2 and m 3 respectively) Example: What are the possible orderings of messages in the following scenario? Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.9 Real-time ordering of events Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Logical time If the clocks are roughly synchronized, then these timestamps will often be in the correct order. But clocks cannot be synchronized perfectly across a distributed system. Logical time: Provide proper ordering of events without reference to physical clocks (Lamport, 1978). Example - In the email scenario: X sends m1 before Y sends m2 Y sends m2 before X receives m2 Y receives m1 before Y sends m2 Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Failure models Processes and communication channels may fail. Failure models define the ways in which failure may occur in order to provide and understanding of the effects of failures. Failure types: Omission failures refer to cases when process or channel fails to do something. Arbitrary failures or Byzantine failures usually refer to sabotage. Timing failures are applicable in synchronous distributed systems where required time bounds are exceeded Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.10 Processes and channels Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Failure models Omission failures Arbitrary failures (Fig 2.11) Process omission failures: crash - halt Communication omission failures does not transport a message (caused by lack of buffer space): send omission failure, receive omission failure, channel omission failure Arbitrary failures (Fig 2.11) the worst failure It arbitrarily omits intended processing steps or takes unintended processing steps can not be detected Timing failures (Fig 2.12) occur only in synchronous distributed systems (limits on execution time). Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.11 Omission and arbitrary failures Class of failure Affects Description Fail-stop Process Process halts and remains halted. Other processes may detect this state. Crash not be able to detect this state. Omission Channel A message inserted in an outgoing message buffer never arrives at the other end’s incoming message buffer. Send-omission A process completes a send, but the message is not put in its outgoing message buffer. Receive-omission A message is put in a process’s incoming message buffer, but that process does not receive it. Arbitrary (Byzantine) Process or channel Process/channel exhibits arbitrary behaviour: it may send/transmit arbitrary messages at arbitrary times, commit omissions; a process may stop or take an incorrect step. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.12 Timing failures Class of Failure Affects Description Clock Process Process’s local clock exceeds the bounds on its rate of drift from real time. Performance Process exceeds the bounds on the interval between two steps. Channel A message’s transmission takes longer than the stated bound. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Failure Models Deal with failures: Masking failures: Masking failures can be done with the knowledge of the failure characteristics. A service masks a failure, either by hiding it altogether or by converting it into a more acceptable type of failure. Masking can be done by replication. A reliable communication service can mask some of those failures. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Reliable communication Validity – Any message in the outgoing message buffer is eventually delivered to the incoming message buffer Integrity - The received message is identical to the sent message and is delivered exactly once. The threats to integrity come from two independent sources: A protocol may accept a message twice. Malicious users may inject a fake message. How does a communication service provide validity and integrity? Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Security model The architectural model provides the basis for our security model. The security of distributed systems can be achieved by: Protect processes and channels from corruption. Protect resources (encapsulated by objects) from unauthorized access. Processes encapsulate objects which may hold a user's private data. To support this, access rights are used for specifying the allowable operations on an object. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Security model A principal is an authority associated with each invocation or each result. The server is responsible for verifying the identity of the principal behind each invocation and checking their rights. It is required to protect processes and their interactions from attack. An enemy is capable of sending any message to any process and reading or copying any message between a pair of processes, as shown in Fig. 2.14. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.13 Objects and principals Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

m m’ p q Figure 2.14 The enemy Copy of The enemy Process Communication channel Copy of m Process p q The enemy m’ Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Threats Threats could do to processes or communication channels. Threats to processes include server attack and client one. An enemy can copy, alter or inject messages as they travel across the network. Defeating security threats by use of secure channel: Cryptography and shared secrets (encryption/ decryption) - Cryptography is based on encryption algorithms that use secret keys. Authentication - Message includes an encrypted portion. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Secure Channel A secure channel has the following properties: Each of the processes knows reliably the identity of the principal on whose behalf the other process is executing. A secure channel ensures the privacy and integrity (protection against tampering) of the data transmitted across it. Each message includes a physical or logical time stamp to prevent messages from being replayed or reordered. Virtual Private Network (VPN) and Transport Layer Security (TLS), the successor of Secure Sockets Layer (SSL) are two instances of secure channels. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Figure 2.15 Secure channels Principal B Principal A Process p Secure channel Process q Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Other Threats Denial of service – Making excessive and pointless invocations on services or message transmissions in a network result in overloading of physical resources. Mobile code – For example, virus, worms, or Trojan horse. The uses of security models need to be balanced against threats. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Summary Middleware (CORBA and Java RMI) provides essential support for distributed application development and is likely to offer greater support in the future. The client-server architecture is predominant, but there are many variations, some of which differ markedly from the client-server model. Three fundamental models do not encompass all of the design issues for distributed systems. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000

Summary The classification of processes and communication channels' failures is useful for the analysis of failures of protocols. Timing failures occur only in synchronous systems. Most failures in distributed systems are benign (e.g. omission but not Byzantine failures). A service may mask the failures of the components from which it is constructed. The security model discusses the possible threats to processes and communication channels. It introduces the concept of a secure channel, which is secure against those threats. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 © Addison-Wesley Publishers 2000