1. Edition 3, © Addison-Wesley 2001
CS Distributed Computing Systems Chapter 2: Architectural Models Chin-Chih Chang,
From Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design
Edition 3, © Addison-Wesley 2001

2. Introduction
Some of the problems that the designers of distributed systems face are:
Widely varying modes of use: The component parts of systems are subject to wide variations in workload.
Wide range of system environments: A distributed system must accommodate heterogeneous hardware, operating systems and networks.
Internal problems: Non-synchronized clocks, conflicting updates, many modes of hardware and software failure.
External threats: Attacks on data integrity and secrecy, denial of service.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

3. Introduction
Systems should be designed to function correctly in any circumstance. This gives rise to the problems of common properties and design issues in the form of descriptive models.
Two types of models:
Architectural model - client-server model and its variations, peer processes
Fundamental model - the interaction model, the failure model, the security model
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

4. Architectural models
The architecture of a system is the structure in terms of separately specified components.
Architecture models provide a high-level view of the distribution of functionality between components and the relationships between them to make system reliable, manageable, adaptable and cost-effective.
Architectural models determine the distribution of data and computational tasks amongst the physical nodes of the system and are helpful when evaluating the performance, reliability, scalability and other properties of distributed systems.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

5. Building a Architectural Model
It simplifies the functions of individual components of a distributed system around the concepts of process and objects. It can be achieved by classifying processes as server processes, client processes and peer processes (for cooperating and communicating).
It decides the placement of the components across a network of computers.
It decides the interrelationships between the components.
More dynamic systems can be built as variations on the client-server model.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

6. Software Layers (Layer Architecture)
The software architecture referred as services.
A server is a process that accepts requests from other process.
A distributed service can be provided by one or more servers
A platform is the lowest-level hardware and software layer that provide services to the layers above.
Intel x86/Windows, SPARC/SunOS, Intel x86/Solaris, Intel x86/Linux, PowerPC/MacOS
A middleware is a software layer that masks heterogeneity and provides a programming model.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

7. Figure 2.1 Software and hardware service layers in distributed systems
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

8. Software Layers (Layer Architecture)
Middleware (continued):
It is represented by processes or objects that interact with each other to implement communication and resource-sharing support for distributed applications.
It also provides essential support for distributed application development and is likely to offer greater support in the future.
Examples – Remote Procedure Call (RPC), Object-oriented middleware products and standards - Common Object Request Broker Architecture (CORBA), JAVA Remote Object Invocation (RMI), Distributed Component Object Model (DCOM), Reference Model for Open Distributed Processing (RM-ODP).
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

9. System Architectures (Process Placement)
It has major implications for the performance, reliability and security of systems.
A client-server model (Fig. 2.2) is most important and widely employed.
Client invocation (request) and server result (reply)
Servers may in turn be clients.
Services provided by multiple servers (Fig. 2.3):
Each web server managers its own set of resources.
Users employ a browser to access a resource at any one of the servers.
Replication is used to increase performance and availability and to improve fault tolerance.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

10. Figure 2.2 Clients invoke individual servers
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

11. Figure 2.3 A service provided by multiple servers
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

12. System Architectures (Process Placement)
Proxy servers and caches (Fig. 2.4):
Cache is a store of recently use data objects that is closer than the objects themselves.
Web browsers maintain a cache of recently visited web pages and other resources in the client's local file system.
Peer processes (Fig. 2.5) are processes interacting cooperatively as peers to perform a distributed activity or computation without any distinction between clients and servers.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

13. Figure 2.4 Web proxy server
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

14. Figure 2.5 A distributed application based on peer processes
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

15. Variations on the client-server model
Factors to consider on the client-server model:
The use of mobile code and mobile agents.
The need for low-cost computers (thin clients).
The requirement to add and remove mobile devices in a convenient manner.
Mobile code (Fig. 2.6) – Code is downloaded from server and runs locally.
Mobile agents - A running program travels from one computer to another in network.
Network Computers - OS and application software are downloaded from network and runs locally.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

16. Figure 2.6 Web applets
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

17. Variations on the client-server model
Thin clients (Fig. 2.7) – A software layer supports a graphic user interface to execute applications and software on a remote computers.
Mobile devices and Spontaneous networking (Fig. 2.8).
Key Features of Spontaneous Networking:
Easy connection to a local network
Easy integration with local services
Security problems from connectivity of mobile users:
Limited connectivity
Security and privacy
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

18. Figure 2.7 Thin clients and compute servers
Network computer or PC
network
Application
Thin
Process
Client
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

19. Figure 2.8 Spontaneous networking in a hotel
Internet
gateway
PDA
service
Music
Discovery
Alarm
Camera
Guests
devices
Laptop
TV/PC
Hotel wireless
network
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

20. Features of Spontaneous Networking
The purpose of a discovery service is to accept and store details of services that are available on the network and to respond to queries from clients.
A discovery service offers two interfaces:
A registration service accepts registration requests from servers and records the details in the discovery service’s database.
A lookup service accepts queries concerning available services and searches its database for registered services that match the queries.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

21. Interface and Objects
The set of functions available for invocation in a process is specified by one or more interface definitions.
Each server process is seen as a single entity with a fixed interface defining the functions that can be invoked in it.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

22. Design Issues Performance issues:
limited processing and communication capacities of computers and networks
Responsiveness, Throughput, Balancing computational loads
Quality of service - Reliability, Security, Performance, Adaptability
Use of caching and replication - Web-caching protocol
Dependability issues - Correctness, Security, Fault tolerance
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

23. Fundamental models
Fundamental models are concerned with a more formal description of the properties that are common in all the architectural models.
What should be captured in a model?
Interaction: How do components interact and coordinate to solve a problem?
Failure: What are the potential failures and how do they affect the results?
Security: What are the system's vulnerabilities and how should they be handled?
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

24. Interaction model
Distributed systems are composed of many processes, interacting in complex way:
Multiple server processes may cooperate with one another to provide a service. For example, Domain Name Service.
A set of peer processes may cooperate with one another to achieve a common goal. For example, a voice conferencing system.
Their behavior and state can be described by distributed algorithm - steps to be taken by each processes, including the transmission of messages between them.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

25. Interaction models
Processes interact through message passing to solve a problem.
Factors affect interacting processes:
Performance of the communication channels
Synchronization of the clocks
Communication performance:
Latency – delay between the start of a message’s transmission from one process and the beginning of its receipt by another
Bandwidth – total amount of information transmitted in a given time
Jitter – variation in time taken to deliver a series of messages.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

26. Clocks and event ordering
Messages are often timestamped to indicate ordering.
Each computer has at least one local clock.
Timestamps are often based on local clock values.
Local clocks aren't synchronized.
The term clock drift rate refers to the relative amount that a computer clock differs from a perfect reference clock.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

27. Clocks and event ordering
Some approaches to correct the times on computer clocks:
Get time readings from GPS
GPS can send timing messages to other computers in the network.
Two variants of the interaction model are defined:
Synchronous distributed system
Asynchronous distributed system
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

28. Synchronous Distributed System
The bounds are defined:
Bounded time for each processing step
Bounded time to transmit each message
Clock drift for each process in a known bound
Modelling an algorithm as a synchronous system may be useful for simulating the real system.
Synchronous distributed system can be built based on if there is sufficient processor cycles and network capacity.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

29. Asynchronous Distributed System
No bounds on:
Time for each processing step
Time to transmit each message
Clock drift for each process.
This exactly models the Internet.
Actual distributed systems are very often asynchronous because of the need for processes to share the processors and for communication channels to share the network.
Some design problems can be solved when some time constraints are used. For example, multimedia.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

30. Event ordering
When does an event (sending or receiving a message) from one process occur relative to an event from another process?
A mailing list contains users X, Y, Z and A:
X sends an to the list with subject Meeting: (m1)
Y and Z reply by sending each sending a message to the list with subject Re: Meeting (m 2 and m 3 respectively)
Example: What are the possible orderings of messages in the following scenario?
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

31. Figure 2.9 Real-time ordering of events
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

32. Logical time
If the clocks are roughly synchronized, then these timestamps will often be in the correct order. But clocks cannot be synchronized perfectly across a distributed system.
Logical time: Provide proper ordering of events without reference to physical clocks (Lamport, 1978).
Example - In the scenario:
X sends m1 before Y sends m2
Y sends m2 before X receives m2
Y receives m1 before Y sends m2
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

33. Failure models
Processes and communication channels may fail. Failure models define the ways in which failure may occur in order to provide and understanding of the effects of failures.
Failure types:
Omission failures refer to cases when process or channel fails to do something.
Arbitrary failures or Byzantine failures usually refer to sabotage.
Timing failures are applicable in synchronous distributed systems where required time bounds are exceeded
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

34. Figure 2.10 Processes and channels
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

35. Failure models Omission failures Arbitrary failures (Fig 2.11)
Process omission failures: crash - halt
Communication omission failures does not transport a message (caused by lack of buffer space): send omission failure, receive omission failure, channel omission failure
Arbitrary failures (Fig 2.11)
the worst failure
It arbitrarily omits intended processing steps or takes unintended processing steps can not be detected
Timing failures (Fig 2.12) occur only in synchronous distributed systems (limits on execution time).
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

36. Figure 2.11 Omission and arbitrary failures
Class of failure
Affects
Description
Fail-stop
Process
Process halts and remains halted. Other processes may
detect this state.
Crash
not be able to detect this state.
Omission
Channel
A message inserted in an outgoing message buffer never
arrives at the other end’s incoming message buffer.
Send-omission
A process completes a
send,
but the message is not put
in its outgoing message buffer.
Receive-omission
A message is put in a process’s incoming message
buffer, but that process does not receive it.
Arbitrary
(Byzantine)
Process or
channel
Process/channel exhibits arbitrary behaviour: it may
send/transmit arbitrary messages at arbitrary times,
commit omissions; a process may stop or take an
incorrect step.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

37. Figure 2.12 Timing failures
Class of Failure
Affects
Description
Clock
Process
Process’s local clock exceeds the bounds on its
rate of drift from real time.
Performance
Process exceeds the bounds on the interval
between two steps.
Channel
A message’s transmission takes longer than the
stated bound.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

38. Failure Models Deal with failures: Masking failures:
Masking failures can be done with the knowledge of the failure characteristics.
A service masks a failure, either by hiding it altogether or by converting it into a more acceptable type of failure.
Masking can be done by replication.
A reliable communication service can mask some of those failures.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

39. Reliable communication
Validity – Any message in the outgoing message buffer is eventually delivered to the incoming message buffer
Integrity - The received message is identical to the sent message and is delivered exactly once.
The threats to integrity come from two independent sources:
A protocol may accept a message twice.
Malicious users may inject a fake message.
How does a communication service provide validity and integrity?
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

40. Security model
The architectural model provides the basis for our security model.
The security of distributed systems can be achieved by:
Protect processes and channels from corruption.
Protect resources (encapsulated by objects) from unauthorized access.
Processes encapsulate objects which may hold a user's private data. To support this, access rights are used for specifying the allowable operations on an object.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

41. Security model
A principal is an authority associated with each invocation or each result.
The server is responsible for verifying the identity of the principal behind each invocation and checking their rights.
It is required to protect processes and their interactions from attack.
An enemy is capable of sending any message to any process and reading or copying any message between a pair of processes, as shown in Fig
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

42. Figure 2.13 Objects and principals
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

43. m m’ p q Figure 2.14 The enemy Copy of The enemy Process
Communication channel
Copy of m
Process p q
The enemy m’
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

44. Threats Threats could do to processes or communication channels.
Threats to processes include server attack and client one.
An enemy can copy, alter or inject messages as they travel across the network.
Defeating security threats by use of secure channel:
Cryptography and shared secrets (encryption/ decryption) - Cryptography is based on encryption algorithms that use secret keys.
Authentication - Message includes an encrypted portion.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

45. Secure Channel A secure channel has the following properties:
Each of the processes knows reliably the identity of the principal on whose behalf the other process is executing.
A secure channel ensures the privacy and integrity (protection against tampering) of the data transmitted across it.
Each message includes a physical or logical time stamp to prevent messages from being replayed or reordered.
Virtual Private Network (VPN) and Transport Layer Security (TLS), the successor of Secure Sockets Layer (SSL) are two instances of secure channels.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

46. Figure 2.15 Secure channels
Principal B
Principal A
Process p
Secure channel
Process q
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

47. Other Threats
Denial of service – Making excessive and pointless invocations on services or message transmissions in a network result in overloading of physical resources.
Mobile code – For example, virus, worms, or Trojan horse.
The uses of security models need to be balanced against threats.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

48. Summary
Middleware (CORBA and Java RMI) provides essential support for distributed application development and is likely to offer greater support in the future.
The client-server architecture is predominant, but there are many variations, some of which differ markedly from the client-server model.
Three fundamental models do not encompass all of the design issues for distributed systems.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000

49. Summary
The classification of processes and communication channels' failures is useful for the analysis of failures of protocols. Timing failures occur only in synchronous systems. Most failures in distributed systems are benign (e.g. omission but not Byzantine failures). A service may mask the failures of the components from which it is constructed.
The security model discusses the possible threats to processes and communication channels. It introduces the concept of a secure channel, which is secure against those threats.
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn © Addison-Wesley Publishers 2000