1. 1 Distributed Processing Chapter 1 : Introduction

2. STEMPusan National University STEM-PNU 2 Problem There are n nodes, each of which has a value. A node wants to know the maximum value among the n nodes. Centralized Approach: A server maintains the values of n nodes and each node reports its value to the server. Then the query node sends a message to ask the maximum value to the server, which will answer to the query. Distributed Approach: Each node communicates with its 6 nearest neighbor nodes to inform its value. Then the query node eventually finds the maximum value by exchanging information with its neighbor nodes.

3. STEMPusan National University STEM-PNU 3 Discussion Question 1: Find the algorithm for distributed approach. Question 2: Compare the performance In terms of the number of communications Question 3: Make a comparison table for the two approaches

4. STEMPusan National University STEM-PNU 4 Definition of a Distributed System Distributed system : 1) A collection of (scalability) 2) independent computers that (heterogeneity) 3) appears to its users as a single coherent system (transparency) Distributed System versus Parallel System Separated Operating System vs. Single Operating System Message Passing vs. Shared Memory

5. STEMPusan National University STEM-PNU 5 Why Distributed System ? Performance Incremental Growth (Scalability) 1 single mainframe of price W N small machines of price W/N Fault Tolerance 1 single mainframe : critical weak point Failure of a machine : replacement by other machines Geographical Distribution and Availability Flexible configuration e.g. 1 Disk server, 3 Computing servers, 1 Graphic server, etc. Geographical availability

6. STEMPusan National University STEM-PNU 6 Distributed System - Scalibility and Heterogeneity A distributed system organized as middleware.  Heterogeneity and Scalability 1.1

7. STEMPusan National University STEM-PNU 7 Distributed System - Transparency Different forms of transparency in a distributed system. TransparencyDescription AccessHide differences in data representation and how a resource is accessed LocationHide where a resource is located MigrationHide that a resource may move to another location RelocationHide that a resource may be moved to another location while in use ReplicationHide that a resource may be shared by several competitive users ConcurrencyHide that a resource may be shared by several competitive users FailureHide the failure and recovery of a resource PersistenceHide whether a (software) resource is in memory or on disk

8. STEMPusan National University STEM-PNU 8 Distributed System : Heterogeneity Server A Driver for A Driver for B Server B Server C Driver for C Application Program or Client Client has to be provided with one different driver for each server

9. STEMPusan National University STEM-PNU 9 Distributed System : Heterogeneity and Object-Oriented Approach Server A Server B Server C Application Program or Client Wrapping with predefined interface Predefined interface Encapsulation : Object-Oriented Approach

10. STEMPusan National University STEM-PNU 10 Hardware Concepts : Multiprocessor 1.6

11. STEMPusan National University STEM-PNU 11 Multiprocessors (1) A bus-based multiprocessor. 1.7

12. STEMPusan National University STEM-PNU 12 Multiprocessors (2) (a) A crossbar switch (b) An omega switching network 1.8

13. STEMPusan National University STEM-PNU 13 Homogeneous Multicomputer Systems (a) Grid (b) Hypercube

14. STEMPusan National University STEM-PNU 14 Software Concepts An overview of DOS (Distributed Operating Systems) NOS (Network Operating Systems) Middleware SystemDescriptionMain Goal DOS Tightly-coupled operating system for multi- processors and homogeneous multicomputers Hide and manage hardware resources NOS Loosely-coupled operating system for heterogeneous multicomputers (LAN and WAN) Offer local services to remote clients Middleware Additional layer atop of NOS implementing general- purpose services Provide distribution transparency

15. STEMPusan National University STEM-PNU 15 Issues in System Design Transparency Flexibility Reliability Performance Scalability Interoperability

16. STEMPusan National University STEM-PNU 16 Transparency Hiding physical details about Location Migration Duplication Relocation Concurrency Parallelism Location Access

17. STEMPusan National University STEM-PNU 17 Flexibility Should be easy to modify functionality and architecture To provide with Configurability, Avalability and Autonomy Micro-Kernel vs. Monolithic Kernel Monolithic Kernel : Provides all functionalities of OS. example. UNIX Micro-Kernel Minimal subset of OS + what users want Example Kernel Watch

18. STEMPusan National University STEM-PNU 18 Reliability Important Goal of Distributed System Reliability Security Fault-Tolerance Failure Probability P Should be P = P 1 ·P 2 ·P 3 … ·P n But often P = P 1 + P 2 + P 3 … + P n in reality

19. STEMPusan National University STEM-PNU 19 Performance and Scalability Improve performance by parallelism Throughput T Ideally should be T = T·n when n is the number of sites In reality T < T·n Due to some Bottleneck Number of sites Throughput ??

20. STEMPusan National University STEM-PNU 20 Granularity of Parallelism Unit of Task Fine-Granularity vs. Coarse Granularity Fine-Granularity Large number of small tasks Need a large amount of inter-task communication Not good for distributed system (good for Parallel system) Coarse-Granularity Small number of big tasks Only small amount of inter-task communication Good for distributed system

21. STEMPusan National University STEM-PNU 21 Interoperability Easy to collaborate with other systems in run-time Compatibility, Portability How to achieve Interoperability Well-Defined API set Standardization

22. STEMPusan National University STEM-PNU 22 Hardware Concepts : Multiprocessor 1.6

23. STEMPusan National University STEM-PNU 23 Multiprocessors (1) A bus-based multiprocessor. 1.7

24. STEMPusan National University STEM-PNU 24 Multiprocessors (2) (a) A crossbar switch (b) An omega switching network 1.8

25. STEMPusan National University STEM-PNU 25 Homogeneous Multicomputer Systems (a) Grid (b) Hypercube