1. Module 3 Distributed Multiprocessor Architectures

2. Syllabi(Chapter 7 KHAB) (7.1.1 and 7.1.2) Loosely coupled and tightly coupled architectures Cluster computing as an application of loosely coupled architecture. Examples –CM* and Hadoop.

3. Some Basics…. Whenever working on projects, it seems as though several people coordinating together makes for a better solution then one person trying to piece things together on their own. This is similar to the concept of multiprocessing. Multiprocessing is n number of p processors working and operating concurrently. A multiprocessing system refers to a system configuration that contains more than one main control processor unit (CPU).

4. Why use a multiprocessing system? First of all, a multiprocessing system is used to increase overall system performance in work being accomplished, also referred to as throughput. By working together problems can be divided up among processor for faster completion, also called “divide and conqueror”. Another reason for using multiprocessing systems is to increase system availability.

5. Introduction Key attributes of “multiprocessors”:- – Single computer that includes multiple processors – Processors may communicate at various levels Message passing or shared memory Multiprocessor and Multicomputer systems – Multiple computer system consist of several autonomous computers which may or may not communicate with each other. – Multiprocessor system is controlled by single operating system which provides mechanism for interactions among processors Architectural models – Tightly coupled multiprocessor – Loosely coupled multiprocessor

6. Tightly coupled multiprocessor(Basics) Communicate via shared memory. Complete connectivity between processor and memory. This Connectivity accomplished by any interconnection network. Drawback-: Performance degradation due to memory conflicts

7. Tightly Coupled Architecture(Details) A tightly coupled multiprocessor system may be used in cases where speed is more of a concern. Models:- – Without private cache – With private cache

8. Architecture(Without Private Cache) This model consists of p number of processors, l memory modules, and d I/O channels. Everything is then connected using a P/M interconnection network (PMIN). The PMIN is a switch that can connect every processor to every memory module. A memory module can satisfy only one processors request in a given memory cycle. This conflict is arbitrated by the PMIN.

9. However, in this system the best way to prevent these types of conflicts is to make l equal to p (i.e. memory modules equal to the number of processors). Another way of eliminating this conflict is to use unmapped local memory (ULM)(Reserved Memory Area For Each Processor) By adding the ULM we are able to reduce the amount of traffic to the PMIN and thereby reducing conflicts to and from memory.

10. Tightly coupled multiprocessor contd. 10 Interrupt signal Interconnection network (ISIN) 0 p-1 Processors.. Input/Output Interconnection network (IOPIN) d-1 0.......... Processor Memory Interconnection network (PMIN) 0 l-1............... Input Output channels disks Shared Memory Modules Unmapped Local Memory Mapped Local Memory

11. Problem In this type of system architecture the memory references made by the processors is usually main memory. Memory reference common to all processor will cause conflicts. PMIN will surely resolve this conflicts but it will cause delay in operation,which increases instruction cycle time,which decreases throughput.. 

12. Solution Delay can be reduced by having cache for each processor which will hold memory reference for each processor. But cache coherance problem should be taken care of. Refer to diagram.

13. Tightly coupled multiprocessor contd. 13 Interrupt signal Interconnection network (ISIN) 0 p-1 Processors.. Input/Output Interconnection network (IOPIN) d-1 0.......... Processor Memory Interconnection network (PMIN) 0 l-1............... Input Output channels disks Shared Memory Modules Unmapped Local Memory Mapped Local Memory Private Caches

14. ISIN permits each processor to interupt to each processor. ISIN also used by failong processor to broadcast message. IOPIN permits processor to communicate with IO channel.

15. Tightly coupled multiprocessor contd. Processor types – Homogeneous, if all processors perform same function – Heterogeneous, if processors perform different functions Note: Two functionally same processor may differ along other parameters like I/O, memory size, etc, i.e. they are asymmetric 15

16. Loosely Coupled Architecture Each processor has its own set of I/O devices and memory where it accesses most of its instructions and data Computer Module: Processor, I/O interface and memory Local memory (LM) Input/Output (I/O) Channel and Arbiter Switch (CAS) Processor (P)

17. Loosely coupled multiprocessor contd. Inter-process communicate over different module happens by exchange of messages, using message transfer system (MTS) Distributed system, degree of coupling is loose Degree of memory conflicts is less 17 LMI/O CAS P Message Transfer System (MTS) LMI/O CAS P Computer Module 0 Computer Module N-1 …………..

18. Loosely coupled multiprocessor contd. Inter module communication – Channel arbiter and switch (CAS) – Arbiter decide when requests from two or more computer module collide in accessing a physical segment of MTS – Also responsible for delaying other request until servicing request is completed. 18

19. Message Transfer System (MTS) – Time based or shared memory – The latter case can be implemented with set of memory modules and processor-memory interconnection network or multiported main memory. – MTS determines the performance of multiprocessor system.

20. For LCS,that use single time shared bus,performance limited by,essage arrival rate on bus,message length and bus capacity. For LCS with shared memory,limiting factor is memory conflict problem imposed by processor memory interconnection network.

21. Cm* Architecture Project at Carnegie Melon University Now what is computer module? Computer module consists of processor,Slocal,local memory and I/O. Slocal similar to CAS in loosely coupled arch. P LM S I/O

22. Cluster of computer Modules Map Bus Intercluster Bus … P LM S I/O Cm 1 K MAP P LM S I/O Cm 10

23. Role Of Slocal Receives and interprets requests for access to P's local and foreign to local memory and the I / O S allows a local P to access external resources Cm To make interpretation of local and external applications software provide: A translation of local addresses

24. Address Translation

25. It uses 4 high order bits along with 1 pSW bit and then they access map table. Map Table determines whether memory is local or not. If memory non local control is given to Kmap via map bus. CM coonected to kmap via map bus. Kmap responsible for routing data between slocals.

26. Kmap Components AP Link Intercluster Bus 1 Map Bus K BUS Intercluster Bus 2 P MAP SERVICE RETURN RUN OUT SEND PORT 2 SEND PORT 1 Cm …

27. Request for non local memory arrives at kbus via map bus. Linc manages communication Between Kmap and another kmap. Pmap ->mapping processor which response to request between kbus and linc.

28. Kmap can simultanously handle 8 processor request. Pmap uses the concept of queue to handle request.

29. Service req signaled to kbus whenver req for non local memort ref. Such computer module called master Cm. Kmap fetches virtual address via map bus and allocates context for pmap. It places the virtual address in pmap run queue. Pmap performs virtual address to physical address translation.

30. Using physical address it can initiate memory acces in any cm. Kmap services the out req by sending physical memoryof memory req via map bus. When destination cm completes memory access it sends return signal to kmap.

31. Intracluster Communication Cm Master initiates a memory access nonlocal Master Cm virtual address issued by KBUS KBUS activates a context (creating specific data structure transition) that the PMAP RUN queue PMAP treats context and do address translation PMAP OUT queue a request for memory cycle Cm Slave of the current cluster Map Bus K BUS P MAP OUT K MAP Cm … RUN SlaveMaster 1 2 3 4 5

32. KBUS send physical address to Cm Slave by Map Bus There is the local slave Cm local memory access cycle. KBUS "allow" the result of memory access operation to be provided by Master Cm Cm Master takes the data, complete and continuous operation during execution Map Bus K BUS P MAP K MAP OUT Cm … RUN SlaveMaster 1 2 3 4 5 6 789

33. Intracluster communication Cm Map Bus K MAP Master Cm … Master 1 2 3 4 Intercluster Bus K MAP Slave Map Bus Cm … Slave 5 1.Cm Master sends a transfer request to KMAP Master 2.Master prepares KMAP message / request package encode intercluster 3.Intercluster message is transmitted on the bus intercluster routing algorithms 4.Slave KMAP decode incoming requests and sends to the cluster or localMemory cycle request is sent to Cm Slave

34. Cm Map Bus K MAP Master Cm … Master 1 2 3 4 8 9 K/U R/W Cm # Page Offset R/W Cm # Page Offset Cop Segment Offset Intercluster Bus K MAP Slave Map Bus Cm … Slave 5 6 7 10 5.Cm Slave Slave transmits the result to KMAP 6.Slave ready KMAP message intercluster (ie context reactivation) 7.KMAP Slave Master transmits the result to KMAP 8.KMAP Master receives and interprets the message received 9.The result is sent to the Master Cm