1. MIMD Shared Memory Multiprocessors

2. MIMD -- Shared Memory u Each processor has a full CPU u Each processors runs its own code –can be the same program as other processors or different u All processors access the same memory –Same address space for all processors –UMA Uniform Memory Access »all memory accessible in same time for every processor –NUMA Non-Uniform Memory Access »memory is localized »each processor can access some memory faster than other

3. MIMD - SM - UMA PROCESSORSPROCESSORS MEMORYMEMORY MODULESMODULES CONNECTIONCONNECTION

4. Options for Connection -- UMA u Bus –Sequential, can be used for one message at a time u Switching Network –Can send many messages at once »depends on connection scheme –Crossbar »Maximal connections »expensive –Omega (also called Butterfly, Banyan) »several permutations of proc-mem possible

5. Bus u Needs smart local cache schemes to reduce bus traffic u Works for low number of processors u Depending on technology 20-50 processors overloads bus, performance degrades u Common on 4, 8 processor SMP servers

6. Bus Cache Processors Bus Memory

7. Crossbar switch u Every permutation of processor to memory can work u Expensive N*M switches where where N = number of processors, M = Number of memory modules

8. Processors MemoryMemory Switches Crossbar switch

9. Omega Network u Every Processor Connects to Every Memory u Many, but not all, permutations possible u An Extra stage adds redundancy and more permutations u Number of switches = (N/2) log N »For N processors, N memory modules u Number of stages = log N (determines latency)

10. Omega Network ProcessorsProcessors MemoryMemory

11. 000 001 010 011 100 101 110 111 000 001 010 011 100 101 110 111 Destination = 101

12. Omega Network -- A Permutation 000 001 010 011 100 101 110 111 000 001 010 011 100 101 110 111 Destination = 101

13. Omega Network with combining u Smart Switches –combine two requests with same destination –make memory accesses equivalent to serial sequence –split return values appropriately u Time trade-off u Used in NYU Ultra-computer –also in IBM RP3 experimental machine u Example: Fetch and Increment

14. Omega Network 000 001 010 011 100 101 110 111 000 001 010 011 100 101 110 111 Destination = 101

15. Options for Connection -- NUMA u Each Processor has a segment of memory closer than others –Could be several different levels of access u All Processors still use same address space u Omega network with wrap around –BBN Butterfly u Hierarchy of Rings (or other switches) –Kendall Square Research KSR-1 –SGI Origin series

16. Hierarchical Rings Directory Nodes Compute Node To higher level ring

17. Issues for MIMD Shared Memory u Memory Access –Can reads be simultaneous? –How to control multiple writes? u Synchronization mechanism needed –semaphores –monitors u Local caches need to be coordinated –cache coherency protocols