MIMD Shared Memory Multiprocessors

MIMD -- Shared Memory u Each processor has a full CPU u Each processors runs its own code –can be the same program as other processors or different u All processors access the same memory –Same address space for all processors –UMA Uniform Memory Access »all memory accessible in same time for every processor –NUMA Non-Uniform Memory Access »memory is localized »each processor can access some memory faster than other

MIMD - SM - UMA PROCESSORSPROCESSORS MEMORYMEMORY MODULESMODULES CONNECTIONCONNECTION

Options for Connection -- UMA u Bus –Sequential, can be used for one message at a time u Switching Network –Can send many messages at once »depends on connection scheme –Crossbar »Maximal connections »expensive –Omega (also called Butterfly, Banyan) »several permutations of proc-mem possible

Bus u Needs smart local cache schemes to reduce bus traffic u Works for low number of processors u Depending on technology processors overloads bus, performance degrades u Common on 4, 8 processor SMP servers

Bus Cache Processors Bus Memory

Crossbar switch u Every permutation of processor to memory can work u Expensive N*M switches where where N = number of processors, M = Number of memory modules

Processors MemoryMemory Switches Crossbar switch

Omega Network u Every Processor Connects to Every Memory u Many, but not all, permutations possible u An Extra stage adds redundancy and more permutations u Number of switches = (N/2) log N »For N processors, N memory modules u Number of stages = log N (determines latency)

Omega Network ProcessorsProcessors MemoryMemory

Destination = 101

Omega Network -- A Permutation Destination = 101

Omega Network with combining u Smart Switches –combine two requests with same destination –make memory accesses equivalent to serial sequence –split return values appropriately u Time trade-off u Used in NYU Ultra-computer –also in IBM RP3 experimental machine u Example: Fetch and Increment

Omega Network Destination = 101

Options for Connection -- NUMA u Each Processor has a segment of memory closer than others –Could be several different levels of access u All Processors still use same address space u Omega network with wrap around –BBN Butterfly u Hierarchy of Rings (or other switches) –Kendall Square Research KSR-1 –SGI Origin series

Hierarchical Rings Directory Nodes Compute Node To higher level ring

Issues for MIMD Shared Memory u Memory Access –Can reads be simultaneous? –How to control multiple writes? u Synchronization mechanism needed –semaphores –monitors u Local caches need to be coordinated –cache coherency protocols