Multiprocessors Andreas Klappenecker CPSC321 Computer Architecture

Idea Build powerful computers by connecting many smaller ones.

Multiprocessors + Good for timesharing + easy to realize - difficult to write good concurrent programs - hard to parallelize tasks - mapping to architecture can be difficult

Questions How do parallel processors share data? — single address space — message passing How do parallel processors coordinate? — synchronization (locks, semaphores) — built into send / receive primitives — operating system protocols How are they implemented? — connected by a single bus — connected by a network

Shared Memory Multiprocessors Problems???Symmetric multiprocessor (SMP)

Distributed Memory Multiprocessors Distributed shared-memory multiprocessor Message passing multiprocessor

Multiprocessors Global MemoryDistributed memory Common Address Space Symmetric Multiprocessor Distributed shared- memory multiprocessor Distributed Address Space does not existMessage passing multiprocessor

Connection Network Static Network fixed connections between nodes Dynamic Network packet switching (packets routed from sender to recipient) circuit switching (connection between nodes can be established by crossbar or switching network)

Static Connection Networks

0 0 1 Circuit Switching: Delta Networks Route from any input x to output y by selecting links determined by successive d-ary digits of y’s label. This process is reversible; we can route from output y back to x by following the links determined by successive digits of x’s label. This self-routing property allows for simple hardware-based routing of cells x = x k x 0 y = y k y 0 y0y0 x k -1 y1y1 x k -2 y k-1 y k-2... x k- 1 y 1 y k -3 x2x2 y k-1 y k-2 x k x 3 y k-3 y k -2 x1x1 y k-1 x k x 2 y k-2 x0x0 y k -1 x k x 1 y k- 1

Network versus Bus

Performance / Unit Cost

Programming lock variables semaphores monitor …

Cache Coherency

Outlook Distributed Algorithms Distributed Systems Parallel Programming Parallel Compilers