1. Multiprocessors Andreas Klappenecker CPSC321 Computer Architecture

2. Idea Build powerful computers by connecting many smaller ones.

3. Multiprocessors + Good for timesharing + easy to realize - difficult to write good concurrent programs - hard to parallelize tasks - mapping to architecture can be difficult

4. Questions How do parallel processors share data? — single address space — message passing How do parallel processors coordinate? — synchronization (locks, semaphores) — built into send / receive primitives — operating system protocols How are they implemented? — connected by a single bus — connected by a network

5. Shared Memory Multiprocessors Problems???Symmetric multiprocessor (SMP)

6. Distributed Memory Multiprocessors Distributed shared-memory multiprocessor Message passing multiprocessor

7. Multiprocessors Global MemoryDistributed memory Common Address Space Symmetric Multiprocessor Distributed shared- memory multiprocessor Distributed Address Space does not existMessage passing multiprocessor

8. Connection Network Static Network fixed connections between nodes Dynamic Network packet switching (packets routed from sender to recipient) circuit switching (connection between nodes can be established by crossbar or switching network)

9. Static Connection Networks

10. 0 0 1 Circuit Switching: Delta Networks Route from any input x to output y by selecting links determined by successive d-ary digits of y’s label. This process is reversible; we can route from output y back to x by following the links determined by successive digits of x’s label. This self-routing property allows for simple hardware-based routing of cells. 1101 0101 0 11 1 1 x = x k -1... x 0 y = y k -1... y 0 y0y0 x k -1 y1y1 x k -2 y k-1 y k-2... x k- 1 y 1 y k -3 x2x2 y k-1 y k-2 x k -1... x 3 y k-3 y k -2 x1x1 y k-1 x k -1... x 2 y k-2 x0x0 y k -1 x k -1... x 1 y k- 1

11. Network versus Bus

12. Performance / Unit Cost

13. Programming lock variables semaphores monitor …

14. Cache Coherency

15. Outlook Distributed Algorithms Distributed Systems Parallel Programming Parallel Compilers