1 Recap

2 No. of Processors C.P.I Computational Power Improvement Multiprocessor Uniprocessor

3 Summing m Numbers Example: summing m numbers On a sequential computer we have, sum = a[0]; for (i=1;i<m;i++) { sum = sum + a[i]; } Would expect the running time be be roughly proportional to m.

4 Summing m Numbers in Parallel What if we have N processors, with each calculating the m/N numbers assigned to it? We must add these partial sums together to get the total sum.

5 Summing Multiprocessors P11P11 P13P13 P12P12 P21P21 P23P23 P22P22 P31P31 P33P33 P32P32 j i Suppose we have N processors. The algorithm is as follows: 1.Each processor finds the local sum of its m/N numbers 2.Each processor passes its local sum to another processor in a coordinated way 3.The global sum is finally in processor P11.

6 Shared Memory Multiprocessors P/C Cache NIC MB P/C Cache NIC MB Bus/Custom-Designed Network Shared Memory

7 MPP (Massively Parallel Processing) Distributed Memory Multiprocessors P/C LM NIC MB P/C LM NIC MB Custom-Designed Network MB : Memory BusNIC : Network Interface Circuitry

8 Cluster Commodity Network (Ethernet, ATM, Myrinet) MB P/C M NIC P/C M Bridge LD NIC IOB LD : Local DiskIOB : I/O Bus

9 Grid P/C SM NIC LD Hub/LAN Internet IOC P/C SM NIC LD Hub/LAN IOC

10 TOP500 architectures