Presentation is loading. Please wait.

Presentation is loading. Please wait.

Parallel Processing1 Parallel Processing (CS 676) Overview Jeremy R. Johnson.

Similar presentations

Presentation on theme: "Parallel Processing1 Parallel Processing (CS 676) Overview Jeremy R. Johnson."— Presentation transcript:

1 Parallel Processing1 Parallel Processing (CS 676) Overview Jeremy R. Johnson

2 Parallel Processing2 Goals Parallelism: To run large and difficult programs fast. Course: To become effective parallel programmers –“How to Write Parallel Programs” –“Parallelism will become, in the not too distant future, an essential part of every programmer’s repertoire” –“Coordination – a general phenomenon of which parallelism is one example – will become a basic and widespread phenomenon in CS” Why? –Some problems require extensive computing power to solve –The most powerful computer by definition is a parallel machine –Parallel computing is becoming ubiquitous –Distributed & networked computers with simultaneous users require coordination

3 Parallel Processing3 Top 500

4 Parallel Processing4 LINPACK Benchmark Solve a dense N  N system of linear equations, y = Ax, using Gaussian Elimination with partial pivoting –2/3N 3 + 2N 2 FLOPS High Performance LINPACK used to measure performance for TOP500 (introduced by Jack Dongarra)

5 Parallel Processing5 Example LU Decomposition Solve the following linear system Find LU decomposition A = PLU

6 Parallel Processing6 Big Machines Cray 2 DoE-Lawrence Livermore National Laboratory (1985) 3.9 gigaflops 8 processor vector machine Cray XMP/4 DoE, LANL,… (1983) 941 megaflops 4 processor vector machine

7 Parallel Processing7 Big Machines Cray Jaguar ORNL (2009) 1.75 petaflops 224,256 AMD Opteron cores Tianhe-1A NSC Tianjin, China (2010) 2.507 petaflops 14,336 Xeon X5670 processors 7,168 Nvidia Tesla M2050 GPUS

8 Parallel Processing8 Need for Parallelism

9 Parallel Processing9 Multicore Intel Core i7

10 Parallel Processing10 Multicore IBM Blue Gene/L 2004-2007 478.2 teraflops 65,536 "compute nodes” Cyclops64 80 gigaflops 80 cores @ 500 megahertz multiply-accumulate

11 Parallel Processing11 Multicore

12 Parallel Processing12 Multicore

13 Parallel Processing13 GPU Nvidia GTX 480 1.34 teraflops 480 SP (700 MHz) Fermi chip 3 billion transistors

14 Parallel Processing14 Google Server 2003: 15,000 servers ranging from 533 MHz Intel Celeron to dual 1.4 GHz Intel Pentium III 2005: 200,000 servers 2006: upwards of servers

15 Drexel Machines Tux 5 nodes –4 Quad-Core AMD Opteron 8378 processors (2.4 GHz) –32 GB RAM Draco 20 nodes –Dual Xeon Processor X5650 (2.66 GHz) –6 GTX 480 –72 GB RAM 4 nodes –6 C2070 GPUs Parallel Processing15

16 Parallel Processing16 Programming Challenge “But the primary challenge for an 80-core chip will be figuring out how to write software that can take advantage of all that horsepower.” Read more: processor/21001006_36158181.html?tag=mncol#ixzz1AHCK 1LEc

17 Parallel Processing17 Basic Idea One way to solve a problem fast is to break the problem into pieces, and arrange for all of the pieces to be solved simultaneously. The more pieces, the faster the job goes - upto a point where the pieces become too small to make the effort of breaking-up and distributing worth the bother. A “parallel program” is a program that uses the breaking up and handing-out approach to solve large or difficult problems.

Download ppt "Parallel Processing1 Parallel Processing (CS 676) Overview Jeremy R. Johnson."

Similar presentations

Ads by Google