# ICS 556 Parallel Algorithms Ebrahim Malalla Office: Bldg 22, Room 124-8.

## Presentation on theme: "ICS 556 Parallel Algorithms Ebrahim Malalla Office: Bldg 22, Room 124-8."— Presentation transcript:

ICS 556 Parallel Algorithms Ebrahim Malalla malalla@kfupm.edu.sa Office: Bldg 22, Room 124-8

Sequential Computers "…sequential computers are approaching a fundamental physical limit on their potential power. Such a limit is the speed of light…"

``o'' are uniprocessors, ``+''denotes modestly parallel computers with 4--16 processors, ``x'' denotes massively parallel computers with 100s or 1000s of processors

Parallelism Is and can be done easily in real life (nature) Is and can be done easily in real life (nature) E.g., ants and pees colony. E.g., ants and pees colony. So why not use it to solve human’s problems! So why not use it to solve human’s problems!

Parallel Computers Supercomputers (high performance computers) are needed to Speedup sequential algorithms Speedup sequential algorithms Solve large complex problems that are almost not practically solvable. Solve large complex problems that are almost not practically solvable.

Applications of Parallelism Weather forecasting Weather forecasting Simulations (DNA, biomedical, flight control) Simulations (DNA, biomedical, flight control) Scientific Computations Scientific Computations Fourier transform, wavelets Fourier transform, wavelets Applied linear algebra, matrix factorization and multiplication Applied linear algebra, matrix factorization and multiplication Image and signal processing Image and signal processing Numerical analysis Numerical analysis PDEs PDEs Huge Databases Huge Databases Real-time systems with large live stream inputs Real-time systems with large live stream inputs

Unfortunately.. Not every algorithm can be parallelized Not every algorithm can be parallelized “One woman can have a child in 9 months But 9 women can’t have a child in 1 month!”

Courses in Parallelism Parallel Architecture Parallel Architecture Parallel Programming Parallel Programming Parallel Algorithms Parallel Algorithms Parallel Algorithms for scientific computations Parallel Algorithms for scientific computations Concurrent Programming Concurrent Programming Distributed systems Distributed systems Distributed Algorithms Distributed Algorithms Cluster Computing Cluster Computing

Parallel Architecture Focuses on the design of hardwares Focuses on the design of hardwares E.g., processors, memory, communications and interconnections,...etc E.g., processors, memory, communications and interconnections,...etc

Parallel Programming Code parallel programs using parallel programming languages, e.g., PVM (Parallel Virtual Machine), MPI (Massage Passing Interface), OpenMP Code parallel programs using parallel programming languages, e.g., PVM (Parallel Virtual Machine), MPI (Massage Passing Interface), OpenMP Study: Parallel machine and programming models, Programming techniques (partitioning, pipelined computations, synchronous computations, load balancing and termination detection) Study: Parallel machine and programming models, Programming techniques (partitioning, pipelined computations, synchronous computations, load balancing and termination detection) Textbooks: Textbooks: Parallel Computing, by Grama, Gupta, Karypis, Kumar, Addison- Wesley, 2003. Parallel Computing, by Grama, Gupta, Karypis, Kumar, Addison- Wesley, 2003. Parallel Programming, by Wilkinson and Allen, Pearson-Prentice Hall Parallel Programming, by Wilkinson and Allen, Pearson-Prentice Hall Fundamentals of Parallel Processing by Jordan and Alaghband, Prentice Hall Fundamentals of Parallel Processing by Jordan and Alaghband, Prentice Hall

Parallel Algorithms Focuses on the design and analysis of parallel algorithms Focuses on the design and analysis of parallel algorithms Given a problem Given a problem partition it into sub-problems, partition it into sub-problems, Distribute these sub-problems to the processors Distribute these sub-problems to the processors Solve the communication problem Solve the communication problem Combine the solution Combine the solution

Parallel Computation Models Parallel algorithms are designed to run on a specific parallel computer Parallel algorithms are designed to run on a specific parallel computer But there many different practical models of them (unlike the sequential computer) But there many different practical models of them (unlike the sequential computer)

Shared Memory Model

Interconnection Network Model Network structure could be array, tree, hypercube, or mesh based design

Combinatorial Circuits Model

Other options Synchronous or asynchronous computation Synchronous or asynchronous computation SIMD MIDM SIMD MIDM........

Massively Parallel Computer

Top 10 Supercomputers (2007) ManufacturerComputerTFLOPSCountry 1IBM BlueGene/L eServer Blue Gene 280.6USA 2Sandia/Cray Red Storm Cray XT3 101.4USA 3IBM BGW eServer Blue Gene 91.29USA 4IBM ASC Purple eServer pSeries p575 75.76USA 5IBM MareNostrum JS21 Cluster, Myrinet 62.63Spain 6Dell Thunderbird PowerEdge 1850, IB 53.00USA Google Map of TOP 100 HPCs

Intel Promised Teraflops chip Intel has introduced a prototype for a prototype for A supercomputer Chip capable of teraflops Performance While consuming Very little power. It has 80 cores and consumes only 62 watts.

Approaches for studying Parallel Algorithms 1. Model Approach: divide the subject according to the parallel computation models and for each study different parallel algorithms design techniques and consider different application problems

Approaches for studying Parallel Algorithms 2. Problem Approach: Divide the subject according to the type of problems that can be solved by parallel algorithms like Sorting and searching Sorting and searching Scientific computation Scientific computation Graph problems Graph problems Computational geometry Computational geometry Simulations, graphics and vision Simulations, graphics and vision Biomedical modeling Biomedical modeling For each type we study design techniques for models of computations that are best suited.

Approaches for studying Parallel Algorithms 3. Models and Methods Approach: mix of the above two approaches. Models: Combinatorial circuits, interconnected networks, shared memory Models: Combinatorial circuits, interconnected networks, shared memory Design Methods: prefix computation, list ranking, divide and conquer, split and plan,.... etc Design Methods: prefix computation, list ranking, divide and conquer, split and plan,.... etc

Textbook Parallel Computation Parallel Computation By Selim Akl Supplementary Supplementary Lecture notes, current research papers

Assessment 4-5 assignments 4-5 assignments 2 exams 2 exams Presentation and report based on research papers Presentation and report based on research papers

Download ppt "ICS 556 Parallel Algorithms Ebrahim Malalla Office: Bldg 22, Room 124-8."

Similar presentations