Download presentation

Presentation is loading. Please wait.

Published byBenjamin Dean Modified about 1 year ago

1
PRAM (Parallel Random Access Machine) David Rodriguez-Velazquez Spring -09 CS-6260 Dr. Elise de Doncker

2
Overview What is a machine model? Why do we need a model? RAM PRAM ◦ Steps in computation ◦ Write conflict ◦ Examples

3
A parallel Machine Model What is a machine model? ◦ Describes a “machine” ◦ Puts a value to the operations on the machine Why do we need a model? ◦ Makes it easy to reason algorithms ◦ Achieve complexity bounds ◦ Analyzes maximum parallelism

4
RAM (Random Access Machine) Unbounded number of local memory cells Each memory cell can hold an integer of unbounded size Instruction set included –simple operations, data operations, comparator, branches All operations take unit time Time complexity = number of instructions executed Space complexity = number of memory cells used

5
PRAM (Parallel Random Access Machine) Definition: ◦ Is an abstract machine for designing the algorithms applicable to parallel computers ◦ M’ is a system of infinitely many RAM’s M1, M2, …, each M i is called a processor of M’. All the processors are assumed to be identical. Each has ability to recognize its own index i Input cells X(1), X(2),…, Output cells Y(1), Y(2),…, Shared memory cells A(1), A(2),…,

6
PRAM (Parallel RAM) Unbounded collection of RAM processors P 0, P 1, …, Processors don’t have tape Each processor has unbounded registers Unbounded collection of share memory cells All processors can access all memory cells in unit time All communication via shared memory

7
PRAM (step in a computation) Consist of 5 phases (carried in parallel by all the processors) each processor: ◦ Reads a value from one of the cells x(1),…, x(N) ◦ Reads one of the shared memory cells A(1), A(2),… ◦ Performs some internal computation ◦ May write into one of the output cells y(1), y(2),… ◦ May write into one of the shared memory cells A(1), A(2),… e.g. for all i, do A[i] = A[i-1] + 1; Read A[i-1], compute add 1, write A[i] happened synchronously

8
PRAM (Parallel RAM) Some subset of the processors can remain idle P0P0 P1P1 P2P2 PNPN Shared Memory Cells Two or more processors may read simultaneously from the same cell A write conflict occurs when two or more processors try to write simultaneously into the same cell

9
Share Memory Access Conflicts PRAM are classified based on their Read/Write abilities (realistic and useful) ◦ Exclusive Read(ER) : all processors can simultaneously read from distinct memory locations ◦ Exclusive Write(EW) : all processors can simultaneously write to distinct memory locations ◦ Concurrent Read(CR) : all processors can simultaneously read from any memory location ◦ Concurrent Write(CW) : all processors can write to any memory location ◦ EREW, CREW, CRCW

10
Concurrent Write (CW) What value gets written finally? ◦ Priority CW: processors have priority based on which value is decided, the highest priority is allowed to complete WRITE ◦ Common CW: all processors are allowed to complete WRITE iff all the values to be written are equal. ◦ Arbitrary/Random CW: one randomly chosen processor is allowed to complete WRITE

11
Strengths of PRAM PRAM is attractive and important model for designers of parallel algorithms Why? ◦ It is natural: the number of operations executed per one cycle on p processors is at most p ◦ It is strong: any processor can read/write any shared memory cell in unit time ◦ It is simple: it abstracts from any communication or synchronization overhead, which makes the complexity and correctness of PRAM algorithm easier ◦ It can be used as a benchmark: If a problem has no feasible/efficient solution on PRAM, it has no feasible/efficient solution for any parallel machine

12
Computational power Model A is computationally stronger than model B (A>=B) iff any algorithm written for B will run unchanged on A in the same parallel time and same basic properties. Priority >= Arbitrary >= Common >=CREW >= EREW MostLeast powerful LeastMost realistic

13
An initial example How do you add N numbers residing in memory location M[0, 1, …, N] Serial Algorithm = O(N) PRAM Algorithm using N processors P 0, P 1, P 2, …, P N ?

14
PRAM Algorithm (Parallel Addition) P 0 +P 1 +P 2 +P 3 + P 2 +P 0 + Step 3 Step 2 Step 1

15
PRAM Algorithm (Parallel Addition) Log (n) steps = time needed n / 2 processors needed Speed-up = n / log(n) Efficiency = 1 / log(n) Applicable for other operations ◦ +, *,, etc.

16
Example 2 p processor PRAM with n numbers (p<=n) Does x exist within the n numbers? P 0 contains x and finally P 0 has to know Algorithm ◦ Inform everyone what x is ◦ Every processor checks [n/p] numbers and sets a flag ◦ Check if any of the flags are set to 1

17
Example 2 EREWCREW CRCW (common) Inform everyone what x is log(p)11 Every processor checks [n/p] numbers and sets a flag n/p Check if any of the flag are set to 1 log(p) 1

18
Some variants of PRAM Bounded number of shared memory cells. Small memory PRAM (input data set exceeds capacity of the share memory i/o values can be distributed evenly among the processors) Bounded number of processor Small PRAM. If # of threads of execution is higher, processors may interleave several threads. Bounded size of a machine word. Word size of PRAM Handling access conflicts. Constraints on simultaneous access to share memory cells

19 Lemma Assume p’

20
Lemma Assume m’

21
Conclusions We need some model to reason, compare, analyze and design algorithms PRAM is simple and easy to understand Rich set of theoretical results Over-simplistic and often not realistic The programs written on these machines are, in general, of type MIMD. Certain special cases such as SIMD may also be handled in such a framework

22
Question Why is PRAM attractive and important model for designers of parallel algorithms ? ◦ It is natural: the number of operations executed per one cycle on p processors is at most p ◦ It is strong: any processor can read/write any shared memory cell in unit time ◦ It is simple: it abstracts from any communication or synchronization overhead, which makes the complexity and correctness of PRAM algorithm easier ◦ It can be used as a benchmark: If a problem has no feasible/efficient solution on PRAM, it has no feasible/efficient solution for any parallel machine

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google