Download presentation

1
**CS 201 Compiler Construction**

Array Dependence Analysis & Loop Parallelization

2
Loop Parallelization Goal: Identify loops whose iterations can be executed in parallel on different processors of a shared-memory multiprocessor system. Matrix Multiplication for I = 1 to n do -- parallel for J = 1 to n do -- parallel for K = 1 to n do –- not parallel C[I,J] = C[I,J] + A[I,K]*B[K,J]

3
Data Dependences Flow Dependence: S1: X = …. S2: … = X Anti Dependence: S1: … = X S2: X = … Output Dependence: S1: X = … S1 δf S2 S1 δa S2 S1 δo S2 S1 S2 δf S1 S2 δa S1 S2 δo

4
**Example: Data Dependences**

do I = 1, 40 S1: A(I+1) = …. S2: … = A(I-1) enddo S1: A(I-1) = … S2: … = A(I+1) S1: A(I+1) = … S2: A(I-1) = … S1 S2 δf S2 S1 δa S1 S2 δo

5
**Sets of Dependences δf Due to A()**

do I = 1, 100 S: A(I) = B(I+2) + 1 T: B(I) = A(I-1) - 1 enddo S T δf Due to A() S(1): A(1) = B(3) + 1 T(1): B(1) = A(0) - 1 S(2): A(2) = B(4) + 1 T(2): B(2) = A(1) - 1 S(3): A(3) = B(5) + 1 T(3): B(3) = A(2) - 1 ………….. S(100): A(100) = B(102) + 1 T(100): B(100) = A(99) - 1 Set of iteration pairs associated with this dependence: {(i,j): j=i+1, 1<=i<=99} Dependence distance: j-i=1 constant in this case.

6
Nested Loops level 1: do I1 = 1, 100 level 2: do I2 = 1, 50 S: A(I1,I2) = A(I1,I2-1) + B(I1,I2) enddo Value computed by S in an iteration (i1,i2) is same as the value used in an iteration (j1,j2): A(i1,i2)A(j1,j2-1) iff i1=j1 and i2=j2-1 S is flow dependent on itself at level 2 (corresponds to inner loop) for fixed value of I1 dependence exists between different iterations of second loop (inner loop). iteration pairs: {((i1,i2),(j1,j2)): j1=i1, j2=i2+1, 1<=i1<=100, 1<=i2<=49}

7
**Nested Loops Contd.. {((i1,i2),(j1,j2)): j1=i1, j2=i2+1,**

Iteration pairs: {((i1,i2),(j1,j2)): j1=i1, j2=i2+1, 1<=i1<=100, 1<=i2<=49} Dependence distance vector: (j1-i1,j2-i2) = (0,1) There is no dependence at level 1.

8
**Computing Dependences - Formulation**

level 1: do I1 = 1, 8 level 2: do I2 = max(I1-3,1), min(I1,5) A(I1+1,I2+1) = A(I1,I2) + B(I1,I2) enddo Can we find iterations (i1,i2) & (j1,j2) such that i1+1=j1; i2+1=j2 and 1<=i1<=8 1<=j1<=8 i1-3<=i2<=i1 j1-3<=j2<=j1 i1-3<=i2<=5 j1-3<=j2<=5 1<=i2<=i1 1<=j2<=j1 1<=i2<=5 1<=j2<=5 i1, i2, j1, j2 are integers.

9
**Computing Dependences Contd..**

Dependence testing is an integer programming problem NP-complete Solutions trade-off Speed and Accuracy of the solver. False positives: imprecise tests may report dependences that actually do not exist – conservative is to report false positives but never miss a dependence. DependenceTests: extreme value test; GCD test; Generalized GCD test; Lambda test; Delta test; Power test; Omega test etc…

10
Extreme Value Test Approximate test which guarantees that a solution exists but it may be a real solution not an integer solution. Let f: Rn R st f is bounded in a set S contained in Rn Let b be a lower bound of f in S and B be an upper bound of f in S: b ≤ f(ā) ≤ B for any ā ε S contained in Rn For a real number c, the equation f(ā) = c will have solutions iff b ≤ c ≤ B.

11
**Extreme Value Test Contd..**

Example: f: R2 R; f(x,y) = 2x + 3y S = {(x,y): 0<=x<=1 & 0<=y<=1} contained in R2 lower bound, b=0; upper bound, B=5 1. Is there a real solution to the equation 2x + 3y = 4 ? Yes. (0.5,1) is a solution, there are many others. 2. Is there an integer solution in S ? No. f(0,0)=0; f(0,1)=3; f(1,0)=2; & f(1,1)=5. For none of the integer points in S, f(x,y)=4. If there are no real solutions, there are no integer solutions. If there are real solutions, then integer solutions may or may not exist.

12
**Extreme Value Test Contd..**

Example: DO I = 1, 10 DO J = 1, 10 A[10*I+J-5] = ….A[10*I+J-10]…. 10*I1+J1-5 = 10*I2+J2-10 10*I1-10*I2+J1-J2 = -5 f: R4 R; f(I1,I2,J1,J2) = 10*I1-10*I2+J1-J2 1<=I1,I2,J1,J2<=10; lower bound, b=-99; upper bound, B=+99 since -99 <= -5 <= +99 there is a dependence.

13
**Extreme Value Test Contd..**

14
**Extreme Value Test Contd..**

15
**Extreme Value Test Contd..**

16
**Extreme Value Test Contd..**

17
**Nested Loops – Multidimensional Arrays**

18
Nested Loops Contd..

Similar presentations

OK

Inter-Iteration Scalar Replacement in the Presence of Control-Flow Mihai Budiu – Microsoft Research, Silicon Valley Seth Copen Goldstein – Carnegie Mellon.

Inter-Iteration Scalar Replacement in the Presence of Control-Flow Mihai Budiu – Microsoft Research, Silicon Valley Seth Copen Goldstein – Carnegie Mellon.

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on producers consumers and decomposers clip Ppt on db2 introduction to algebra Ppt on strategic brand management process Ppt on popular social networking sites Ppt on council of ministers of ethiopia Ppt on the history of space flight Ppt on south indian cuisine Ppt on field study 3 Ppt on synthesis and degradation of purines and pyrimidines Ppt on power line carrier communication ppt