1. CMPUT680 - Fall 2006 Topic A: Data Dependence in Loops José Nelson Amaral http://www.cs.ualberta.ca/~amaral/courses/680

2. CMPUT 680 - Compiler Design and Optimization2 Reading Wolfe, Michael, High Performance Compilers for Parallel Computing, Addison-Wesley, 1996 Chapter 5 Randy Allen, Ken Kennedy, Optimizing Compilers for Modern Architectures: A Dependence-based Approach, Morgan Kauffman, 200. Chapter 2.

3. CMPUT 680 - Compiler Design and Optimization3 Basic Concept and Motivation  A loop-carried data dependence occurs when a memory access in the iteration i of a loop cannot occur before an access in some iteration i-k is performed. zThere is data dependence between an access a at iteration i-k and an access b at iteration i if: ya and b access the same memory location yThere is a path from a to b yEither a or b is a write

4. CMPUT 680 - Compiler Design and Optimization4 Three Types of Data Dependence X = = X... Flow dependence = X X =... Anti-dependence X =... 0 Output dependence

5. CMPUT 680 - Compiler Design and Optimization5 Data Dependence Example 1: S1: A = 0 S2: B = A S3: A = B + 1 S4: C = A S1 S2 S3 S4  S 2 is flow dependent on S 1 S 1  f S 2 S 1  S 2 (Wolfe, pp. 138) S 1 is the source and S 2 is the target of the dependence.

6. CMPUT 680 - Compiler Design and Optimization6 Data Dependence S 2  S 3 : S 3 is flow-dependent on S 2 S 1  0 S 3 : S 3 is output-dependent on S 1 S 2  -1 S 3 : S 3 is anti-dependent on S 2 S1 S2 S3 S4 Example 1: S1: A = 0 S2: B = A S3: A = B + 1 S4: C = A

7. CMPUT 680 - Compiler Design and Optimization7 Parameterized Dependences DO I = 1, N S 1 A(I+1) = A(I) + B(I) ENDDO “Statement S 1 depends upon itself.” DO I = 1, N S 1 A(I+2) = A(I) + B(I) ENDDO “Statement S 1 depends on an instance of itself two iterations previous.” We need to be able to describe such dependences formally. (Allen-Kennedy, pp. 39)

8. CMPUT 680 - Compiler Design and Optimization8 Loop Normalization DO I = L, U STEP S …. ENDDO Given a loop of the form: The normalized value of an iteration k can be obtained from: Normalized(k) = (k-L+S)/S DO I = 5, 26 STEP 3 …. ENDDO Example 5 8 11 14 … 26 Iteration Space 1 2 3 4 … 8 Normalized Iteration Space (Allen-Kennedy, pp. 39)

9. CMPUT 680 - Compiler Design and Optimization9 Data Dependences Loop carried: between two statements instances in two different iterations of a loop. Loop independent: between two statements instances in the same loop iteration. Lexically forward: the source comes before the target. Lexically backward: otherwise. The right-hand side of an assignment is considered to precede the left-hand side.

10. CMPUT 680 - Compiler Design and Optimization10 Review of Linear Algebra Lexicographic Order Two n-vectors a and b are equal, a = b, if a i = b i, 1  i  n. We say that a is less than b, a<b, if a i <b i, 1  i  n. We say that a is lexicographically less than b, at level j, a « j b, if a i = b i, 1  i < j and a j <b j. We say that a is lexicographically less than b, a « b, if there is a j, 1  j  n, such that a « j b. (Wolfe, pp. 86)

11. CMPUT 680 - Compiler Design and Optimization11 Lexicographic Order Example of vectors

12. CMPUT 680 - Compiler Design and Optimization12 Properties of Lexicographic Order Let n  1, and i, j, k denote arbitrary vectors in R n 1 For each u in 1  u  n, the relation « u in R n is irreflexive and transitive. 2 The n relations « u are pairwise disjoint: i « u j and i « v j imply that u = v. 3 If i  j, there is a unique integer u such that 1  u  n and exactly one of the following two conditions holds: i « u j or j « u i. 4 i « u j and j « v k together imply that i « w k, where w = min (u,v).

13. CMPUT 680 - Compiler Design and Optimization13 Data Dependence in Loops An Example Find the dependence relations due to the array X in the program below: (S 1 ) for i = 2 to 9 do (S 2 ) X[i] = Y[i] + Z[i] (S 3 ) A[i] = X[i-1] + 1 (S 4 )end for Solution To find the data dependence relations in a simple loop, we can unroll the loop and see which statement instances depend on which others: i = 2i = 3i = 4 (s2) X[2]=Y[2]+Z[2] X[3] =Y[3]+Z[3] X[4]=Y[4]+Z[4] (s3) A[2]=X[1]+1 A[3] =X[2]+1 A[4]=X[3]+1 (Wolfe, pp. 140)

14. CMPUT 680 - Compiler Design and Optimization14 There is a loop-carried, lexically forward, flow dependence from S 2 to S 3. Data Dependence in Loops S2S2 S3S3 (1,3) Data dependence graph for statements in a loop (1,3) := iteration distance is 1, latency is 3. (S 1 ) for i = 2 to 9 do (S 2 ) X[i] = Y[i] + Z[i] (S 3 ) A[i] = X[i-1] + 1 (S 4 )end for i = 2i = 3i = 4 (s2) X[2]=Y[2]+Z[2] X[3]=Y[3]+Z[3] X[4]=Y[4]+Z[4] (s3) A[2]=X[1]+1 A[3]=X[2]+1 A[4]=X[3]+1

15. CMPUT 680 - Compiler Design and Optimization15 zIteration space and iteration-space- dependence-graph Example Show the iteration space dependence graph for the loop in our example. Solution 0 1 2 3 4 5 6 7 8 9 Iteration space dependence graph We need an abstraction for this. Iteration Space (an informal introduction)

16. CMPUT 680 - Compiler Design and Optimization16 Iteration Space (an informal introduction) (S 1 ) for i = 3 to 9 do (S 2 ) X[i] = Y[i] + Z[i] (S 3 ) A[i] = X[i-2] + 1 (S 4 ) B[i] = A[i-1] + 2 (S 5 )end for i 2 S (X) = [3; 4; 5; 6; 7; 8; 9] i 3 T (X) = [1; 2; 3; 4; 5; 6; 7] i 3 S (A) = [3; 4; 5; 6; 7; 8; 9] i 4 T (A) = [2; 3; 4; 5; 6; 7; 8] For each dependency, there is an iteration vector for the source and one for the target Iteration Vector: a vector formed by the index variable used to access an array in the loop. S2S2 S3S3 S4S4

17. CMPUT 680 - Compiler Design and Optimization17 d(X) = i 3 T (X) - i 2 S (X) d(X) = [-2; -2; -2; -2; -2; -2; -2] d(A) = i 4 T (A) - i 3 S (A) d(A) = [-1; -1; -1; -1; -1; -1; -1] i 2 S (X) = [3; 4; 5; 6; 7; 8; 9] i 3 T (X) = [1; 2; 3; 4; 5; 6; 7] i 3 S (A) = [3; 4; 5; 6; 7; 8; 9] i 4 T (A) = [2; 3; 4; 5; 6; 7; 8] Distance Vector: a vector formed by the difference between the iteration vectors of the source and target of a dependency. (S 1 ) for i = 3 to 9 do (S 2 ) X[i] = Y[i] + Z[i] (S 3 ) A[i] = X[i-2] + 1 (S 4 ) B[i] = A[i-1] + 2 (S 5 )end for Iteration Space (an informal introduction) S2S2 S3S3 S4S4

18. CMPUT 680 - Compiler Design and Optimization18 dir(X) = [<;<;<;<;<;<;<] dir(A) = [<;<;<;<;<;<;<] The elements of a direction vector are, and =. Other authors use +, -, 0. (S 1 ) for i = 3 to 9 do (S 2 ) X[i] = Y[i] + Z[i] (S 3 ) A[i] = X[i-2] + 1 (S 4 ) B[i] = A[i-1] + 2 (S 5 )end for Direction Vector: contain only information about the direction of the dependence but no iteration distance information. Iteration Space (an informal introduction) S2S2 S3S3 S4S4

19. CMPUT 680 - Compiler Design and Optimization19 zEach element of the direction vector can be stored in two bits. zGiven a distance vector, we can compute the direction vector, but not vice-versa. Iteration Space (an informal introduction)

20. CMPUT 680 - Compiler Design and Optimization20 Example Show the index variable iteration vectors and normalized iteration vectors for the iterations in the loop below: (1)for i = 2 to 6 do (2) for j = 6 to 2 by -2 do (3) A[i, j] = A[i, j+2] +1 (4) end for (5)end for Solution Since there are two nested loops, the iteration space has two dimensions. Iteration Space (an informal introduction)

21. CMPUT 680 - Compiler Design and Optimization21 i Iteration space dependence graph corresponding to the index variable iteration vectors. 26345 j 6 2 4 Iteration Space (an informal introduction) (1)for i = 2 to 6 do (2) for j = 6 to 2 by -2 do (3) A[i, j] = A[i, j+2] +1 (4) end for (5)end for

22. CMPUT 680 - Compiler Design and Optimization22 Distance/Direction Vectors zIt is often convenient to deal with incompletely specified direction vectors Example 1: {(0, 0, 0, 1), (0, -1, 0, 1), (0, 0, 1, 1), (0, -1, 1, 1)} ==> {(0,  0,  0, 1)} Example 2: {(0, -1, 0, -1), (0, 0, 0, -1), (0, 1, 0, -1)} ==> {(0, *, 0, -1)}

23. CMPUT 680 - Compiler Design and Optimization23 Distance/Direction Vectors zLet a, b denote two vectors in R n and s their direction vector. Then a « b if and only if s has one of the following forms: (1, *, *, …, *) (0, 1, *, …, *) (0, 0, 1, *, …, *) (0, 0, …, 0, 1). More precisely, a « u b for u in 1  u  n, if and only if s has the form with a leading 1 after (u - 1) zeros. zNotation (0, 1, -1)  (=, >, <)

24. CMPUT 680 - Compiler Design and Optimization24 do i = 3, 100 S:A[2i] = B[i] + 2 T:C[i] = D[i] + 2  A[2i+1] + A[2i - 4] + A[i] done What are the dependences and the dependence distance vectors in the example above? An Example

25. CMPUT 680 - Compiler Design and Optimization25 do i = 3, 100 S:A[2i] = B[i] + 2 T1:TEMP1 = D[i] + 2  A[2i + 1] T2: TEMP2 = TEMP1 + A[2i - 4] T3: C(i) = TEMP2+ A[i] done i S (A) = [6; 8; 10; 12; 14; 16; …; 198; 200] i T1 (A) = [7; 9; 11; 13; 15; 17; …; 199; 201] i T2 (A) = [2; 4; 6; 8; 10; 12; …; 194; 196] i T3 (A) = [3; 4; 5; 6; 7; 8; …; 99; 100] An Example

26. CMPUT 680 - Compiler Design and Optimization26 An Example i S (A) = [6; 8; 10; 12; 14; 16; …; 198; 200] i T1 (A) = [7; 9; 11; 13; 15; 17; …; 199; 201] T1 is flow dependent on S with dependence distance 1. d(T1,S) = i T1 (A) - i S (A) do i = 3, 100 S:A[2i] = B[i] + 2 T1:TEMP1 = D[i] + 2  A[2i + 1] T2: TEMP2 = TEMP1 + A[2i - 4] T3: C(i) = TEMP2+ A[i] done

27. CMPUT 680 - Compiler Design and Optimization27 i S (A) = [6; 8; 10; 12; 14; 16; …; 198; 200] i T2 (A) = [2; 4; 6; 8; 10; 12; …; 194; 196] d(T2,S) = i T2 (A) - i S (A) T2 is flow dependent on S with dependence distance -4. do i = 3, 100 S:A[2i] = B[i] + 2 T1:TEMP1 = D[i] + 2  A[2i + 1] T2: TEMP2 = TEMP1 + A[2i - 4] T3: C(i) = TEMP2+ A[i] done An Example

28. CMPUT 680 - Compiler Design and Optimization28 i S (A) = [6; 8; 10; 12; 14; 16; …; 198; 200] i T3 (A) = [3; 4; 5; 6; 7; 8; …; 99; 100] d(T3,S) = i T3 (A) - i S (A) T3 is flow dependent on S with dependence distance (i-2i) = -i do i = 3, 100 S:A[2i] = B[i] + 2 T1:TEMP1 = D[i] + 2  A[2i + 1] T2: TEMP2 = TEMP1 + A[2i - 4] T3: C(i) = TEMP2+ A[i] done An Example

29. CMPUT 680 - Compiler Design and Optimization29 Wolfe’s Definition From Michael Wolfe’s, pg. 140: “An anti-dependence from a statement to itself is considered lexically forward”: S k : x[i]  x[i+1] + 1 “A dependence is lexically forward when the source comes before the target without passing through a loop back edge”: x[1]  x[2] + 1 x[2]  x[3] + 1 x[3]  x[4] + 1 (back edge)

30. CMPUT 680 - Compiler Design and Optimization30 Wolfe’s Definition From Michael Wolfe’s, pg. 140: “A self-flow dependence is lexically backward”: S k : x[i]  x[i-1] + 1 x[1]  x[0] + 1 x[2]  x[1] + 1 x[3]  x[2] + 1 (back edge)

31. CMPUT 680 - Compiler Design and Optimization31 Allen-Kennedy Definition From Allen-Kennedy’s, pg. 45: “Suppose that there is a dependence from statement S 1 on iteration i of a loop nest of n loops and statement S 2 on iteration j; then the dependence distance vector d(i,j) is defined as a vector of length n such that:

32. CMPUT 680 - Compiler Design and Optimization32 Allen-Kennedy Definition From Allen-Kennedy’s, pg. 46: “Suppose that there is a dependence from statement S 1 on iteration i of a loop nest of n loops and statement S 2 on iteration j; then the dependence direction vector D(i,j) is defined as a vector of length n such that:

33. CMPUT 680 - Compiler Design and Optimization33 Allen-Kennedy Definition From Allen-Kennedy’s, pg. 50: “Statement S 2 has a loop-carried dependence on statement S 1 if and only if S 1 references location M on iteration j, and d(i,j) > 0 (that is, D(i,j) contains a “<“ as its leftmost non-”=“ component).” “A loop-carried dependence from statement S 1 to statement S 2 is said to be backward if S 2 appears before S 1 in the loop body or if S 1 and S 2 are the same statement. The carried dependence is said to be forward if S 2 appears after S 1 in the loop body.